Sep 09 20:40:37.220213 ip-10-0-129-99 systemd[1]: Starting Kubernetes Kubelet...
Sep 09 20:40:37.602718 ip-10-0-129-99 kubenswrapper[2581]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 09 20:40:37.602718 ip-10-0-129-99 kubenswrapper[2581]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 09 20:40:37.602718 ip-10-0-129-99 kubenswrapper[2581]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 09 20:40:37.602718 ip-10-0-129-99 kubenswrapper[2581]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 09 20:40:37.602718 ip-10-0-129-99 kubenswrapper[2581]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 09 20:40:37.603386 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.602737    2581 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 09 20:40:37.608411 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608392    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 20:40:37.608411 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608410    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 20:40:37.608483 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608414    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 20:40:37.608483 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608417    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 20:40:37.608483 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608421    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 20:40:37.608483 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608424    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 20:40:37.608483 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608428    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 20:40:37.608483 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608432    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 20:40:37.608483 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608434    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 20:40:37.608483 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608437    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 20:40:37.608483 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608440    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 20:40:37.608483 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608443    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 20:40:37.608483 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608446    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 20:40:37.608483 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608449    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 20:40:37.608483 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608452    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 20:40:37.608483 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608455    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 20:40:37.608483 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608458    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 20:40:37.608483 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608461    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 20:40:37.608483 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608464    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 20:40:37.608483 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608467    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 20:40:37.608483 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608470    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 20:40:37.608483 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608472    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 20:40:37.609033 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608475    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 20:40:37.609033 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608478    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 20:40:37.609033 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608480    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 20:40:37.609033 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608483    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 20:40:37.609033 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608486    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 20:40:37.609033 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608489    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 20:40:37.609033 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608492    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 20:40:37.609033 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608496    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 20:40:37.609033 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608498    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 20:40:37.609033 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608501    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 20:40:37.609033 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608504    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 20:40:37.609033 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608506    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 20:40:37.609033 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608509    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 20:40:37.609033 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608511    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 20:40:37.609033 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608515    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 20:40:37.609033 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608517    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 20:40:37.609033 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608520    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 20:40:37.609033 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608522    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 20:40:37.609033 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608525    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 20:40:37.609033 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608527    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 20:40:37.609525 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608530    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 20:40:37.609525 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608533    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 20:40:37.609525 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608535    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 20:40:37.609525 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608538    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 20:40:37.609525 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608540    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 20:40:37.609525 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608543    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 20:40:37.609525 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608546    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 20:40:37.609525 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608548    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 20:40:37.609525 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608553    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 20:40:37.609525 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608557    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 20:40:37.609525 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608560    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 20:40:37.609525 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608563    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 20:40:37.609525 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608565    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 20:40:37.609525 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608569    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 20:40:37.609525 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608572    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 20:40:37.609525 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608575    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 20:40:37.609525 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608578    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 20:40:37.609525 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608581    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 20:40:37.609525 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608583    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 20:40:37.610013 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608586    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 20:40:37.610013 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608591    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 20:40:37.610013 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608595    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 20:40:37.610013 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608598    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 20:40:37.610013 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608601    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 20:40:37.610013 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608604    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 20:40:37.610013 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608607    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 20:40:37.610013 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608610    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 20:40:37.610013 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608613    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 20:40:37.610013 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608617    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 20:40:37.610013 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608620    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 20:40:37.610013 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608623    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 20:40:37.610013 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608626    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 20:40:37.610013 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608628    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 20:40:37.610013 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608631    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 20:40:37.610013 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608634    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 20:40:37.610013 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608638    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 20:40:37.610013 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608641    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 20:40:37.610013 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608644    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 20:40:37.610480 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608647    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 20:40:37.610480 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608651    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 20:40:37.610480 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608653    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 20:40:37.610480 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608657    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 20:40:37.610480 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608660    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 20:40:37.610480 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608663    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 20:40:37.610480 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608666    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 20:40:37.610480 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608668    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 20:40:37.610480 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608671    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 20:40:37.610480 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608674    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 20:40:37.610480 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608676    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 20:40:37.610480 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608679    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 20:40:37.610480 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608681    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 20:40:37.610480 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608684    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 20:40:37.610480 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608687    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 20:40:37.610480 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608690    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 20:40:37.610480 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608692    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 20:40:37.610480 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608695    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 20:40:37.610480 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.608697    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 20:40:37.610480 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609444    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 20:40:37.610982 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609451    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 20:40:37.610982 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609455    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 20:40:37.610982 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609457    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 20:40:37.610982 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609460    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 20:40:37.610982 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609463    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 20:40:37.610982 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609466    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 20:40:37.610982 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609469    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 20:40:37.610982 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609472    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 20:40:37.610982 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609474    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 20:40:37.610982 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609477    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 20:40:37.610982 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609480    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 20:40:37.610982 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609482    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 20:40:37.610982 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609485    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 20:40:37.610982 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609488    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 20:40:37.610982 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609490    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 20:40:37.610982 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609494    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 20:40:37.610982 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609496    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 20:40:37.610982 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609499    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 20:40:37.610982 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609502    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 20:40:37.610982 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609505    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 20:40:37.611521 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609508    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 20:40:37.611521 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609510    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 20:40:37.611521 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609513    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 20:40:37.611521 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609516    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 20:40:37.611521 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609519    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 20:40:37.611521 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609524    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 20:40:37.611521 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609528    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 20:40:37.611521 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609531    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 20:40:37.611521 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609534    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 20:40:37.611521 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609538    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 20:40:37.611521 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609541    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 20:40:37.611521 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609544    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 20:40:37.611521 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609547    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 20:40:37.611521 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609550    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 20:40:37.611521 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609552    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 20:40:37.611521 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609555    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 20:40:37.611521 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609557    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 20:40:37.611521 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609561    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 20:40:37.611521 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609564    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 20:40:37.612100 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609566    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 20:40:37.612100 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609569    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 20:40:37.612100 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609572    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 20:40:37.612100 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609575    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 20:40:37.612100 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609577    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 20:40:37.612100 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609580    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 20:40:37.612100 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609582    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 20:40:37.612100 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609585    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 20:40:37.612100 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609588    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 20:40:37.612100 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609591    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 20:40:37.612100 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609593    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 20:40:37.612100 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609596    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 20:40:37.612100 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609599    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 20:40:37.612100 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609601    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 20:40:37.612100 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609604    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 20:40:37.612100 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609606    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 20:40:37.612100 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609609    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 20:40:37.612100 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609612    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 20:40:37.612100 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609615    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 20:40:37.612100 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609618    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 20:40:37.612591 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609620    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 20:40:37.612591 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609623    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 20:40:37.612591 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609625    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 20:40:37.612591 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609628    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 20:40:37.612591 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609630    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 20:40:37.612591 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609633    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 20:40:37.612591 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609635    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 20:40:37.612591 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609638    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 20:40:37.612591 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609641    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 20:40:37.612591 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609643    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 20:40:37.612591 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609646    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 20:40:37.612591 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609649    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 20:40:37.612591 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609651    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 20:40:37.612591 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609657    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 20:40:37.612591 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609659    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 20:40:37.612591 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609662    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 20:40:37.612591 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609664    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 20:40:37.612591 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609669    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 20:40:37.612591 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609673    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 20:40:37.612591 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609676    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 20:40:37.613108 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609680    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 20:40:37.613108 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609682    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 20:40:37.613108 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609685    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 20:40:37.613108 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609688    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 20:40:37.613108 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609691    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 20:40:37.613108 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609694    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 20:40:37.613108 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609696    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 20:40:37.613108 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609699    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 20:40:37.613108 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609702    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 20:40:37.613108 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609705    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 20:40:37.613108 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609708    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 20:40:37.613108 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609710    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 20:40:37.613108 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609713    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 20:40:37.613108 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609717    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 20:40:37.613108 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609720    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 20:40:37.613108 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609723    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 20:40:37.613108 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609726    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 20:40:37.613108 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609729    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 20:40:37.613108 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.609732    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 20:40:37.613596 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610153    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 20:40:37.613596 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610160    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 20:40:37.613596 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610164    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 20:40:37.613596 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610167    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 20:40:37.613596 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610170    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 20:40:37.613596 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610173    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 20:40:37.613596 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610178    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 20:40:37.613596 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610181    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 20:40:37.613596 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610184    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 20:40:37.613596 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610186    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 20:40:37.613596 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610189    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 20:40:37.613596 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610192    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 20:40:37.613596 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610195    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 20:40:37.613596 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610198    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 20:40:37.613596 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610200    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 20:40:37.613596 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610203    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 20:40:37.613596 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610206    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 20:40:37.613596 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610209    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 20:40:37.613596 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610212    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 20:40:37.614085 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610215    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 20:40:37.614085 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610217    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 20:40:37.614085 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610220    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 20:40:37.614085 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610223    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 20:40:37.614085 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610226    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 20:40:37.614085 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610228    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 20:40:37.614085 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610231    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 20:40:37.614085 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610234    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 20:40:37.614085 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610237    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 20:40:37.614085 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610239    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 20:40:37.614085 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610242    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 20:40:37.614085 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610244    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 20:40:37.614085 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610247    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 20:40:37.614085 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610249    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 20:40:37.614085 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610252    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 20:40:37.614085 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610255    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 20:40:37.614085 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610257    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 20:40:37.614085 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610260    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 20:40:37.614085 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610263    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 20:40:37.614549 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610267    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 20:40:37.614549 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610270    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 20:40:37.614549 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610273    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 20:40:37.614549 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610275    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 20:40:37.614549 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610278    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 20:40:37.614549 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610280    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 20:40:37.614549 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610283    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 20:40:37.614549 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610286    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 20:40:37.614549 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610289    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 20:40:37.614549 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610292    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 20:40:37.614549 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610295    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 20:40:37.614549 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610297    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 20:40:37.614549 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610300    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 20:40:37.614549 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610303    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 20:40:37.614549 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610305    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 20:40:37.614549 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610308    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 20:40:37.614549 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610310    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 20:40:37.614549 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610313    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 20:40:37.614549 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610318    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 20:40:37.615018 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610321    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 20:40:37.615018 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610324    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 20:40:37.615018 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610327    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 20:40:37.615018 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610329    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 20:40:37.615018 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610332    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 20:40:37.615018 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610335    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 20:40:37.615018 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610338    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 20:40:37.615018 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610340    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 20:40:37.615018 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610343    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 20:40:37.615018 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610345    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 20:40:37.615018 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610348    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 20:40:37.615018 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610350    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 20:40:37.615018 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610353    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 20:40:37.615018 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610357    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 20:40:37.615018 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610360    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 20:40:37.615018 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610362    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 20:40:37.615018 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610365    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 20:40:37.615018 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610368    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 20:40:37.615018 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610370    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 20:40:37.615018 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610374    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 20:40:37.615509 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610377    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 20:40:37.615509 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610380    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 20:40:37.615509 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610383    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 20:40:37.615509 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610385    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 20:40:37.615509 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610388    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 20:40:37.615509 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610392    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 20:40:37.615509 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610395    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 20:40:37.615509 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610398    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 20:40:37.615509 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610400    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 20:40:37.615509 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610403    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 20:40:37.615509 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610405    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 20:40:37.615509 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610408    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 20:40:37.615509 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610411    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 20:40:37.615509 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610413    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 20:40:37.615509 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610416    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 20:40:37.615509 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610418    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 20:40:37.615509 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610421    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 20:40:37.615509 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610424    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 20:40:37.615509 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610426    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 20:40:37.615509 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610429    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 20:40:37.616019 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610431    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 20:40:37.616019 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.610434    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 20:40:37.616019 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610514    2581 flags.go:64] FLAG: --address="0.0.0.0"
Sep 09 20:40:37.616019 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610547    2581 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 09 20:40:37.616019 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610555    2581 flags.go:64] FLAG: --anonymous-auth="true"
Sep 09 20:40:37.616019 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610560    2581 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 09 20:40:37.616019 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610565    2581 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 09 20:40:37.616019 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610569    2581 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 09 20:40:37.616019 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610573    2581 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 09 20:40:37.616019 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610577    2581 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 09 20:40:37.616019 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610581    2581 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 09 20:40:37.616019 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610584    2581 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 09 20:40:37.616019 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610588    2581 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 09 20:40:37.616019 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610591    2581 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 09 20:40:37.616019 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610594    2581 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 09 20:40:37.616019 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610597    2581 flags.go:64] FLAG: --cgroup-root=""
Sep 09 20:40:37.616019 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610600    2581 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 09 20:40:37.616019 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610603    2581 flags.go:64] FLAG: --client-ca-file=""
Sep 09 20:40:37.616019 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610606    2581 flags.go:64] FLAG: --cloud-provider="external"
Sep 09 20:40:37.616019 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610609    2581 flags.go:64] FLAG: --cluster-dns="[]"
Sep 09 20:40:37.616019 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610613    2581 flags.go:64] FLAG: --cluster-domain=""
Sep 09 20:40:37.616019 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610616    2581 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 09 20:40:37.616019 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610619    2581 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 09 20:40:37.616019 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610622    2581 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 09 20:40:37.616599 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610626    2581 flags.go:64] FLAG: --container-log-max-files="5"
Sep 09 20:40:37.616599 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610629    2581 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 09 20:40:37.616599 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610632    2581 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 09 20:40:37.616599 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610635    2581 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 09 20:40:37.616599 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610638    2581 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 09 20:40:37.616599 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610641    2581 flags.go:64] FLAG: --contention-profiling="false"
Sep 09 20:40:37.616599 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610644    2581 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 09 20:40:37.616599 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610647    2581 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 09 20:40:37.616599 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610650    2581 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 09 20:40:37.616599 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610653    2581 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 09 20:40:37.616599 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610657    2581 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 09 20:40:37.616599 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610660    2581 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 09 20:40:37.616599 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610664    2581 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 09 20:40:37.616599 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610667    2581 flags.go:64] FLAG: --enable-load-reader="false"
Sep 09 20:40:37.616599 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610670    2581 flags.go:64] FLAG: --enable-server="true"
Sep 09 20:40:37.616599 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610676    2581 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 09 20:40:37.616599 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610681    2581 flags.go:64] FLAG: --event-burst="100"
Sep 09 20:40:37.616599 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610685    2581 flags.go:64] FLAG: --event-qps="50"
Sep 09 20:40:37.616599 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610688    2581 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 09 20:40:37.616599 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610691    2581 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 09 20:40:37.616599 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610694    2581 flags.go:64] FLAG: --eviction-hard=""
Sep 09 20:40:37.616599 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610698    2581 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 09 20:40:37.616599 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610701    2581 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 09 20:40:37.616599 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610704    2581 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 09 20:40:37.616599 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610707    2581 flags.go:64] FLAG: --eviction-soft=""
Sep 09 20:40:37.617275 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610710    2581 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 09 20:40:37.617275 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610713    2581 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 09 20:40:37.617275 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610716    2581 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 09 20:40:37.617275 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610719    2581 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 09 20:40:37.617275 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610722    2581 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 09 20:40:37.617275 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610725    2581 flags.go:64] FLAG: --fail-swap-on="true"
Sep 09 20:40:37.617275 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610727    2581 flags.go:64] FLAG: --feature-gates=""
Sep 09 20:40:37.617275 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610732    2581 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 09 20:40:37.617275 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610735    2581 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 09 20:40:37.617275 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610738    2581 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 09 20:40:37.617275 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610742    2581 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 09 20:40:37.617275 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610745    2581 flags.go:64] FLAG: --healthz-port="10248"
Sep 09 20:40:37.617275 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610748    2581 flags.go:64] FLAG: --help="false"
Sep 09 20:40:37.617275 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610750    2581 flags.go:64] FLAG: --hostname-override="ip-10-0-129-99.ec2.internal"
Sep 09 20:40:37.617275 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610753    2581 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 09 20:40:37.617275 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610756    2581 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 09 20:40:37.617275 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610759    2581 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 09 20:40:37.617275 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610763    2581 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 09 20:40:37.617275 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610766    2581 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 09 20:40:37.617275 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610769    2581 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 09 20:40:37.617275 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610772    2581 flags.go:64] FLAG: --image-service-endpoint=""
Sep 09 20:40:37.617275 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610775    2581 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 09 20:40:37.617275 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610778    2581 flags.go:64] FLAG: --kube-api-burst="100"
Sep 09 20:40:37.617928 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610782    2581 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 09 20:40:37.617928 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610786    2581 flags.go:64] FLAG: --kube-api-qps="50"
Sep 09 20:40:37.617928 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610789    2581 flags.go:64] FLAG: --kube-reserved=""
Sep 09 20:40:37.617928 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610792    2581 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 09 20:40:37.617928 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610795    2581 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 09 20:40:37.617928 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610798    2581 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 09 20:40:37.617928 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610801    2581 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 09 20:40:37.617928 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610804    2581 flags.go:64] FLAG: --lock-file=""
Sep 09 20:40:37.617928 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610807    2581 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 09 20:40:37.617928 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610810    2581 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 09 20:40:37.617928 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610813    2581 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 09 20:40:37.617928 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610818    2581 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 09 20:40:37.617928 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610821    2581 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 09 20:40:37.617928 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610825    2581 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 09 20:40:37.617928 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610828    2581 flags.go:64] FLAG: --logging-format="text"
Sep 09 20:40:37.617928 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610830    2581 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 09 20:40:37.617928 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610834    2581 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 09 20:40:37.617928 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610837    2581 flags.go:64] FLAG: --manifest-url=""
Sep 09 20:40:37.617928 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610840    2581 flags.go:64] FLAG: --manifest-url-header=""
Sep 09 20:40:37.617928 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610845    2581 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 09 20:40:37.617928 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610861    2581 flags.go:64] FLAG: --max-open-files="1000000"
Sep 09 20:40:37.617928 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610865    2581 flags.go:64] FLAG: --max-pods="110"
Sep 09 20:40:37.617928 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610868    2581 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 09 20:40:37.617928 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610872    2581 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 09 20:40:37.617928 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610875    2581 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 09 20:40:37.618546 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610878    2581 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 09 20:40:37.618546 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610881    2581 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 09 20:40:37.618546 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610884    2581 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 09 20:40:37.618546 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610887    2581 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 09 20:40:37.618546 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610894    2581 flags.go:64] FLAG: --node-status-max-images="50"
Sep 09 20:40:37.618546 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610897    2581 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 09 20:40:37.618546 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610900    2581 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 09 20:40:37.618546 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610903    2581 flags.go:64] FLAG: --pod-cidr=""
Sep 09 20:40:37.618546 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610907    2581 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 09 20:40:37.618546 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610912    2581 flags.go:64] FLAG: --pod-manifest-path=""
Sep 09 20:40:37.618546 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610915    2581 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 09 20:40:37.618546 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610919    2581 flags.go:64] FLAG: --pods-per-core="0"
Sep 09 20:40:37.618546 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610922    2581 flags.go:64] FLAG: --port="10250"
Sep 09 20:40:37.618546 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610924    2581 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 09 20:40:37.618546 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610928    2581 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-020e15e5ac2f5148e"
Sep 09 20:40:37.618546 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610931    2581 flags.go:64] FLAG: --qos-reserved=""
Sep 09 20:40:37.618546 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610934    2581 flags.go:64] FLAG: --read-only-port="10255"
Sep 09 20:40:37.618546 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610937    2581 flags.go:64] FLAG: --register-node="true"
Sep 09 20:40:37.618546 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610940    2581 flags.go:64] FLAG: --register-with-taints=""
Sep 09 20:40:37.618546 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610943    2581 flags.go:64] FLAG: --registry-burst="10"
Sep 09 20:40:37.618546 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610948    2581 flags.go:64] FLAG: --registry-qps="5"
Sep 09 20:40:37.618546 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610951    2581 flags.go:64] FLAG: --reserved-cpus=""
Sep 09 20:40:37.618546 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610954    2581 flags.go:64] FLAG: --reserved-memory=""
Sep 09 20:40:37.618546 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610957    2581 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 09 20:40:37.618546 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610961    2581 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 09 20:40:37.619169 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610964    2581 flags.go:64] FLAG: --rotate-certificates="false"
Sep 09 20:40:37.619169 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610967    2581 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 09 20:40:37.619169 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610970    2581 flags.go:64] FLAG: --runonce="false"
Sep 09 20:40:37.619169 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610972    2581 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 09 20:40:37.619169 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610976    2581 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 09 20:40:37.619169 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610979    2581 flags.go:64] FLAG: --seccomp-default="false"
Sep 09 20:40:37.619169 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610982    2581 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 09 20:40:37.619169 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610985    2581 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 09 20:40:37.619169 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610988    2581 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 09 20:40:37.619169 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610991    2581 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 09 20:40:37.619169 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610995    2581 flags.go:64] FLAG: --storage-driver-password="root"
Sep 09 20:40:37.619169 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.610998    2581 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 09 20:40:37.619169 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.611000    2581 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 09 20:40:37.619169 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.611003    2581 flags.go:64] FLAG: --storage-driver-user="root"
Sep 09 20:40:37.619169 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.611006    2581 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 09 20:40:37.619169 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.611012    2581 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 09 20:40:37.619169 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.611016    2581 flags.go:64] FLAG: --system-cgroups=""
Sep 09 20:40:37.619169 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.611019    2581 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 09 20:40:37.619169 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.611024    2581 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 09 20:40:37.619169 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.611027    2581 flags.go:64] FLAG: --tls-cert-file=""
Sep 09 20:40:37.619169 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.611030    2581 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 09 20:40:37.619169 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.611034    2581 flags.go:64] FLAG: --tls-min-version=""
Sep 09 20:40:37.619169 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.611037    2581 flags.go:64] FLAG: --tls-private-key-file=""
Sep 09 20:40:37.619169 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.611040    2581 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 09 20:40:37.619169 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.611042    2581 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 09 20:40:37.619794 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.611045    2581 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 09 20:40:37.619794 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.611048    2581 flags.go:64] FLAG: --v="2"
Sep 09 20:40:37.619794 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.611057    2581 flags.go:64] FLAG: --version="false"
Sep 09 20:40:37.619794 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.611060    2581 flags.go:64] FLAG: --vmodule=""
Sep 09 20:40:37.619794 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.611064    2581 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 09 20:40:37.619794 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.611067    2581 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 09 20:40:37.619794 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611185    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 20:40:37.619794 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611189    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 20:40:37.619794 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611193    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 20:40:37.619794 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611196    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 20:40:37.619794 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611200    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 20:40:37.619794 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611203    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 20:40:37.619794 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611205    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 20:40:37.619794 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611208    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 20:40:37.619794 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611211    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 20:40:37.619794 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611214    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 20:40:37.619794 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611217    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 20:40:37.619794 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611220    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 20:40:37.619794 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611223    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 20:40:37.619794 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611225    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 20:40:37.619794 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611228    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 20:40:37.620335 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611230    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 20:40:37.620335 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611235    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 20:40:37.620335 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611237    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 20:40:37.620335 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611241    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 20:40:37.620335 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611244    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 20:40:37.620335 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611246    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 20:40:37.620335 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611249    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 20:40:37.620335 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611252    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 20:40:37.620335 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611254    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 20:40:37.620335 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611257    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 20:40:37.620335 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611260    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 20:40:37.620335 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611262    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 20:40:37.620335 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611265    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 20:40:37.620335 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611269    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 20:40:37.620335 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611271    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 20:40:37.620335 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611274    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 20:40:37.620335 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611277    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 20:40:37.620335 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611279    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 20:40:37.620335 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611282    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 20:40:37.620335 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611284    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 20:40:37.620335 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611287    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 20:40:37.620901 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611290    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 20:40:37.620901 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611292    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 20:40:37.620901 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611295    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 20:40:37.620901 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611298    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 20:40:37.620901 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611301    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 20:40:37.620901 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611305    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 20:40:37.620901 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611308    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 20:40:37.620901 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611311    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 20:40:37.620901 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611314    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 20:40:37.620901 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611317    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 20:40:37.620901 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611319    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 20:40:37.620901 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611322    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 20:40:37.620901 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611326    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 20:40:37.620901 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611329    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 20:40:37.620901 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611331    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 20:40:37.620901 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611334    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 20:40:37.620901 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611337    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 20:40:37.620901 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611339    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 20:40:37.620901 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611342    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 20:40:37.621368 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611345    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 20:40:37.621368 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611349    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 20:40:37.621368 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611352    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 20:40:37.621368 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611355    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 20:40:37.621368 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611358    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 20:40:37.621368 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611362    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 20:40:37.621368 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611365    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 20:40:37.621368 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611368    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 20:40:37.621368 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611370    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 20:40:37.621368 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611373    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 20:40:37.621368 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611375    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 20:40:37.621368 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611378    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 20:40:37.621368 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611381    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 20:40:37.621368 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611383    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 20:40:37.621368 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611386    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 20:40:37.621368 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611388    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 20:40:37.621368 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611391    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 20:40:37.621368 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611394    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 20:40:37.621368 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611396    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 20:40:37.621862 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611399    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 20:40:37.621862 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611402    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 20:40:37.621862 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611405    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 20:40:37.621862 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611409    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 20:40:37.621862 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611413    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 20:40:37.621862 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611417    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 20:40:37.621862 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611423    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 20:40:37.621862 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611426    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 20:40:37.621862 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611428    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 20:40:37.621862 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611431    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 20:40:37.621862 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611434    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 20:40:37.621862 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611437    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 20:40:37.621862 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611440    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 20:40:37.621862 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611443    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 20:40:37.621862 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611445    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 20:40:37.621862 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611448    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 20:40:37.621862 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611451    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 20:40:37.621862 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611453    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 20:40:37.621862 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611457    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 20:40:37.622334 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611461    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 20:40:37.622334 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611464    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 20:40:37.622334 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611467    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 20:40:37.622334 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611469    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 20:40:37.622334 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611472    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 20:40:37.622334 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.611475    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 20:40:37.622334 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.612007    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 09 20:40:37.622334 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.618938    2581 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 09 20:40:37.622334 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.618954    2581 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 09 20:40:37.622334 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.618968    2581 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 09 20:40:37.622334 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.618974    2581 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 09 20:40:37.622334 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619030    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 20:40:37.622334 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619036    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 20:40:37.622334 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619040    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 20:40:37.622334 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619043    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 20:40:37.622722 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619046    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 20:40:37.622722 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619049    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 20:40:37.622722 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619052    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 20:40:37.622722 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619055    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 20:40:37.622722 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619058    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 20:40:37.622722 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619061    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 20:40:37.622722 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619063    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 20:40:37.622722 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619066    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 20:40:37.622722 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619069    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 20:40:37.622722 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619072    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 20:40:37.622722 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619074    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 20:40:37.622722 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619078    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 20:40:37.622722 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619080    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 20:40:37.622722 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619083    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 20:40:37.622722 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619086    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 20:40:37.622722 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619088    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 20:40:37.622722 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619091    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 20:40:37.622722 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619094    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 20:40:37.622722 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619097    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 20:40:37.622722 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619099    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 20:40:37.623227 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619102    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 20:40:37.623227 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619104    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 20:40:37.623227 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619107    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 20:40:37.623227 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619109    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 20:40:37.623227 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619112    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 20:40:37.623227 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619116    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 20:40:37.623227 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619121    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 20:40:37.623227 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619125    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 20:40:37.623227 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619129    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 20:40:37.623227 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619134    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 20:40:37.623227 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619137    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 20:40:37.623227 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619140    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 20:40:37.623227 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619143    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 20:40:37.623227 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619146    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 20:40:37.623227 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619149    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 20:40:37.623227 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619152    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 20:40:37.623227 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619154    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 20:40:37.623227 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619157    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 20:40:37.623227 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619160    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 20:40:37.623697 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619162    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 20:40:37.623697 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619165    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 20:40:37.623697 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619168    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 20:40:37.623697 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619170    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 20:40:37.623697 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619174    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 20:40:37.623697 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619177    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 20:40:37.623697 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619180    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 20:40:37.623697 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619183    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 20:40:37.623697 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619186    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 20:40:37.623697 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619188    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 20:40:37.623697 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619191    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 20:40:37.623697 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619194    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 20:40:37.623697 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619197    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 20:40:37.623697 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619199    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 20:40:37.623697 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619202    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 20:40:37.623697 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619205    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 20:40:37.623697 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619207    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 20:40:37.623697 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619210    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 20:40:37.623697 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619213    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 20:40:37.623697 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619217    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 20:40:37.624243 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619219    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 20:40:37.624243 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619222    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 20:40:37.624243 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619225    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 20:40:37.624243 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619227    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 20:40:37.624243 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619230    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 20:40:37.624243 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619232    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 20:40:37.624243 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619235    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 20:40:37.624243 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619237    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 20:40:37.624243 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619240    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 20:40:37.624243 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619242    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 20:40:37.624243 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619245    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 20:40:37.624243 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619248    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 20:40:37.624243 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619250    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 20:40:37.624243 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619253    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 20:40:37.624243 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619255    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 20:40:37.624243 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619258    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 20:40:37.624243 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619261    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 20:40:37.624243 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619263    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 20:40:37.624243 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619266    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 20:40:37.624243 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619268    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 20:40:37.624721 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619271    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 20:40:37.624721 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619274    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 20:40:37.624721 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619276    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 20:40:37.624721 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619279    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 20:40:37.624721 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619281    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 20:40:37.624721 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619284    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 20:40:37.624721 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619288    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 20:40:37.624721 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619291    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 20:40:37.624721 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619294    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 20:40:37.624721 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619297    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 20:40:37.624721 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619300    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 20:40:37.624721 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619303    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 20:40:37.624721 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619306    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 20:40:37.624721 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619309    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 20:40:37.624721 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619311    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 20:40:37.624721 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619314    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 20:40:37.625137 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.619319    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 09 20:40:37.625137 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619437    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 20:40:37.625137 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619442    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 20:40:37.625137 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619445    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 20:40:37.625137 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619448    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 20:40:37.625137 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619450    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 20:40:37.625137 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619453    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 20:40:37.625137 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619456    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 20:40:37.625137 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619459    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 20:40:37.625137 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619461    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 20:40:37.625137 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619464    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 20:40:37.625137 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619467    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 20:40:37.625137 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619469    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 20:40:37.625137 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619472    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 20:40:37.625137 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619475    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 20:40:37.625678 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619477    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 20:40:37.625678 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619480    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 20:40:37.625678 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619482    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 20:40:37.625678 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619485    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 20:40:37.625678 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619487    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 20:40:37.625678 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619490    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 20:40:37.625678 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619493    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 20:40:37.625678 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619495    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 20:40:37.625678 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619498    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 20:40:37.625678 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619500    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 20:40:37.625678 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619503    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 20:40:37.625678 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619506    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 20:40:37.625678 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619509    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 20:40:37.625678 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619512    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 20:40:37.625678 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619515    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 20:40:37.625678 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619517    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 20:40:37.625678 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619520    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 20:40:37.625678 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619523    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 20:40:37.625678 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619526    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 20:40:37.626219 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619531    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 20:40:37.626219 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619534    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 20:40:37.626219 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619537    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 20:40:37.626219 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619539    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 20:40:37.626219 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619542    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 20:40:37.626219 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619545    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 20:40:37.626219 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619548    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 20:40:37.626219 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619550    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 20:40:37.626219 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619553    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 20:40:37.626219 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619556    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 20:40:37.626219 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619559    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 20:40:37.626219 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619563    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 20:40:37.626219 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619567    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 20:40:37.626219 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619570    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 20:40:37.626219 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619573    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 20:40:37.626219 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619576    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 20:40:37.626219 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619579    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 20:40:37.626219 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619582    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 20:40:37.626219 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619585    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 20:40:37.626705 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619588    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 20:40:37.626705 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619591    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 20:40:37.626705 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619594    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 20:40:37.626705 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619596    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 20:40:37.626705 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619599    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 20:40:37.626705 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619602    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 20:40:37.626705 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619605    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 20:40:37.626705 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619608    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 20:40:37.626705 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619610    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 20:40:37.626705 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619614    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 20:40:37.626705 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619617    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 20:40:37.626705 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619619    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 20:40:37.626705 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619622    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 20:40:37.626705 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619624    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 20:40:37.626705 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619627    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 20:40:37.626705 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619629    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 20:40:37.626705 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619632    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 20:40:37.626705 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619634    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 20:40:37.626705 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619637    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 20:40:37.626705 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619640    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 20:40:37.627278 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619642    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 20:40:37.627278 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619644    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 20:40:37.627278 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619647    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 20:40:37.627278 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619650    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 20:40:37.627278 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619652    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 20:40:37.627278 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619655    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 20:40:37.627278 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619657    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 20:40:37.627278 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619660    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 20:40:37.627278 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619662    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 20:40:37.627278 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619665    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 20:40:37.627278 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619668    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 20:40:37.627278 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619670    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 20:40:37.627278 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619673    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 20:40:37.627278 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619675    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 20:40:37.627278 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619678    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 20:40:37.627278 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619681    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 20:40:37.627278 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619683    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 20:40:37.627278 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619687    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 20:40:37.627278 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619689    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 20:40:37.627278 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619692    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 20:40:37.627764 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619694    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 20:40:37.627764 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619697    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 20:40:37.627764 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619700    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 20:40:37.627764 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619703    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 20:40:37.627764 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619705    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 20:40:37.627764 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619708    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 20:40:37.627764 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:37.619710    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 20:40:37.627764 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.619715    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 09 20:40:37.627764 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.619836    2581 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 09 20:40:37.627764 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.621379    2581 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 09 20:40:37.627764 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.622251    2581 server.go:1019] "Starting client certificate rotation"
Sep 09 20:40:37.627764 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.622349    2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 09 20:40:37.627764 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.622381    2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 09 20:40:37.628232 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.625444    2581 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 09 20:40:37.630490 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.630461    2581 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 09 20:40:37.639890 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.639874    2581 log.go:25] "Validated CRI v1 runtime API"
Sep 09 20:40:37.642786 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.642770    2581 log.go:25] "Validated CRI v1 image API"
Sep 09 20:40:37.643621 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.643607    2581 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 09 20:40:37.645842 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.645822    2581 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 bb6ef077-20ec-40a9-adab-31db5a3172d2:/dev/nvme0n1p3 f342f511-31ca-4caa-a323-f0a38d480d22:/dev/nvme0n1p4]
Sep 09 20:40:37.645924 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.645840    2581 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 09 20:40:37.650845 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.650740    2581 manager.go:216] Machine: {Timestamp:2026-09-09 20:40:37.649408319 +0000 UTC m=+0.319286485 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099798 MemoryCapacity:33164484608 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2fbc36b51d3327c2d01678d16feb00 SystemUUID:ec2fbc36-b51d-3327-c2d0-1678d16feb00 BootID:fa7bb73c-abce-49e9-a83b-71ae02580da4 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:57:39:d0:44:95 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:57:39:d0:44:95 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:be:f7:c2:d8:3e:f7 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164484608 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 09 20:40:37.650845 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.650843    2581 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 09 20:40:37.650962 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.650933    2581 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 09 20:40:37.653570 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.653543    2581 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 09 20:40:37.653707 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.653572    2581 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-99.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 09 20:40:37.654274 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.654259    2581 topology_manager.go:138] "Creating topology manager with none policy"
Sep 09 20:40:37.654274 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.654272    2581 container_manager_linux.go:309] "Creating device plugin manager"
Sep 09 20:40:37.654367 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.654279    2581 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 09 20:40:37.654554 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.654539    2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 09 20:40:37.654972 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.654961    2581 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 09 20:40:37.655757 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.655748    2581 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 09 20:40:37.659438 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.659427    2581 state_mem.go:36] "Initialized new in-memory state store"
Sep 09 20:40:37.659547 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.659538    2581 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 09 20:40:37.659579 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.659568    2581 kubelet.go:486] "Attempting to sync node with API server"
Sep 09 20:40:37.659606 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.659582    2581 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 09 20:40:37.659606 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.659599    2581 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 09 20:40:37.659659 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.659614    2581 kubelet.go:398] "Adding apiserver pod source"
Sep 09 20:40:37.659659 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.659635    2581 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 09 20:40:37.660112 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.660101    2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 09 20:40:37.660146 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.660122    2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 09 20:40:37.663108 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.663093    2581 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 09 20:40:37.665624 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.665607    2581 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 09 20:40:37.665712 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.665646    2581 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 09 20:40:37.666268 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.666255    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 09 20:40:37.666330 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.666276    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 09 20:40:37.666330 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.666285    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 09 20:40:37.666330 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.666294    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 09 20:40:37.666330 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.666307    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 09 20:40:37.666330 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.666315    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 09 20:40:37.666517 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.666413    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 09 20:40:37.666517 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.666423    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 09 20:40:37.666517 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.666433    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 09 20:40:37.666517 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.666441    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 09 20:40:37.666517 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.666454    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 09 20:40:37.666517 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.666466    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 09 20:40:37.667258 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.667248    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 09 20:40:37.667318 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.667260    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 09 20:40:37.669528 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.669507    2581 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-99.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 09 20:40:37.669630 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:37.669540    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 09 20:40:37.669630 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:37.669542    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-99.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 09 20:40:37.670905 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.670890    2581 server.go:1268] "Started kubelet"
Sep 09 20:40:37.670986 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.670952    2581 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 09 20:40:37.671084 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.671031    2581 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 09 20:40:37.671124 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.671105    2581 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 09 20:40:37.671632 ip-10-0-129-99 systemd[1]: Started Kubernetes Kubelet.
Sep 09 20:40:37.675125 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.674783    2581 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 09 20:40:37.675684 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.675632    2581 server.go:310] "Adding debug handlers to kubelet server"
Sep 09 20:40:37.677866 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:37.677830    2581 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 09 20:40:37.680676 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.680658    2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 09 20:40:37.681208 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.681192    2581 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 09 20:40:37.681331 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:37.681309    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-99.ec2.internal\" not found"
Sep 09 20:40:37.681814 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.681791    2581 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 09 20:40:37.681814 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.681815    2581 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 09 20:40:37.682012 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.681790    2581 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 09 20:40:37.682012 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.681953    2581 reconstruct.go:97] "Volume reconstruction finished"
Sep 09 20:40:37.682012 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.681963    2581 reconciler.go:29] "Reconciler: start to sync state"
Sep 09 20:40:37.682205 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:37.681088    2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-99.ec2.internal.18d3c1afc55caf18  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-99.ec2.internal,UID:ip-10-0-129-99.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-99.ec2.internal,},FirstTimestamp:2026-09-09 20:40:37.670842136 +0000 UTC m=+0.340720302,LastTimestamp:2026-09-09 20:40:37.670842136 +0000 UTC m=+0.340720302,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-99.ec2.internal,}"
Sep 09 20:40:37.682205 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.682110    2581 factory.go:55] Registering systemd factory
Sep 09 20:40:37.682205 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.682122    2581 factory.go:223] Registration of the systemd container factory successfully
Sep 09 20:40:37.682603 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.682330    2581 factory.go:153] Registering CRI-O factory
Sep 09 20:40:37.682603 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.682345    2581 factory.go:223] Registration of the crio container factory successfully
Sep 09 20:40:37.683202 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.683181    2581 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 09 20:40:37.683280 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.683206    2581 factory.go:103] Registering Raw factory
Sep 09 20:40:37.683280 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.683221    2581 manager.go:1196] Started watching for new ooms in manager
Sep 09 20:40:37.683589 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.683574    2581 manager.go:349] Starting recovery of all containers
Sep 09 20:40:37.683981 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.683959    2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-snzpc"
Sep 09 20:40:37.688528 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:37.688505    2581 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-99.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 09 20:40:37.688614 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:37.688582    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 09 20:40:37.692151 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.692127    2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-snzpc"
Sep 09 20:40:37.693546 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.693526    2581 manager.go:354] Recovery completed
Sep 09 20:40:37.694656 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:37.694631    2581 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Sep 09 20:40:37.698157 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.698142    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 20:40:37.699984 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.699969    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-99.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 20:40:37.700049 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.699996    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-99.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 20:40:37.700049 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.700006    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-99.ec2.internal" event="NodeHasSufficientPID"
Sep 09 20:40:37.700532 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.700519    2581 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 09 20:40:37.700532 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.700531    2581 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 09 20:40:37.700630 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.700545    2581 state_mem.go:36] "Initialized new in-memory state store"
Sep 09 20:40:37.700630 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.700553    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 20:40:37.701284 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.701272    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-99.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 20:40:37.701337 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.701297    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-99.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 20:40:37.701337 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.701309    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-99.ec2.internal" event="NodeHasSufficientPID"
Sep 09 20:40:37.702260 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:37.702166    2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-99.ec2.internal.18d3c1afc71959c0  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-99.ec2.internal,UID:ip-10-0-129-99.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-99.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-99.ec2.internal,},FirstTimestamp:2026-09-09 20:40:37.699983808 +0000 UTC m=+0.369861974,LastTimestamp:2026-09-09 20:40:37.699983808 +0000 UTC m=+0.369861974,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-99.ec2.internal,}"
Sep 09 20:40:37.702585 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.702574    2581 policy_none.go:49] "None policy: Start"
Sep 09 20:40:37.702635 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.702590    2581 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 09 20:40:37.702635 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.702604    2581 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 09 20:40:37.704542 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.704529    2581 policy_none.go:47] "Start"
Sep 09 20:40:37.731560 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.731539    2581 manager.go:337] "Starting Device Plugin manager"
Sep 09 20:40:37.754412 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:37.731586    2581 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 09 20:40:37.754412 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.731601    2581 server.go:87] "Starting device plugin registration server"
Sep 09 20:40:37.754412 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.731984    2581 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 09 20:40:37.754412 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.731996    2581 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 09 20:40:37.754412 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.732120    2581 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 09 20:40:37.754412 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.732212    2581 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 09 20:40:37.754412 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.732222    2581 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 09 20:40:37.754412 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:37.732838    2581 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 09 20:40:37.754412 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:37.732908    2581 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-99.ec2.internal\" not found"
Sep 09 20:40:37.769974 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.769939    2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 09 20:40:37.771170 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.771148    2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 09 20:40:37.771253 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.771208    2581 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 09 20:40:37.771307 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.771294    2581 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 09 20:40:37.771421 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:37.771405    2581 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 09 20:40:37.773482 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.773464    2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 20:40:37.832918 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.832889    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 20:40:37.833990 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.833970    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-99.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 20:40:37.834094 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.834005    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-99.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 20:40:37.834094 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.834021    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-99.ec2.internal" event="NodeHasSufficientPID"
Sep 09 20:40:37.834094 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.834052    2581 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-99.ec2.internal"
Sep 09 20:40:37.842191 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.842168    2581 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-99.ec2.internal"
Sep 09 20:40:37.842191 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:37.842194    2581 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-99.ec2.internal\": node \"ip-10-0-129-99.ec2.internal\" not found"
Sep 09 20:40:37.865356 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:37.865288    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-99.ec2.internal\" not found"
Sep 09 20:40:37.872357 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.872324    2581 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-99.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-99.ec2.internal"]
Sep 09 20:40:37.872469 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.872404    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 20:40:37.873447 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.873434    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-99.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 20:40:37.873519 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.873461    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-99.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 20:40:37.873519 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.873471    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-99.ec2.internal" event="NodeHasSufficientPID"
Sep 09 20:40:37.875845 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.875832    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 20:40:37.875978 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.875963    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-99.ec2.internal"
Sep 09 20:40:37.876014 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.875999    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 20:40:37.876616 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.876598    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-99.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 20:40:37.876709 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.876629    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-99.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 20:40:37.876709 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.876644    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-99.ec2.internal" event="NodeHasSufficientPID"
Sep 09 20:40:37.876709 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.876599    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-99.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 20:40:37.876709 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.876699    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-99.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 20:40:37.876709 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.876708    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-99.ec2.internal" event="NodeHasSufficientPID"
Sep 09 20:40:37.878817 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.878801    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-99.ec2.internal"
Sep 09 20:40:37.878924 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.878827    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 20:40:37.879879 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.879845    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-99.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 20:40:37.879978 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.879890    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-99.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 20:40:37.879978 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.879906    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-99.ec2.internal" event="NodeHasSufficientPID"
Sep 09 20:40:37.883707 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.883688    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/77a4e0e482d02ec9e357614bf125373e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-99.ec2.internal\" (UID: \"77a4e0e482d02ec9e357614bf125373e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-99.ec2.internal"
Sep 09 20:40:37.883788 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.883712    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/75cd1de7188eb1602ddc4211bd282ec4-config\") pod \"kube-apiserver-proxy-ip-10-0-129-99.ec2.internal\" (UID: \"75cd1de7188eb1602ddc4211bd282ec4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-99.ec2.internal"
Sep 09 20:40:37.883788 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.883730    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/77a4e0e482d02ec9e357614bf125373e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-99.ec2.internal\" (UID: \"77a4e0e482d02ec9e357614bf125373e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-99.ec2.internal"
Sep 09 20:40:37.905612 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:37.905589    2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-99.ec2.internal\" not found" node="ip-10-0-129-99.ec2.internal"
Sep 09 20:40:37.910018 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:37.909994    2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-99.ec2.internal\" not found" node="ip-10-0-129-99.ec2.internal"
Sep 09 20:40:37.965541 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:37.965520    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-99.ec2.internal\" not found"
Sep 09 20:40:37.984490 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.984464    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/77a4e0e482d02ec9e357614bf125373e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-99.ec2.internal\" (UID: \"77a4e0e482d02ec9e357614bf125373e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-99.ec2.internal"
Sep 09 20:40:37.984569 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.984491    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/77a4e0e482d02ec9e357614bf125373e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-99.ec2.internal\" (UID: \"77a4e0e482d02ec9e357614bf125373e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-99.ec2.internal"
Sep 09 20:40:37.984569 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.984514    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/75cd1de7188eb1602ddc4211bd282ec4-config\") pod \"kube-apiserver-proxy-ip-10-0-129-99.ec2.internal\" (UID: \"75cd1de7188eb1602ddc4211bd282ec4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-99.ec2.internal"
Sep 09 20:40:37.984569 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.984566    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/77a4e0e482d02ec9e357614bf125373e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-99.ec2.internal\" (UID: \"77a4e0e482d02ec9e357614bf125373e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-99.ec2.internal"
Sep 09 20:40:37.984658 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.984568    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/77a4e0e482d02ec9e357614bf125373e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-99.ec2.internal\" (UID: \"77a4e0e482d02ec9e357614bf125373e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-99.ec2.internal"
Sep 09 20:40:37.984658 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:37.984605    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/75cd1de7188eb1602ddc4211bd282ec4-config\") pod \"kube-apiserver-proxy-ip-10-0-129-99.ec2.internal\" (UID: \"75cd1de7188eb1602ddc4211bd282ec4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-99.ec2.internal"
Sep 09 20:40:38.066557 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:38.066534    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-99.ec2.internal\" not found"
Sep 09 20:40:38.167451 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:38.167376    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-99.ec2.internal\" not found"
Sep 09 20:40:38.206916 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:38.206887    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-99.ec2.internal"
Sep 09 20:40:38.212912 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:38.212894    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-99.ec2.internal"
Sep 09 20:40:38.268290 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:38.268262    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-99.ec2.internal\" not found"
Sep 09 20:40:39.411691 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:39.411651    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-99.ec2.internal\" not found"
Sep 09 20:40:39.512298 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:39.512230    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-99.ec2.internal\" not found"
Sep 09 20:40:39.597524 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.597499    2581 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 20:40:39.624614 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.624591    2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-99.ec2.internal"
Sep 09 20:40:39.635723 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.635709    2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 09 20:40:39.637649 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.637636    2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-99.ec2.internal"
Sep 09 20:40:39.647307 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.647290    2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 09 20:40:39.664429 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.664403    2581 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 09 20:40:39.664554 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.664538    2581 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 09 20:40:39.664596 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.664538    2581 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 09 20:40:39.702765 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.702729    2581 apiserver.go:52] "Watching apiserver"
Sep 09 20:40:39.720484 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.720452    2581 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 09 20:40:39.720802 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.720783    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-2c5hn","openshift-network-operator/iptables-alerter-ft4fb","openshift-ovn-kubernetes/ovnkube-node-75wj4","kube-system/konnectivity-agent-lx79f","kube-system/kube-apiserver-proxy-ip-10-0-129-99.ec2.internal","openshift-cluster-node-tuning-operator/tuned-v6pmz","openshift-dns/node-resolver-cfg6s","openshift-image-registry/node-ca-f8mqt","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-99.ec2.internal","openshift-multus/multus-additional-cni-plugins-p6642","openshift-multus/multus-rj5rk","kube-system/global-pull-secret-syncer-bjwzq","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jq8ls","openshift-multus/network-metrics-daemon-2fzgg"]
Sep 09 20:40:39.723783 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.723731    2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 09 20:40:39.724239 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.724202    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rj5rk"
Sep 09 20:40:39.726374 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.726353    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 09 20:40:39.726479 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.726441    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-ft4fb"
Sep 09 20:40:39.726540 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.726496    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-7d425\""
Sep 09 20:40:39.726593 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.726543    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 09 20:40:39.726716 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.726697    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 09 20:40:39.726909 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.726894    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 09 20:40:39.728591 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.728575    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-jcxcx\""
Sep 09 20:40:39.728663 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.728583    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 09 20:40:39.728779 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.728761    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 09 20:40:39.728898 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.728885    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-75wj4"
Sep 09 20:40:39.729448 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.729433    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 09 20:40:39.730456 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.730439    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d1d25b48-1c88-4b60-93a1-665b90df19da-iptables-alerter-script\") pod \"iptables-alerter-ft4fb\" (UID: \"d1d25b48-1c88-4b60-93a1-665b90df19da\") " pod="openshift-network-operator/iptables-alerter-ft4fb"
Sep 09 20:40:39.730520 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.730465    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f-system-cni-dir\") pod \"multus-rj5rk\" (UID: \"964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f\") " pod="openshift-multus/multus-rj5rk"
Sep 09 20:40:39.730520 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.730481    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f-cnibin\") pod \"multus-rj5rk\" (UID: \"964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f\") " pod="openshift-multus/multus-rj5rk"
Sep 09 20:40:39.730520 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.730495    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f-multus-socket-dir-parent\") pod \"multus-rj5rk\" (UID: \"964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f\") " pod="openshift-multus/multus-rj5rk"
Sep 09 20:40:39.730520 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.730513    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d1d25b48-1c88-4b60-93a1-665b90df19da-host-slash\") pod \"iptables-alerter-ft4fb\" (UID: \"d1d25b48-1c88-4b60-93a1-665b90df19da\") " pod="openshift-network-operator/iptables-alerter-ft4fb"
Sep 09 20:40:39.730648 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.730548    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f-cni-binary-copy\") pod \"multus-rj5rk\" (UID: \"964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f\") " pod="openshift-multus/multus-rj5rk"
Sep 09 20:40:39.730648 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.730569    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f-host-var-lib-cni-bin\") pod \"multus-rj5rk\" (UID: \"964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f\") " pod="openshift-multus/multus-rj5rk"
Sep 09 20:40:39.730648 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.730612    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f-host-var-lib-kubelet\") pod \"multus-rj5rk\" (UID: \"964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f\") " pod="openshift-multus/multus-rj5rk"
Sep 09 20:40:39.730648 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.730629    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f-multus-conf-dir\") pod \"multus-rj5rk\" (UID: \"964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f\") " pod="openshift-multus/multus-rj5rk"
Sep 09 20:40:39.730648 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.730646    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f-multus-daemon-config\") pod \"multus-rj5rk\" (UID: \"964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f\") " pod="openshift-multus/multus-rj5rk"
Sep 09 20:40:39.730823 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.730660    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f-host-run-multus-certs\") pod \"multus-rj5rk\" (UID: \"964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f\") " pod="openshift-multus/multus-rj5rk"
Sep 09 20:40:39.730823 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.730675    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f-etc-kubernetes\") pod \"multus-rj5rk\" (UID: \"964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f\") " pod="openshift-multus/multus-rj5rk"
Sep 09 20:40:39.730823 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.730690    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f-host-var-lib-cni-multus\") pod \"multus-rj5rk\" (UID: \"964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f\") " pod="openshift-multus/multus-rj5rk"
Sep 09 20:40:39.730823 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.730708    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hvsj\" (UniqueName: \"kubernetes.io/projected/d1d25b48-1c88-4b60-93a1-665b90df19da-kube-api-access-5hvsj\") pod \"iptables-alerter-ft4fb\" (UID: \"d1d25b48-1c88-4b60-93a1-665b90df19da\") " pod="openshift-network-operator/iptables-alerter-ft4fb"
Sep 09 20:40:39.730823 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.730721    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f-multus-cni-dir\") pod \"multus-rj5rk\" (UID: \"964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f\") " pod="openshift-multus/multus-rj5rk"
Sep 09 20:40:39.730823 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.730747    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f-host-run-k8s-cni-cncf-io\") pod \"multus-rj5rk\" (UID: \"964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f\") " pod="openshift-multus/multus-rj5rk"
Sep 09 20:40:39.730823 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.730771    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f-host-run-netns\") pod \"multus-rj5rk\" (UID: \"964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f\") " pod="openshift-multus/multus-rj5rk"
Sep 09 20:40:39.730823 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.730786    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f-hostroot\") pod \"multus-rj5rk\" (UID: \"964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f\") " pod="openshift-multus/multus-rj5rk"
Sep 09 20:40:39.730823 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.730800    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mbcx\" (UniqueName: \"kubernetes.io/projected/964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f-kube-api-access-2mbcx\") pod \"multus-rj5rk\" (UID: \"964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f\") " pod="openshift-multus/multus-rj5rk"
Sep 09 20:40:39.731138 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.730828    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 09 20:40:39.731188 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.731151    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-lx79f"
Sep 09 20:40:39.731771 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.731758    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 09 20:40:39.731815 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.731773    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 09 20:40:39.731815 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.731811    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 09 20:40:39.732251 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.732240    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 09 20:40:39.732418 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.732397    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 09 20:40:39.732498 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.732449    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-w54k8\""
Sep 09 20:40:39.733291 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.733276    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-4mbpn\""
Sep 09 20:40:39.733372 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.733282    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 09 20:40:39.733372 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.733348    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 09 20:40:39.733544 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.733530    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2fzgg"
Sep 09 20:40:39.733638 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:39.733596    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2fzgg" podUID="1105e43b-4cb7-4b23-89ba-cf6c5370f185"
Sep 09 20:40:39.735591 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.735576    2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 09 20:40:39.735680 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.735663    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-v6pmz"
Sep 09 20:40:39.737831 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.737817    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2c5hn"
Sep 09 20:40:39.737906 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.737841    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 09 20:40:39.737970 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:39.737925    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2c5hn" podUID="2f3c30c5-3a66-4a51-ba58-a125dce436ab"
Sep 09 20:40:39.738243 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.738225    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-7q7rm\""
Sep 09 20:40:39.738313 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.738299    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 09 20:40:39.739975 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.739961    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-f8mqt"
Sep 09 20:40:39.741235 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.741195    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-08 20:35:38 +0000 UTC" deadline="2028-05-24 15:56:21.220439714 +0000 UTC"
Sep 09 20:40:39.741235 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.741233    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14947h15m41.479209357s"
Sep 09 20:40:39.742234 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.742210    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 09 20:40:39.742234 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.742225    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 09 20:40:39.742377 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.742224    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-cfg6s"
Sep 09 20:40:39.742377 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.742259    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-vng9c\""
Sep 09 20:40:39.742679 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.742662    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 09 20:40:39.745040 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.745024    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-p6642"
Sep 09 20:40:39.746053 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.746035    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 09 20:40:39.746216 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.746203    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-b5ntm\""
Sep 09 20:40:39.746330 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.746318    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 09 20:40:39.748306 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.748279    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bjwzq"
Sep 09 20:40:39.748411 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.748347    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 09 20:40:39.748411 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:39.748380    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bjwzq" podUID="ad64d253-c155-4e3c-b872-cdebb0616a0e"
Sep 09 20:40:39.748555 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.748423    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 09 20:40:39.748555 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.748506    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-cvldf\""
Sep 09 20:40:39.751241 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.751223    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jq8ls"
Sep 09 20:40:39.753487 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.753467    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-28xmp\""
Sep 09 20:40:39.753646 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.753631    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 09 20:40:39.753856 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.753840    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 09 20:40:39.754038 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.754014    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 09 20:40:39.767095 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.767046    2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-qc7qt"
Sep 09 20:40:39.774655 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.774636    2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-qc7qt"
Sep 09 20:40:39.825268 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.825241    2581 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 09 20:40:39.830976 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.830954    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f-cni-binary-copy\") pod \"multus-rj5rk\" (UID: \"964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f\") " pod="openshift-multus/multus-rj5rk"
Sep 09 20:40:39.831083 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.830979    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f-multus-daemon-config\") pod \"multus-rj5rk\" (UID: \"964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f\") " pod="openshift-multus/multus-rj5rk"
Sep 09 20:40:39.831083 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.831001    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/82c490e6-edea-4f5c-9fe5-111fe55ccee9-cni-binary-copy\") pod \"multus-additional-cni-plugins-p6642\" (UID: \"82c490e6-edea-4f5c-9fe5-111fe55ccee9\") " pod="openshift-multus/multus-additional-cni-plugins-p6642"
Sep 09 20:40:39.831083 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.831017    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/55b4fbae-29dc-41bf-a907-b1412b852cef-ovnkube-script-lib\") pod \"ovnkube-node-75wj4\" (UID: \"55b4fbae-29dc-41bf-a907-b1412b852cef\") " pod="openshift-ovn-kubernetes/ovnkube-node-75wj4"
Sep 09 20:40:39.831083 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.831031    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c45d31bc-82ca-49b3-9bfa-9bdae327bef7-etc-systemd\") pod \"tuned-v6pmz\" (UID: \"c45d31bc-82ca-49b3-9bfa-9bdae327bef7\") " pod="openshift-cluster-node-tuning-operator/tuned-v6pmz"
Sep 09 20:40:39.831083 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.831044    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c45d31bc-82ca-49b3-9bfa-9bdae327bef7-lib-modules\") pod \"tuned-v6pmz\" (UID: \"c45d31bc-82ca-49b3-9bfa-9bdae327bef7\") " pod="openshift-cluster-node-tuning-operator/tuned-v6pmz"
Sep 09 20:40:39.831083 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.831063    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c45d31bc-82ca-49b3-9bfa-9bdae327bef7-host\") pod \"tuned-v6pmz\" (UID: \"c45d31bc-82ca-49b3-9bfa-9bdae327bef7\") " pod="openshift-cluster-node-tuning-operator/tuned-v6pmz"
Sep 09 20:40:39.831385 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.831138    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7bdq\" (UniqueName: \"kubernetes.io/projected/0104bf49-c8d9-493a-b189-e570ebf93e62-kube-api-access-s7bdq\") pod \"node-ca-f8mqt\" (UID: \"0104bf49-c8d9-493a-b189-e570ebf93e62\") " pod="openshift-image-registry/node-ca-f8mqt"
Sep 09 20:40:39.831385 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.831193    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hvsj\" (UniqueName: \"kubernetes.io/projected/d1d25b48-1c88-4b60-93a1-665b90df19da-kube-api-access-5hvsj\") pod \"iptables-alerter-ft4fb\" (UID: \"d1d25b48-1c88-4b60-93a1-665b90df19da\") " pod="openshift-network-operator/iptables-alerter-ft4fb"
Sep 09 20:40:39.831385 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.831215    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4ljn\" (UniqueName: \"kubernetes.io/projected/5096efd2-6fbf-45a2-930b-acdc3970eabd-kube-api-access-z4ljn\") pod \"node-resolver-cfg6s\" (UID: \"5096efd2-6fbf-45a2-930b-acdc3970eabd\") " pod="openshift-dns/node-resolver-cfg6s"
Sep 09 20:40:39.831385 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.831232    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/55b4fbae-29dc-41bf-a907-b1412b852cef-node-log\") pod \"ovnkube-node-75wj4\" (UID: \"55b4fbae-29dc-41bf-a907-b1412b852cef\") " pod="openshift-ovn-kubernetes/ovnkube-node-75wj4"
Sep 09 20:40:39.831385 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.831251    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c45d31bc-82ca-49b3-9bfa-9bdae327bef7-etc-kubernetes\") pod \"tuned-v6pmz\" (UID: \"c45d31bc-82ca-49b3-9bfa-9bdae327bef7\") " pod="openshift-cluster-node-tuning-operator/tuned-v6pmz"
Sep 09 20:40:39.831385 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.831281    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c45d31bc-82ca-49b3-9bfa-9bdae327bef7-etc-sysctl-conf\") pod \"tuned-v6pmz\" (UID: \"c45d31bc-82ca-49b3-9bfa-9bdae327bef7\") " pod="openshift-cluster-node-tuning-operator/tuned-v6pmz"
Sep 09 20:40:39.831385 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.831299    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/10602efd-a422-4354-9f8e-bc58dd9ec7dd-socket-dir\") pod \"aws-ebs-csi-driver-node-jq8ls\" (UID: \"10602efd-a422-4354-9f8e-bc58dd9ec7dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jq8ls"
Sep 09 20:40:39.831385 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.831319    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f-host-run-netns\") pod \"multus-rj5rk\" (UID: \"964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f\") " pod="openshift-multus/multus-rj5rk"
Sep 09 20:40:39.831385 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.831364    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f-host-run-netns\") pod \"multus-rj5rk\" (UID: \"964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f\") " pod="openshift-multus/multus-rj5rk"
Sep 09 20:40:39.831385 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.831365    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1105e43b-4cb7-4b23-89ba-cf6c5370f185-metrics-certs\") pod \"network-metrics-daemon-2fzgg\" (UID: \"1105e43b-4cb7-4b23-89ba-cf6c5370f185\") " pod="openshift-multus/network-metrics-daemon-2fzgg"
Sep 09 20:40:39.831846 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.831428    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgmj6\" (UniqueName: \"kubernetes.io/projected/82c490e6-edea-4f5c-9fe5-111fe55ccee9-kube-api-access-zgmj6\") pod \"multus-additional-cni-plugins-p6642\" (UID: \"82c490e6-edea-4f5c-9fe5-111fe55ccee9\") " pod="openshift-multus/multus-additional-cni-plugins-p6642"
Sep 09 20:40:39.831846 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.831467    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/55b4fbae-29dc-41bf-a907-b1412b852cef-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-75wj4\" (UID: \"55b4fbae-29dc-41bf-a907-b1412b852cef\") " pod="openshift-ovn-kubernetes/ovnkube-node-75wj4"
Sep 09 20:40:39.831846 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.831506    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/55b4fbae-29dc-41bf-a907-b1412b852cef-ovn-node-metrics-cert\") pod \"ovnkube-node-75wj4\" (UID: \"55b4fbae-29dc-41bf-a907-b1412b852cef\") " pod="openshift-ovn-kubernetes/ovnkube-node-75wj4"
Sep 09 20:40:39.831846 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.831532    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ad64d253-c155-4e3c-b872-cdebb0616a0e-kubelet-config\") pod \"global-pull-secret-syncer-bjwzq\" (UID: \"ad64d253-c155-4e3c-b872-cdebb0616a0e\") " pod="kube-system/global-pull-secret-syncer-bjwzq"
Sep 09 20:40:39.831846 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.831556    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ad64d253-c155-4e3c-b872-cdebb0616a0e-original-pull-secret\") pod \"global-pull-secret-syncer-bjwzq\" (UID: \"ad64d253-c155-4e3c-b872-cdebb0616a0e\") " pod="kube-system/global-pull-secret-syncer-bjwzq"
Sep 09 20:40:39.831846 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.831573    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f-cni-binary-copy\") pod \"multus-rj5rk\" (UID: \"964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f\") " pod="openshift-multus/multus-rj5rk"
Sep 09 20:40:39.831846 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.831581    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/82c490e6-edea-4f5c-9fe5-111fe55ccee9-system-cni-dir\") pod \"multus-additional-cni-plugins-p6642\" (UID: \"82c490e6-edea-4f5c-9fe5-111fe55ccee9\") " pod="openshift-multus/multus-additional-cni-plugins-p6642"
Sep 09 20:40:39.831846 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.831608    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/55b4fbae-29dc-41bf-a907-b1412b852cef-log-socket\") pod \"ovnkube-node-75wj4\" (UID: \"55b4fbae-29dc-41bf-a907-b1412b852cef\") " pod="openshift-ovn-kubernetes/ovnkube-node-75wj4"
Sep 09 20:40:39.831846 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.831575    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f-multus-daemon-config\") pod \"multus-rj5rk\" (UID: \"964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f\") " pod="openshift-multus/multus-rj5rk"
Sep 09 20:40:39.831846 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.831632    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/55b4fbae-29dc-41bf-a907-b1412b852cef-host-cni-netd\") pod \"ovnkube-node-75wj4\" (UID: \"55b4fbae-29dc-41bf-a907-b1412b852cef\") " pod="openshift-ovn-kubernetes/ovnkube-node-75wj4"
Sep 09 20:40:39.831846 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.831656    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ad64d253-c155-4e3c-b872-cdebb0616a0e-dbus\") pod \"global-pull-secret-syncer-bjwzq\" (UID: \"ad64d253-c155-4e3c-b872-cdebb0616a0e\") " pod="kube-system/global-pull-secret-syncer-bjwzq"
Sep 09 20:40:39.831846 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.831683    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f-host-var-lib-kubelet\") pod \"multus-rj5rk\" (UID: \"964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f\") " pod="openshift-multus/multus-rj5rk"
Sep 09 20:40:39.831846 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.831720    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0bd19a8a-4d11-4e06-b2f9-7f04ccd35574-agent-certs\") pod \"konnectivity-agent-lx79f\" (UID: \"0bd19a8a-4d11-4e06-b2f9-7f04ccd35574\") " pod="kube-system/konnectivity-agent-lx79f"
Sep 09 20:40:39.831846 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.831742    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f-host-var-lib-kubelet\") pod \"multus-rj5rk\" (UID: \"964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f\") " pod="openshift-multus/multus-rj5rk"
Sep 09 20:40:39.831846 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.831745    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0bd19a8a-4d11-4e06-b2f9-7f04ccd35574-konnectivity-ca\") pod \"konnectivity-agent-lx79f\" (UID: \"0bd19a8a-4d11-4e06-b2f9-7f04ccd35574\") " pod="kube-system/konnectivity-agent-lx79f"
Sep 09 20:40:39.831846 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.831780    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/82c490e6-edea-4f5c-9fe5-111fe55ccee9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-p6642\" (UID: \"82c490e6-edea-4f5c-9fe5-111fe55ccee9\") " pod="openshift-multus/multus-additional-cni-plugins-p6642"
Sep 09 20:40:39.832491 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.831799    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c45d31bc-82ca-49b3-9bfa-9bdae327bef7-etc-sysctl-d\") pod \"tuned-v6pmz\" (UID: \"c45d31bc-82ca-49b3-9bfa-9bdae327bef7\") " pod="openshift-cluster-node-tuning-operator/tuned-v6pmz"
Sep 09 20:40:39.832491 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.831812    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c45d31bc-82ca-49b3-9bfa-9bdae327bef7-tmp\") pod \"tuned-v6pmz\" (UID: \"c45d31bc-82ca-49b3-9bfa-9bdae327bef7\") " pod="openshift-cluster-node-tuning-operator/tuned-v6pmz"
Sep 09 20:40:39.832491 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.831826    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5096efd2-6fbf-45a2-930b-acdc3970eabd-tmp-dir\") pod \"node-resolver-cfg6s\" (UID: \"5096efd2-6fbf-45a2-930b-acdc3970eabd\") " pod="openshift-dns/node-resolver-cfg6s"
Sep 09 20:40:39.832491 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.831843    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/82c490e6-edea-4f5c-9fe5-111fe55ccee9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-p6642\" (UID: \"82c490e6-edea-4f5c-9fe5-111fe55ccee9\") " pod="openshift-multus/multus-additional-cni-plugins-p6642"
Sep 09 20:40:39.832491 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.831872    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/55b4fbae-29dc-41bf-a907-b1412b852cef-ovnkube-config\") pod \"ovnkube-node-75wj4\" (UID: \"55b4fbae-29dc-41bf-a907-b1412b852cef\") " pod="openshift-ovn-kubernetes/ovnkube-node-75wj4"
Sep 09 20:40:39.832491 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.831906    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0104bf49-c8d9-493a-b189-e570ebf93e62-serviceca\") pod \"node-ca-f8mqt\" (UID: \"0104bf49-c8d9-493a-b189-e570ebf93e62\") " pod="openshift-image-registry/node-ca-f8mqt"
Sep 09 20:40:39.832491 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.831932    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mbcx\" (UniqueName: \"kubernetes.io/projected/964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f-kube-api-access-2mbcx\") pod \"multus-rj5rk\" (UID: \"964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f\") " pod="openshift-multus/multus-rj5rk"
Sep 09 20:40:39.832491 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.831956    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d1d25b48-1c88-4b60-93a1-665b90df19da-iptables-alerter-script\") pod \"iptables-alerter-ft4fb\" (UID: \"d1d25b48-1c88-4b60-93a1-665b90df19da\") " pod="openshift-network-operator/iptables-alerter-ft4fb"
Sep 09 20:40:39.832491 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.832004    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/55b4fbae-29dc-41bf-a907-b1412b852cef-var-lib-openvswitch\") pod \"ovnkube-node-75wj4\" (UID: \"55b4fbae-29dc-41bf-a907-b1412b852cef\") " pod="openshift-ovn-kubernetes/ovnkube-node-75wj4"
Sep 09 20:40:39.832491 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.832072    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d1d25b48-1c88-4b60-93a1-665b90df19da-host-slash\") pod \"iptables-alerter-ft4fb\" (UID: \"d1d25b48-1c88-4b60-93a1-665b90df19da\") " pod="openshift-network-operator/iptables-alerter-ft4fb"
Sep 09 20:40:39.832491 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.832096    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d1d25b48-1c88-4b60-93a1-665b90df19da-host-slash\") pod \"iptables-alerter-ft4fb\" (UID: \"d1d25b48-1c88-4b60-93a1-665b90df19da\") " pod="openshift-network-operator/iptables-alerter-ft4fb"
Sep 09 20:40:39.832491 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.832096    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/55b4fbae-29dc-41bf-a907-b1412b852cef-host-run-netns\") pod \"ovnkube-node-75wj4\" (UID: \"55b4fbae-29dc-41bf-a907-b1412b852cef\") " pod="openshift-ovn-kubernetes/ovnkube-node-75wj4"
Sep 09 20:40:39.832491 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.832143    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/55b4fbae-29dc-41bf-a907-b1412b852cef-etc-openvswitch\") pod \"ovnkube-node-75wj4\" (UID: \"55b4fbae-29dc-41bf-a907-b1412b852cef\") " pod="openshift-ovn-kubernetes/ovnkube-node-75wj4"
Sep 09 20:40:39.832491 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.832172    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/55b4fbae-29dc-41bf-a907-b1412b852cef-env-overrides\") pod \"ovnkube-node-75wj4\" (UID: \"55b4fbae-29dc-41bf-a907-b1412b852cef\") " pod="openshift-ovn-kubernetes/ovnkube-node-75wj4"
Sep 09 20:40:39.832491 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.832199    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c45d31bc-82ca-49b3-9bfa-9bdae327bef7-var-lib-kubelet\") pod \"tuned-v6pmz\" (UID: \"c45d31bc-82ca-49b3-9bfa-9bdae327bef7\") " pod="openshift-cluster-node-tuning-operator/tuned-v6pmz"
Sep 09 20:40:39.832491 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.832218    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c45d31bc-82ca-49b3-9bfa-9bdae327bef7-etc-tuned\") pod \"tuned-v6pmz\" (UID: \"c45d31bc-82ca-49b3-9bfa-9bdae327bef7\") " pod="openshift-cluster-node-tuning-operator/tuned-v6pmz"
Sep 09 20:40:39.832491 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.832244    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2xwb\" (UniqueName: \"kubernetes.io/projected/10602efd-a422-4354-9f8e-bc58dd9ec7dd-kube-api-access-z2xwb\") pod \"aws-ebs-csi-driver-node-jq8ls\" (UID: \"10602efd-a422-4354-9f8e-bc58dd9ec7dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jq8ls"
Sep 09 20:40:39.833130 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.832276    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f-multus-conf-dir\") pod \"multus-rj5rk\" (UID: \"964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f\") " pod="openshift-multus/multus-rj5rk"
Sep 09 20:40:39.833130 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.832306    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f-host-run-multus-certs\") pod \"multus-rj5rk\" (UID: \"964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f\") " pod="openshift-multus/multus-rj5rk"
Sep 09 20:40:39.833130 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.832341    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f-multus-conf-dir\") pod \"multus-rj5rk\" (UID: \"964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f\") " pod="openshift-multus/multus-rj5rk"
Sep 09 20:40:39.833130 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.832345    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/82c490e6-edea-4f5c-9fe5-111fe55ccee9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-p6642\" (UID: \"82c490e6-edea-4f5c-9fe5-111fe55ccee9\") " pod="openshift-multus/multus-additional-cni-plugins-p6642"
Sep 09 20:40:39.833130 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.832379    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f-host-run-multus-certs\") pod \"multus-rj5rk\" (UID: \"964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f\") " pod="openshift-multus/multus-rj5rk"
Sep 09 20:40:39.833130 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.832381    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/55b4fbae-29dc-41bf-a907-b1412b852cef-host-kubelet\") pod \"ovnkube-node-75wj4\" (UID: \"55b4fbae-29dc-41bf-a907-b1412b852cef\") " pod="openshift-ovn-kubernetes/ovnkube-node-75wj4"
Sep 09 20:40:39.833130 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.832401    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d1d25b48-1c88-4b60-93a1-665b90df19da-iptables-alerter-script\") pod \"iptables-alerter-ft4fb\" (UID: \"d1d25b48-1c88-4b60-93a1-665b90df19da\") " pod="openshift-network-operator/iptables-alerter-ft4fb"
Sep 09 20:40:39.833130 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.832420    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/55b4fbae-29dc-41bf-a907-b1412b852cef-host-slash\") pod \"ovnkube-node-75wj4\" (UID: \"55b4fbae-29dc-41bf-a907-b1412b852cef\") " pod="openshift-ovn-kubernetes/ovnkube-node-75wj4"
Sep 09 20:40:39.833130 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.832436    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c45d31bc-82ca-49b3-9bfa-9bdae327bef7-etc-modprobe-d\") pod \"tuned-v6pmz\" (UID: \"c45d31bc-82ca-49b3-9bfa-9bdae327bef7\") " pod="openshift-cluster-node-tuning-operator/tuned-v6pmz"
Sep 09 20:40:39.833130 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.832449    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0104bf49-c8d9-493a-b189-e570ebf93e62-host\") pod \"node-ca-f8mqt\" (UID: \"0104bf49-c8d9-493a-b189-e570ebf93e62\") " pod="openshift-image-registry/node-ca-f8mqt"
Sep 09 20:40:39.833130 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.832469    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/10602efd-a422-4354-9f8e-bc58dd9ec7dd-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jq8ls\" (UID: \"10602efd-a422-4354-9f8e-bc58dd9ec7dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jq8ls"
Sep 09 20:40:39.833130 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.832505    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5096efd2-6fbf-45a2-930b-acdc3970eabd-hosts-file\") pod \"node-resolver-cfg6s\" (UID: \"5096efd2-6fbf-45a2-930b-acdc3970eabd\") " pod="openshift-dns/node-resolver-cfg6s"
Sep 09 20:40:39.833130 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.832519    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/82c490e6-edea-4f5c-9fe5-111fe55ccee9-cnibin\") pod \"multus-additional-cni-plugins-p6642\" (UID: \"82c490e6-edea-4f5c-9fe5-111fe55ccee9\") " pod="openshift-multus/multus-additional-cni-plugins-p6642"
Sep 09 20:40:39.833130 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.832533    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/10602efd-a422-4354-9f8e-bc58dd9ec7dd-etc-selinux\") pod \"aws-ebs-csi-driver-node-jq8ls\" (UID: \"10602efd-a422-4354-9f8e-bc58dd9ec7dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jq8ls"
Sep 09 20:40:39.833130 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.832555    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f-host-run-k8s-cni-cncf-io\") pod \"multus-rj5rk\" (UID: \"964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f\") " pod="openshift-multus/multus-rj5rk"
Sep 09 20:40:39.833130 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.832600    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/55b4fbae-29dc-41bf-a907-b1412b852cef-run-systemd\") pod \"ovnkube-node-75wj4\" (UID: \"55b4fbae-29dc-41bf-a907-b1412b852cef\") " pod="openshift-ovn-kubernetes/ovnkube-node-75wj4"
Sep 09 20:40:39.833130 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.832614    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/55b4fbae-29dc-41bf-a907-b1412b852cef-host-cni-bin\") pod \"ovnkube-node-75wj4\" (UID: \"55b4fbae-29dc-41bf-a907-b1412b852cef\") " pod="openshift-ovn-kubernetes/ovnkube-node-75wj4"
Sep 09 20:40:39.833617 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.832628    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4vmc\" (UniqueName: \"kubernetes.io/projected/55b4fbae-29dc-41bf-a907-b1412b852cef-kube-api-access-d4vmc\") pod \"ovnkube-node-75wj4\" (UID: \"55b4fbae-29dc-41bf-a907-b1412b852cef\") " pod="openshift-ovn-kubernetes/ovnkube-node-75wj4"
Sep 09 20:40:39.833617 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.832649    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f-multus-socket-dir-parent\") pod \"multus-rj5rk\" (UID: \"964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f\") " pod="openshift-multus/multus-rj5rk"
Sep 09 20:40:39.833617 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.832670    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c45d31bc-82ca-49b3-9bfa-9bdae327bef7-run\") pod \"tuned-v6pmz\" (UID: \"c45d31bc-82ca-49b3-9bfa-9bdae327bef7\") " pod="openshift-cluster-node-tuning-operator/tuned-v6pmz"
Sep 09 20:40:39.833617 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.832696    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f-host-run-k8s-cni-cncf-io\") pod \"multus-rj5rk\" (UID: \"964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f\") " pod="openshift-multus/multus-rj5rk"
Sep 09 20:40:39.833617 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.832691    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/10602efd-a422-4354-9f8e-bc58dd9ec7dd-device-dir\") pod \"aws-ebs-csi-driver-node-jq8ls\" (UID: \"10602efd-a422-4354-9f8e-bc58dd9ec7dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jq8ls"
Sep 09 20:40:39.833617 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.832703    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f-multus-socket-dir-parent\") pod \"multus-rj5rk\" (UID: \"964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f\") " pod="openshift-multus/multus-rj5rk"
Sep 09 20:40:39.833617 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.832730    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f-host-var-lib-cni-bin\") pod \"multus-rj5rk\" (UID: \"964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f\") " pod="openshift-multus/multus-rj5rk"
Sep 09 20:40:39.833617 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.832747    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f-etc-kubernetes\") pod \"multus-rj5rk\" (UID: \"964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f\") " pod="openshift-multus/multus-rj5rk"
Sep 09 20:40:39.833617 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.832762    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/10602efd-a422-4354-9f8e-bc58dd9ec7dd-registration-dir\") pod \"aws-ebs-csi-driver-node-jq8ls\" (UID: \"10602efd-a422-4354-9f8e-bc58dd9ec7dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jq8ls"
Sep 09 20:40:39.833617 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.832781    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f-host-var-lib-cni-multus\") pod \"multus-rj5rk\" (UID: \"964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f\") " pod="openshift-multus/multus-rj5rk"
Sep 09 20:40:39.833617 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.832796    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/55b4fbae-29dc-41bf-a907-b1412b852cef-run-ovn\") pod \"ovnkube-node-75wj4\" (UID: \"55b4fbae-29dc-41bf-a907-b1412b852cef\") " pod="openshift-ovn-kubernetes/ovnkube-node-75wj4"
Sep 09 20:40:39.833617 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.832798    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f-host-var-lib-cni-bin\") pod \"multus-rj5rk\" (UID: \"964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f\") " pod="openshift-multus/multus-rj5rk"
Sep 09 20:40:39.833617 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.832822    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/55b4fbae-29dc-41bf-a907-b1412b852cef-host-run-ovn-kubernetes\") pod \"ovnkube-node-75wj4\" (UID: \"55b4fbae-29dc-41bf-a907-b1412b852cef\") " pod="openshift-ovn-kubernetes/ovnkube-node-75wj4"
Sep 09 20:40:39.833617 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.832831    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f-host-var-lib-cni-multus\") pod \"multus-rj5rk\" (UID: \"964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f\") " pod="openshift-multus/multus-rj5rk"
Sep 09 20:40:39.833617 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.832828    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f-etc-kubernetes\") pod \"multus-rj5rk\" (UID: \"964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f\") " pod="openshift-multus/multus-rj5rk"
Sep 09 20:40:39.833617 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.832859    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c45d31bc-82ca-49b3-9bfa-9bdae327bef7-etc-sysconfig\") pod \"tuned-v6pmz\" (UID: \"c45d31bc-82ca-49b3-9bfa-9bdae327bef7\") " pod="openshift-cluster-node-tuning-operator/tuned-v6pmz"
Sep 09 20:40:39.833617 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.832873    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzbcs\" (UniqueName: \"kubernetes.io/projected/c45d31bc-82ca-49b3-9bfa-9bdae327bef7-kube-api-access-xzbcs\") pod \"tuned-v6pmz\" (UID: \"c45d31bc-82ca-49b3-9bfa-9bdae327bef7\") " pod="openshift-cluster-node-tuning-operator/tuned-v6pmz"
Sep 09 20:40:39.834165 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.832896    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f-multus-cni-dir\") pod \"multus-rj5rk\" (UID: \"964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f\") " pod="openshift-multus/multus-rj5rk"
Sep 09 20:40:39.834165 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.832911    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f-hostroot\") pod \"multus-rj5rk\" (UID: \"964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f\") " pod="openshift-multus/multus-rj5rk"
Sep 09 20:40:39.834165 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.832931    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqgzr\" (UniqueName: \"kubernetes.io/projected/1105e43b-4cb7-4b23-89ba-cf6c5370f185-kube-api-access-zqgzr\") pod \"network-metrics-daemon-2fzgg\" (UID: \"1105e43b-4cb7-4b23-89ba-cf6c5370f185\") " pod="openshift-multus/network-metrics-daemon-2fzgg"
Sep 09 20:40:39.834165 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.832965    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f-hostroot\") pod \"multus-rj5rk\" (UID: \"964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f\") " pod="openshift-multus/multus-rj5rk"
Sep 09 20:40:39.834165 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.832972    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9gvv\" (UniqueName: \"kubernetes.io/projected/2f3c30c5-3a66-4a51-ba58-a125dce436ab-kube-api-access-x9gvv\") pod \"network-check-target-2c5hn\" (UID: \"2f3c30c5-3a66-4a51-ba58-a125dce436ab\") " pod="openshift-network-diagnostics/network-check-target-2c5hn"
Sep 09 20:40:39.834165 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.832981    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f-multus-cni-dir\") pod \"multus-rj5rk\" (UID: \"964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f\") " pod="openshift-multus/multus-rj5rk"
Sep 09 20:40:39.834165 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.832998    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/55b4fbae-29dc-41bf-a907-b1412b852cef-systemd-units\") pod \"ovnkube-node-75wj4\" (UID: \"55b4fbae-29dc-41bf-a907-b1412b852cef\") " pod="openshift-ovn-kubernetes/ovnkube-node-75wj4"
Sep 09 20:40:39.834165 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.833013    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/55b4fbae-29dc-41bf-a907-b1412b852cef-run-openvswitch\") pod \"ovnkube-node-75wj4\" (UID: \"55b4fbae-29dc-41bf-a907-b1412b852cef\") " pod="openshift-ovn-kubernetes/ovnkube-node-75wj4"
Sep 09 20:40:39.834165 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.833035    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f-system-cni-dir\") pod \"multus-rj5rk\" (UID: \"964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f\") " pod="openshift-multus/multus-rj5rk"
Sep 09 20:40:39.834165 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.833061    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f-cnibin\") pod \"multus-rj5rk\" (UID: \"964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f\") " pod="openshift-multus/multus-rj5rk"
Sep 09 20:40:39.834165 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.833081    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c45d31bc-82ca-49b3-9bfa-9bdae327bef7-sys\") pod \"tuned-v6pmz\" (UID: \"c45d31bc-82ca-49b3-9bfa-9bdae327bef7\") " pod="openshift-cluster-node-tuning-operator/tuned-v6pmz"
Sep 09 20:40:39.834165 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.833113    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/10602efd-a422-4354-9f8e-bc58dd9ec7dd-sys-fs\") pod \"aws-ebs-csi-driver-node-jq8ls\" (UID: \"10602efd-a422-4354-9f8e-bc58dd9ec7dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jq8ls"
Sep 09 20:40:39.834165 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.833124    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f-system-cni-dir\") pod \"multus-rj5rk\" (UID: \"964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f\") " pod="openshift-multus/multus-rj5rk"
Sep 09 20:40:39.834165 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.833142    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f-cnibin\") pod \"multus-rj5rk\" (UID: \"964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f\") " pod="openshift-multus/multus-rj5rk"
Sep 09 20:40:39.841569 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.841549    2581 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 09 20:40:39.845205 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.845182    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mbcx\" (UniqueName: \"kubernetes.io/projected/964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f-kube-api-access-2mbcx\") pod \"multus-rj5rk\" (UID: \"964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f\") " pod="openshift-multus/multus-rj5rk"
Sep 09 20:40:39.845282 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.845183    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hvsj\" (UniqueName: \"kubernetes.io/projected/d1d25b48-1c88-4b60-93a1-665b90df19da-kube-api-access-5hvsj\") pod \"iptables-alerter-ft4fb\" (UID: \"d1d25b48-1c88-4b60-93a1-665b90df19da\") " pod="openshift-network-operator/iptables-alerter-ft4fb"
Sep 09 20:40:39.933686 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.933653    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzbcs\" (UniqueName: \"kubernetes.io/projected/c45d31bc-82ca-49b3-9bfa-9bdae327bef7-kube-api-access-xzbcs\") pod \"tuned-v6pmz\" (UID: \"c45d31bc-82ca-49b3-9bfa-9bdae327bef7\") " pod="openshift-cluster-node-tuning-operator/tuned-v6pmz"
Sep 09 20:40:39.933686 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.933688    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqgzr\" (UniqueName: \"kubernetes.io/projected/1105e43b-4cb7-4b23-89ba-cf6c5370f185-kube-api-access-zqgzr\") pod \"network-metrics-daemon-2fzgg\" (UID: \"1105e43b-4cb7-4b23-89ba-cf6c5370f185\") " pod="openshift-multus/network-metrics-daemon-2fzgg"
Sep 09 20:40:39.933922 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.933703    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9gvv\" (UniqueName: \"kubernetes.io/projected/2f3c30c5-3a66-4a51-ba58-a125dce436ab-kube-api-access-x9gvv\") pod \"network-check-target-2c5hn\" (UID: \"2f3c30c5-3a66-4a51-ba58-a125dce436ab\") " pod="openshift-network-diagnostics/network-check-target-2c5hn"
Sep 09 20:40:39.934004 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.933986    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/55b4fbae-29dc-41bf-a907-b1412b852cef-systemd-units\") pod \"ovnkube-node-75wj4\" (UID: \"55b4fbae-29dc-41bf-a907-b1412b852cef\") " pod="openshift-ovn-kubernetes/ovnkube-node-75wj4"
Sep 09 20:40:39.934054 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.934021    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/55b4fbae-29dc-41bf-a907-b1412b852cef-run-openvswitch\") pod \"ovnkube-node-75wj4\" (UID: \"55b4fbae-29dc-41bf-a907-b1412b852cef\") " pod="openshift-ovn-kubernetes/ovnkube-node-75wj4"
Sep 09 20:40:39.934054 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.934040    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c45d31bc-82ca-49b3-9bfa-9bdae327bef7-sys\") pod \"tuned-v6pmz\" (UID: \"c45d31bc-82ca-49b3-9bfa-9bdae327bef7\") " pod="openshift-cluster-node-tuning-operator/tuned-v6pmz"
Sep 09 20:40:39.934161 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.934056    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/10602efd-a422-4354-9f8e-bc58dd9ec7dd-sys-fs\") pod \"aws-ebs-csi-driver-node-jq8ls\" (UID: \"10602efd-a422-4354-9f8e-bc58dd9ec7dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jq8ls"
Sep 09 20:40:39.934161 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.934082    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/55b4fbae-29dc-41bf-a907-b1412b852cef-run-openvswitch\") pod \"ovnkube-node-75wj4\" (UID: \"55b4fbae-29dc-41bf-a907-b1412b852cef\") " pod="openshift-ovn-kubernetes/ovnkube-node-75wj4"
Sep 09 20:40:39.934161 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.934119    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/82c490e6-edea-4f5c-9fe5-111fe55ccee9-cni-binary-copy\") pod \"multus-additional-cni-plugins-p6642\" (UID: \"82c490e6-edea-4f5c-9fe5-111fe55ccee9\") " pod="openshift-multus/multus-additional-cni-plugins-p6642"
Sep 09 20:40:39.934161 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.934149    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/55b4fbae-29dc-41bf-a907-b1412b852cef-ovnkube-script-lib\") pod \"ovnkube-node-75wj4\" (UID: \"55b4fbae-29dc-41bf-a907-b1412b852cef\") " pod="openshift-ovn-kubernetes/ovnkube-node-75wj4"
Sep 09 20:40:39.934316 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.934165    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/10602efd-a422-4354-9f8e-bc58dd9ec7dd-sys-fs\") pod \"aws-ebs-csi-driver-node-jq8ls\" (UID: \"10602efd-a422-4354-9f8e-bc58dd9ec7dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jq8ls"
Sep 09 20:40:39.934316 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.934175    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c45d31bc-82ca-49b3-9bfa-9bdae327bef7-etc-systemd\") pod \"tuned-v6pmz\" (UID: \"c45d31bc-82ca-49b3-9bfa-9bdae327bef7\") " pod="openshift-cluster-node-tuning-operator/tuned-v6pmz"
Sep 09 20:40:39.934316 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.934177    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c45d31bc-82ca-49b3-9bfa-9bdae327bef7-sys\") pod \"tuned-v6pmz\" (UID: \"c45d31bc-82ca-49b3-9bfa-9bdae327bef7\") " pod="openshift-cluster-node-tuning-operator/tuned-v6pmz"
Sep 09 20:40:39.934316 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.934080    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/55b4fbae-29dc-41bf-a907-b1412b852cef-systemd-units\") pod \"ovnkube-node-75wj4\" (UID: \"55b4fbae-29dc-41bf-a907-b1412b852cef\") " pod="openshift-ovn-kubernetes/ovnkube-node-75wj4"
Sep 09 20:40:39.934316 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.934207    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c45d31bc-82ca-49b3-9bfa-9bdae327bef7-lib-modules\") pod \"tuned-v6pmz\" (UID: \"c45d31bc-82ca-49b3-9bfa-9bdae327bef7\") " pod="openshift-cluster-node-tuning-operator/tuned-v6pmz"
Sep 09 20:40:39.934316 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.934233    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c45d31bc-82ca-49b3-9bfa-9bdae327bef7-etc-systemd\") pod \"tuned-v6pmz\" (UID: \"c45d31bc-82ca-49b3-9bfa-9bdae327bef7\") " pod="openshift-cluster-node-tuning-operator/tuned-v6pmz"
Sep 09 20:40:39.934316 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.934286    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c45d31bc-82ca-49b3-9bfa-9bdae327bef7-host\") pod \"tuned-v6pmz\" (UID: \"c45d31bc-82ca-49b3-9bfa-9bdae327bef7\") " pod="openshift-cluster-node-tuning-operator/tuned-v6pmz"
Sep 09 20:40:39.934316 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.934313    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7bdq\" (UniqueName: \"kubernetes.io/projected/0104bf49-c8d9-493a-b189-e570ebf93e62-kube-api-access-s7bdq\") pod \"node-ca-f8mqt\" (UID: \"0104bf49-c8d9-493a-b189-e570ebf93e62\") " pod="openshift-image-registry/node-ca-f8mqt"
Sep 09 20:40:39.934673 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.934332    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4ljn\" (UniqueName: \"kubernetes.io/projected/5096efd2-6fbf-45a2-930b-acdc3970eabd-kube-api-access-z4ljn\") pod \"node-resolver-cfg6s\" (UID: \"5096efd2-6fbf-45a2-930b-acdc3970eabd\") " pod="openshift-dns/node-resolver-cfg6s"
Sep 09 20:40:39.934673 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.934340    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c45d31bc-82ca-49b3-9bfa-9bdae327bef7-host\") pod \"tuned-v6pmz\" (UID: \"c45d31bc-82ca-49b3-9bfa-9bdae327bef7\") " pod="openshift-cluster-node-tuning-operator/tuned-v6pmz"
Sep 09 20:40:39.934673 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.934346    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/55b4fbae-29dc-41bf-a907-b1412b852cef-node-log\") pod \"ovnkube-node-75wj4\" (UID: \"55b4fbae-29dc-41bf-a907-b1412b852cef\") " pod="openshift-ovn-kubernetes/ovnkube-node-75wj4"
Sep 09 20:40:39.934673 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.934379    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/55b4fbae-29dc-41bf-a907-b1412b852cef-node-log\") pod \"ovnkube-node-75wj4\" (UID: \"55b4fbae-29dc-41bf-a907-b1412b852cef\") " pod="openshift-ovn-kubernetes/ovnkube-node-75wj4"
Sep 09 20:40:39.934673 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.934388    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c45d31bc-82ca-49b3-9bfa-9bdae327bef7-lib-modules\") pod \"tuned-v6pmz\" (UID: \"c45d31bc-82ca-49b3-9bfa-9bdae327bef7\") " pod="openshift-cluster-node-tuning-operator/tuned-v6pmz"
Sep 09 20:40:39.934673 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.934392    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c45d31bc-82ca-49b3-9bfa-9bdae327bef7-etc-kubernetes\") pod \"tuned-v6pmz\" (UID: \"c45d31bc-82ca-49b3-9bfa-9bdae327bef7\") " pod="openshift-cluster-node-tuning-operator/tuned-v6pmz"
Sep 09 20:40:39.934673 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.934417    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c45d31bc-82ca-49b3-9bfa-9bdae327bef7-etc-sysctl-conf\") pod \"tuned-v6pmz\" (UID: \"c45d31bc-82ca-49b3-9bfa-9bdae327bef7\") " pod="openshift-cluster-node-tuning-operator/tuned-v6pmz"
Sep 09 20:40:39.934673 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.934444    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/10602efd-a422-4354-9f8e-bc58dd9ec7dd-socket-dir\") pod \"aws-ebs-csi-driver-node-jq8ls\" (UID: \"10602efd-a422-4354-9f8e-bc58dd9ec7dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jq8ls"
Sep 09 20:40:39.934673 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.934598    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/82c490e6-edea-4f5c-9fe5-111fe55ccee9-cni-binary-copy\") pod \"multus-additional-cni-plugins-p6642\" (UID: \"82c490e6-edea-4f5c-9fe5-111fe55ccee9\") " pod="openshift-multus/multus-additional-cni-plugins-p6642"
Sep 09 20:40:39.934673 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.934618    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c45d31bc-82ca-49b3-9bfa-9bdae327bef7-etc-kubernetes\") pod \"tuned-v6pmz\" (UID: \"c45d31bc-82ca-49b3-9bfa-9bdae327bef7\") " pod="openshift-cluster-node-tuning-operator/tuned-v6pmz"
Sep 09 20:40:39.934673 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.934651    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1105e43b-4cb7-4b23-89ba-cf6c5370f185-metrics-certs\") pod \"network-metrics-daemon-2fzgg\" (UID: \"1105e43b-4cb7-4b23-89ba-cf6c5370f185\") " pod="openshift-multus/network-metrics-daemon-2fzgg"
Sep 09 20:40:39.935226 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:39.934722    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 20:40:39.935226 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.934744    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/55b4fbae-29dc-41bf-a907-b1412b852cef-ovnkube-script-lib\") pod \"ovnkube-node-75wj4\" (UID: \"55b4fbae-29dc-41bf-a907-b1412b852cef\") " pod="openshift-ovn-kubernetes/ovnkube-node-75wj4"
Sep 09 20:40:39.935226 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.934759    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgmj6\" (UniqueName: \"kubernetes.io/projected/82c490e6-edea-4f5c-9fe5-111fe55ccee9-kube-api-access-zgmj6\") pod \"multus-additional-cni-plugins-p6642\" (UID: \"82c490e6-edea-4f5c-9fe5-111fe55ccee9\") " pod="openshift-multus/multus-additional-cni-plugins-p6642"
Sep 09 20:40:39.935226 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.934770    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c45d31bc-82ca-49b3-9bfa-9bdae327bef7-etc-sysctl-conf\") pod \"tuned-v6pmz\" (UID: \"c45d31bc-82ca-49b3-9bfa-9bdae327bef7\") " pod="openshift-cluster-node-tuning-operator/tuned-v6pmz"
Sep 09 20:40:39.935226 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.934772    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/10602efd-a422-4354-9f8e-bc58dd9ec7dd-socket-dir\") pod \"aws-ebs-csi-driver-node-jq8ls\" (UID: \"10602efd-a422-4354-9f8e-bc58dd9ec7dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jq8ls"
Sep 09 20:40:39.935226 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:39.934784    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1105e43b-4cb7-4b23-89ba-cf6c5370f185-metrics-certs podName:1105e43b-4cb7-4b23-89ba-cf6c5370f185 nodeName:}" failed. No retries permitted until 2026-09-09 20:40:40.434764618 +0000 UTC m=+2.062392473 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1105e43b-4cb7-4b23-89ba-cf6c5370f185-metrics-certs") pod "network-metrics-daemon-2fzgg" (UID: "1105e43b-4cb7-4b23-89ba-cf6c5370f185") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 20:40:39.935226 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.934837    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/55b4fbae-29dc-41bf-a907-b1412b852cef-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-75wj4\" (UID: \"55b4fbae-29dc-41bf-a907-b1412b852cef\") " pod="openshift-ovn-kubernetes/ovnkube-node-75wj4"
Sep 09 20:40:39.935226 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.934867    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/55b4fbae-29dc-41bf-a907-b1412b852cef-ovn-node-metrics-cert\") pod \"ovnkube-node-75wj4\" (UID: \"55b4fbae-29dc-41bf-a907-b1412b852cef\") " pod="openshift-ovn-kubernetes/ovnkube-node-75wj4"
Sep 09 20:40:39.935226 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.934893    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ad64d253-c155-4e3c-b872-cdebb0616a0e-kubelet-config\") pod \"global-pull-secret-syncer-bjwzq\" (UID: \"ad64d253-c155-4e3c-b872-cdebb0616a0e\") " pod="kube-system/global-pull-secret-syncer-bjwzq"
Sep 09 20:40:39.935226 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.934896    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/55b4fbae-29dc-41bf-a907-b1412b852cef-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-75wj4\" (UID: \"55b4fbae-29dc-41bf-a907-b1412b852cef\") " pod="openshift-ovn-kubernetes/ovnkube-node-75wj4"
Sep 09 20:40:39.935226 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.934917    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ad64d253-c155-4e3c-b872-cdebb0616a0e-original-pull-secret\") pod \"global-pull-secret-syncer-bjwzq\" (UID: \"ad64d253-c155-4e3c-b872-cdebb0616a0e\") " pod="kube-system/global-pull-secret-syncer-bjwzq"
Sep 09 20:40:39.935226 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.934941    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/82c490e6-edea-4f5c-9fe5-111fe55ccee9-system-cni-dir\") pod \"multus-additional-cni-plugins-p6642\" (UID: \"82c490e6-edea-4f5c-9fe5-111fe55ccee9\") " pod="openshift-multus/multus-additional-cni-plugins-p6642"
Sep 09 20:40:39.935226 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.934959    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ad64d253-c155-4e3c-b872-cdebb0616a0e-kubelet-config\") pod \"global-pull-secret-syncer-bjwzq\" (UID: \"ad64d253-c155-4e3c-b872-cdebb0616a0e\") " pod="kube-system/global-pull-secret-syncer-bjwzq"
Sep 09 20:40:39.935226 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.934964    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/55b4fbae-29dc-41bf-a907-b1412b852cef-log-socket\") pod \"ovnkube-node-75wj4\" (UID: \"55b4fbae-29dc-41bf-a907-b1412b852cef\") " pod="openshift-ovn-kubernetes/ovnkube-node-75wj4"
Sep 09 20:40:39.935226 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:39.934994    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 20:40:39.935226 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.935000    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/55b4fbae-29dc-41bf-a907-b1412b852cef-log-socket\") pod \"ovnkube-node-75wj4\" (UID: \"55b4fbae-29dc-41bf-a907-b1412b852cef\") " pod="openshift-ovn-kubernetes/ovnkube-node-75wj4"
Sep 09 20:40:39.935226 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.935004    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/55b4fbae-29dc-41bf-a907-b1412b852cef-host-cni-netd\") pod \"ovnkube-node-75wj4\" (UID: \"55b4fbae-29dc-41bf-a907-b1412b852cef\") " pod="openshift-ovn-kubernetes/ovnkube-node-75wj4"
Sep 09 20:40:39.936141 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.935030    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ad64d253-c155-4e3c-b872-cdebb0616a0e-dbus\") pod \"global-pull-secret-syncer-bjwzq\" (UID: \"ad64d253-c155-4e3c-b872-cdebb0616a0e\") " pod="kube-system/global-pull-secret-syncer-bjwzq"
Sep 09 20:40:39.936141 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.935033    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/82c490e6-edea-4f5c-9fe5-111fe55ccee9-system-cni-dir\") pod \"multus-additional-cni-plugins-p6642\" (UID: \"82c490e6-edea-4f5c-9fe5-111fe55ccee9\") " pod="openshift-multus/multus-additional-cni-plugins-p6642"
Sep 09 20:40:39.936141 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:39.935043    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad64d253-c155-4e3c-b872-cdebb0616a0e-original-pull-secret podName:ad64d253-c155-4e3c-b872-cdebb0616a0e nodeName:}" failed. No retries permitted until 2026-09-09 20:40:40.43502549 +0000 UTC m=+2.062653340 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ad64d253-c155-4e3c-b872-cdebb0616a0e-original-pull-secret") pod "global-pull-secret-syncer-bjwzq" (UID: "ad64d253-c155-4e3c-b872-cdebb0616a0e") : object "kube-system"/"original-pull-secret" not registered
Sep 09 20:40:39.936141 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.935071    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/55b4fbae-29dc-41bf-a907-b1412b852cef-host-cni-netd\") pod \"ovnkube-node-75wj4\" (UID: \"55b4fbae-29dc-41bf-a907-b1412b852cef\") " pod="openshift-ovn-kubernetes/ovnkube-node-75wj4"
Sep 09 20:40:39.936141 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.935078    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ad64d253-c155-4e3c-b872-cdebb0616a0e-dbus\") pod \"global-pull-secret-syncer-bjwzq\" (UID: \"ad64d253-c155-4e3c-b872-cdebb0616a0e\") " pod="kube-system/global-pull-secret-syncer-bjwzq"
Sep 09 20:40:39.936141 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.935077    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0bd19a8a-4d11-4e06-b2f9-7f04ccd35574-agent-certs\") pod \"konnectivity-agent-lx79f\" (UID: \"0bd19a8a-4d11-4e06-b2f9-7f04ccd35574\") " pod="kube-system/konnectivity-agent-lx79f"
Sep 09 20:40:39.936141 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.935137    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0bd19a8a-4d11-4e06-b2f9-7f04ccd35574-konnectivity-ca\") pod \"konnectivity-agent-lx79f\" (UID: \"0bd19a8a-4d11-4e06-b2f9-7f04ccd35574\") " pod="kube-system/konnectivity-agent-lx79f"
Sep 09 20:40:39.936141 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.935166    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/82c490e6-edea-4f5c-9fe5-111fe55ccee9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-p6642\" (UID: \"82c490e6-edea-4f5c-9fe5-111fe55ccee9\") " pod="openshift-multus/multus-additional-cni-plugins-p6642"
Sep 09 20:40:39.936141 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.935189    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c45d31bc-82ca-49b3-9bfa-9bdae327bef7-etc-sysctl-d\") pod \"tuned-v6pmz\" (UID: \"c45d31bc-82ca-49b3-9bfa-9bdae327bef7\") " pod="openshift-cluster-node-tuning-operator/tuned-v6pmz"
Sep 09 20:40:39.936141 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.935297    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c45d31bc-82ca-49b3-9bfa-9bdae327bef7-etc-sysctl-d\") pod \"tuned-v6pmz\" (UID: \"c45d31bc-82ca-49b3-9bfa-9bdae327bef7\") " pod="openshift-cluster-node-tuning-operator/tuned-v6pmz"
Sep 09 20:40:39.936141 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.935299    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c45d31bc-82ca-49b3-9bfa-9bdae327bef7-tmp\") pod \"tuned-v6pmz\" (UID: \"c45d31bc-82ca-49b3-9bfa-9bdae327bef7\") " pod="openshift-cluster-node-tuning-operator/tuned-v6pmz"
Sep 09 20:40:39.936141 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.935330    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5096efd2-6fbf-45a2-930b-acdc3970eabd-tmp-dir\") pod \"node-resolver-cfg6s\" (UID: \"5096efd2-6fbf-45a2-930b-acdc3970eabd\") " pod="openshift-dns/node-resolver-cfg6s"
Sep 09 20:40:39.936141 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.935359    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/82c490e6-edea-4f5c-9fe5-111fe55ccee9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-p6642\" (UID: \"82c490e6-edea-4f5c-9fe5-111fe55ccee9\") " pod="openshift-multus/multus-additional-cni-plugins-p6642"
Sep 09 20:40:39.936141 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.935386    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/55b4fbae-29dc-41bf-a907-b1412b852cef-ovnkube-config\") pod \"ovnkube-node-75wj4\" (UID: \"55b4fbae-29dc-41bf-a907-b1412b852cef\") " pod="openshift-ovn-kubernetes/ovnkube-node-75wj4"
Sep 09 20:40:39.936141 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.935413    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0104bf49-c8d9-493a-b189-e570ebf93e62-serviceca\") pod \"node-ca-f8mqt\" (UID: \"0104bf49-c8d9-493a-b189-e570ebf93e62\") " pod="openshift-image-registry/node-ca-f8mqt"
Sep 09 20:40:39.936141 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.935439    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/55b4fbae-29dc-41bf-a907-b1412b852cef-var-lib-openvswitch\") pod \"ovnkube-node-75wj4\" (UID: \"55b4fbae-29dc-41bf-a907-b1412b852cef\") " pod="openshift-ovn-kubernetes/ovnkube-node-75wj4"
Sep 09 20:40:39.936930 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.935466    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/55b4fbae-29dc-41bf-a907-b1412b852cef-host-run-netns\") pod \"ovnkube-node-75wj4\" (UID: \"55b4fbae-29dc-41bf-a907-b1412b852cef\") " pod="openshift-ovn-kubernetes/ovnkube-node-75wj4"
Sep 09 20:40:39.936930 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.935490    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/55b4fbae-29dc-41bf-a907-b1412b852cef-etc-openvswitch\") pod \"ovnkube-node-75wj4\" (UID: \"55b4fbae-29dc-41bf-a907-b1412b852cef\") " pod="openshift-ovn-kubernetes/ovnkube-node-75wj4"
Sep 09 20:40:39.936930 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.935513    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/55b4fbae-29dc-41bf-a907-b1412b852cef-env-overrides\") pod \"ovnkube-node-75wj4\" (UID: \"55b4fbae-29dc-41bf-a907-b1412b852cef\") " pod="openshift-ovn-kubernetes/ovnkube-node-75wj4"
Sep 09 20:40:39.936930 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.935537    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c45d31bc-82ca-49b3-9bfa-9bdae327bef7-var-lib-kubelet\") pod \"tuned-v6pmz\" (UID: \"c45d31bc-82ca-49b3-9bfa-9bdae327bef7\") " pod="openshift-cluster-node-tuning-operator/tuned-v6pmz"
Sep 09 20:40:39.936930 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.935560    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c45d31bc-82ca-49b3-9bfa-9bdae327bef7-etc-tuned\") pod \"tuned-v6pmz\" (UID: \"c45d31bc-82ca-49b3-9bfa-9bdae327bef7\") " pod="openshift-cluster-node-tuning-operator/tuned-v6pmz"
Sep 09 20:40:39.936930 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.935567    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/82c490e6-edea-4f5c-9fe5-111fe55ccee9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-p6642\" (UID: \"82c490e6-edea-4f5c-9fe5-111fe55ccee9\") " pod="openshift-multus/multus-additional-cni-plugins-p6642"
Sep 09 20:40:39.936930 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.935583    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2xwb\" (UniqueName: \"kubernetes.io/projected/10602efd-a422-4354-9f8e-bc58dd9ec7dd-kube-api-access-z2xwb\") pod \"aws-ebs-csi-driver-node-jq8ls\" (UID: \"10602efd-a422-4354-9f8e-bc58dd9ec7dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jq8ls"
Sep 09 20:40:39.936930 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.935616    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/82c490e6-edea-4f5c-9fe5-111fe55ccee9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-p6642\" (UID: \"82c490e6-edea-4f5c-9fe5-111fe55ccee9\") " pod="openshift-multus/multus-additional-cni-plugins-p6642"
Sep 09 20:40:39.936930 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.935632    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/55b4fbae-29dc-41bf-a907-b1412b852cef-var-lib-openvswitch\") pod \"ovnkube-node-75wj4\" (UID: \"55b4fbae-29dc-41bf-a907-b1412b852cef\") " pod="openshift-ovn-kubernetes/ovnkube-node-75wj4"
Sep 09 20:40:39.936930 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.935712    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0bd19a8a-4d11-4e06-b2f9-7f04ccd35574-konnectivity-ca\") pod \"konnectivity-agent-lx79f\" (UID: \"0bd19a8a-4d11-4e06-b2f9-7f04ccd35574\") " pod="kube-system/konnectivity-agent-lx79f"
Sep 09 20:40:39.936930 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.936041    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/55b4fbae-29dc-41bf-a907-b1412b852cef-env-overrides\") pod \"ovnkube-node-75wj4\" (UID: \"55b4fbae-29dc-41bf-a907-b1412b852cef\") " pod="openshift-ovn-kubernetes/ovnkube-node-75wj4"
Sep 09 20:40:39.936930 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.936119    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/82c490e6-edea-4f5c-9fe5-111fe55ccee9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-p6642\" (UID: \"82c490e6-edea-4f5c-9fe5-111fe55ccee9\") " pod="openshift-multus/multus-additional-cni-plugins-p6642"
Sep 09 20:40:39.936930 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.936129    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/55b4fbae-29dc-41bf-a907-b1412b852cef-ovnkube-config\") pod \"ovnkube-node-75wj4\" (UID: \"55b4fbae-29dc-41bf-a907-b1412b852cef\") " pod="openshift-ovn-kubernetes/ovnkube-node-75wj4"
Sep 09 20:40:39.936930 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.936180    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c45d31bc-82ca-49b3-9bfa-9bdae327bef7-var-lib-kubelet\") pod \"tuned-v6pmz\" (UID: \"c45d31bc-82ca-49b3-9bfa-9bdae327bef7\") " pod="openshift-cluster-node-tuning-operator/tuned-v6pmz"
Sep 09 20:40:39.936930 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.936182    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/55b4fbae-29dc-41bf-a907-b1412b852cef-etc-openvswitch\") pod \"ovnkube-node-75wj4\" (UID: \"55b4fbae-29dc-41bf-a907-b1412b852cef\") " pod="openshift-ovn-kubernetes/ovnkube-node-75wj4"
Sep 09 20:40:39.936930 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.936220    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/55b4fbae-29dc-41bf-a907-b1412b852cef-host-kubelet\") pod \"ovnkube-node-75wj4\" (UID: \"55b4fbae-29dc-41bf-a907-b1412b852cef\") " pod="openshift-ovn-kubernetes/ovnkube-node-75wj4"
Sep 09 20:40:39.936930 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.936221    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/82c490e6-edea-4f5c-9fe5-111fe55ccee9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-p6642\" (UID: \"82c490e6-edea-4f5c-9fe5-111fe55ccee9\") " pod="openshift-multus/multus-additional-cni-plugins-p6642"
Sep 09 20:40:39.937770 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.936231    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/55b4fbae-29dc-41bf-a907-b1412b852cef-host-run-netns\") pod \"ovnkube-node-75wj4\" (UID: \"55b4fbae-29dc-41bf-a907-b1412b852cef\") " pod="openshift-ovn-kubernetes/ovnkube-node-75wj4"
Sep 09 20:40:39.937770 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.936263    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/55b4fbae-29dc-41bf-a907-b1412b852cef-host-slash\") pod \"ovnkube-node-75wj4\" (UID: \"55b4fbae-29dc-41bf-a907-b1412b852cef\") " pod="openshift-ovn-kubernetes/ovnkube-node-75wj4"
Sep 09 20:40:39.937770 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.936273    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/55b4fbae-29dc-41bf-a907-b1412b852cef-host-kubelet\") pod \"ovnkube-node-75wj4\" (UID: \"55b4fbae-29dc-41bf-a907-b1412b852cef\") " pod="openshift-ovn-kubernetes/ovnkube-node-75wj4"
Sep 09 20:40:39.937770 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.936298    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c45d31bc-82ca-49b3-9bfa-9bdae327bef7-etc-modprobe-d\") pod \"tuned-v6pmz\" (UID: \"c45d31bc-82ca-49b3-9bfa-9bdae327bef7\") " pod="openshift-cluster-node-tuning-operator/tuned-v6pmz"
Sep 09 20:40:39.937770 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.936310    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/55b4fbae-29dc-41bf-a907-b1412b852cef-host-slash\") pod \"ovnkube-node-75wj4\" (UID: \"55b4fbae-29dc-41bf-a907-b1412b852cef\") " pod="openshift-ovn-kubernetes/ovnkube-node-75wj4"
Sep 09 20:40:39.937770 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.936324    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0104bf49-c8d9-493a-b189-e570ebf93e62-host\") pod \"node-ca-f8mqt\" (UID: \"0104bf49-c8d9-493a-b189-e570ebf93e62\") " pod="openshift-image-registry/node-ca-f8mqt"
Sep 09 20:40:39.937770 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.936347    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/10602efd-a422-4354-9f8e-bc58dd9ec7dd-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jq8ls\" (UID: \"10602efd-a422-4354-9f8e-bc58dd9ec7dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jq8ls"
Sep 09 20:40:39.937770 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.936369    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5096efd2-6fbf-45a2-930b-acdc3970eabd-hosts-file\") pod \"node-resolver-cfg6s\" (UID: \"5096efd2-6fbf-45a2-930b-acdc3970eabd\") " pod="openshift-dns/node-resolver-cfg6s"
Sep 09 20:40:39.937770 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.936386    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c45d31bc-82ca-49b3-9bfa-9bdae327bef7-etc-modprobe-d\") pod \"tuned-v6pmz\" (UID: \"c45d31bc-82ca-49b3-9bfa-9bdae327bef7\") " pod="openshift-cluster-node-tuning-operator/tuned-v6pmz"
Sep 09 20:40:39.937770 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.936388    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0104bf49-c8d9-493a-b189-e570ebf93e62-serviceca\") pod \"node-ca-f8mqt\" (UID: \"0104bf49-c8d9-493a-b189-e570ebf93e62\") " pod="openshift-image-registry/node-ca-f8mqt"
Sep 09 20:40:39.937770 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.936391    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/82c490e6-edea-4f5c-9fe5-111fe55ccee9-cnibin\") pod \"multus-additional-cni-plugins-p6642\" (UID: \"82c490e6-edea-4f5c-9fe5-111fe55ccee9\") " pod="openshift-multus/multus-additional-cni-plugins-p6642"
Sep 09 20:40:39.937770 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.936424    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/82c490e6-edea-4f5c-9fe5-111fe55ccee9-cnibin\") pod \"multus-additional-cni-plugins-p6642\" (UID: \"82c490e6-edea-4f5c-9fe5-111fe55ccee9\") " pod="openshift-multus/multus-additional-cni-plugins-p6642"
Sep 09 20:40:39.937770 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.936431    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0104bf49-c8d9-493a-b189-e570ebf93e62-host\") pod \"node-ca-f8mqt\" (UID: \"0104bf49-c8d9-493a-b189-e570ebf93e62\") " pod="openshift-image-registry/node-ca-f8mqt"
Sep 09 20:40:39.937770 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.936445    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/10602efd-a422-4354-9f8e-bc58dd9ec7dd-etc-selinux\") pod \"aws-ebs-csi-driver-node-jq8ls\" (UID: \"10602efd-a422-4354-9f8e-bc58dd9ec7dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jq8ls"
Sep 09 20:40:39.937770 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.936464    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5096efd2-6fbf-45a2-930b-acdc3970eabd-hosts-file\") pod \"node-resolver-cfg6s\" (UID: \"5096efd2-6fbf-45a2-930b-acdc3970eabd\") " pod="openshift-dns/node-resolver-cfg6s"
Sep 09 20:40:39.937770 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.936473    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/55b4fbae-29dc-41bf-a907-b1412b852cef-run-systemd\") pod \"ovnkube-node-75wj4\" (UID: \"55b4fbae-29dc-41bf-a907-b1412b852cef\") " pod="openshift-ovn-kubernetes/ovnkube-node-75wj4"
Sep 09 20:40:39.937770 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.936496    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/55b4fbae-29dc-41bf-a907-b1412b852cef-host-cni-bin\") pod \"ovnkube-node-75wj4\" (UID: \"55b4fbae-29dc-41bf-a907-b1412b852cef\") " pod="openshift-ovn-kubernetes/ovnkube-node-75wj4"
Sep 09 20:40:39.937770 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.936511    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/55b4fbae-29dc-41bf-a907-b1412b852cef-run-systemd\") pod \"ovnkube-node-75wj4\" (UID: \"55b4fbae-29dc-41bf-a907-b1412b852cef\") " pod="openshift-ovn-kubernetes/ovnkube-node-75wj4"
Sep 09 20:40:39.938372 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.936519    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4vmc\" (UniqueName: \"kubernetes.io/projected/55b4fbae-29dc-41bf-a907-b1412b852cef-kube-api-access-d4vmc\") pod \"ovnkube-node-75wj4\" (UID: \"55b4fbae-29dc-41bf-a907-b1412b852cef\") " pod="openshift-ovn-kubernetes/ovnkube-node-75wj4"
Sep 09 20:40:39.938372 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.936515    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/10602efd-a422-4354-9f8e-bc58dd9ec7dd-etc-selinux\") pod \"aws-ebs-csi-driver-node-jq8ls\" (UID: \"10602efd-a422-4354-9f8e-bc58dd9ec7dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jq8ls"
Sep 09 20:40:39.938372 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.936545    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c45d31bc-82ca-49b3-9bfa-9bdae327bef7-run\") pod \"tuned-v6pmz\" (UID: \"c45d31bc-82ca-49b3-9bfa-9bdae327bef7\") " pod="openshift-cluster-node-tuning-operator/tuned-v6pmz"
Sep 09 20:40:39.938372 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.936473    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/10602efd-a422-4354-9f8e-bc58dd9ec7dd-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jq8ls\" (UID: \"10602efd-a422-4354-9f8e-bc58dd9ec7dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jq8ls"
Sep 09 20:40:39.938372 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.936549    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/55b4fbae-29dc-41bf-a907-b1412b852cef-host-cni-bin\") pod \"ovnkube-node-75wj4\" (UID: \"55b4fbae-29dc-41bf-a907-b1412b852cef\") " pod="openshift-ovn-kubernetes/ovnkube-node-75wj4"
Sep 09 20:40:39.938372 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.936512    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5096efd2-6fbf-45a2-930b-acdc3970eabd-tmp-dir\") pod \"node-resolver-cfg6s\" (UID: \"5096efd2-6fbf-45a2-930b-acdc3970eabd\") " pod="openshift-dns/node-resolver-cfg6s"
Sep 09 20:40:39.938372 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.936568    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/10602efd-a422-4354-9f8e-bc58dd9ec7dd-device-dir\") pod \"aws-ebs-csi-driver-node-jq8ls\" (UID: \"10602efd-a422-4354-9f8e-bc58dd9ec7dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jq8ls"
Sep 09 20:40:39.938372 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.936586    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c45d31bc-82ca-49b3-9bfa-9bdae327bef7-run\") pod \"tuned-v6pmz\" (UID: \"c45d31bc-82ca-49b3-9bfa-9bdae327bef7\") " pod="openshift-cluster-node-tuning-operator/tuned-v6pmz"
Sep 09 20:40:39.938372 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.936597    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/10602efd-a422-4354-9f8e-bc58dd9ec7dd-registration-dir\") pod \"aws-ebs-csi-driver-node-jq8ls\" (UID: \"10602efd-a422-4354-9f8e-bc58dd9ec7dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jq8ls"
Sep 09 20:40:39.938372 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.936622    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/55b4fbae-29dc-41bf-a907-b1412b852cef-run-ovn\") pod \"ovnkube-node-75wj4\" (UID: \"55b4fbae-29dc-41bf-a907-b1412b852cef\") " pod="openshift-ovn-kubernetes/ovnkube-node-75wj4"
Sep 09 20:40:39.938372 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.936637    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/10602efd-a422-4354-9f8e-bc58dd9ec7dd-device-dir\") pod \"aws-ebs-csi-driver-node-jq8ls\" (UID: \"10602efd-a422-4354-9f8e-bc58dd9ec7dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jq8ls"
Sep 09 20:40:39.938372 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.936645    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/55b4fbae-29dc-41bf-a907-b1412b852cef-host-run-ovn-kubernetes\") pod \"ovnkube-node-75wj4\" (UID: \"55b4fbae-29dc-41bf-a907-b1412b852cef\") " pod="openshift-ovn-kubernetes/ovnkube-node-75wj4"
Sep 09 20:40:39.938372 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.936653    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/10602efd-a422-4354-9f8e-bc58dd9ec7dd-registration-dir\") pod \"aws-ebs-csi-driver-node-jq8ls\" (UID: \"10602efd-a422-4354-9f8e-bc58dd9ec7dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jq8ls"
Sep 09 20:40:39.938372 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.936668    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c45d31bc-82ca-49b3-9bfa-9bdae327bef7-etc-sysconfig\") pod \"tuned-v6pmz\" (UID: \"c45d31bc-82ca-49b3-9bfa-9bdae327bef7\") " pod="openshift-cluster-node-tuning-operator/tuned-v6pmz"
Sep 09 20:40:39.938372 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.936676    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/55b4fbae-29dc-41bf-a907-b1412b852cef-run-ovn\") pod \"ovnkube-node-75wj4\" (UID: \"55b4fbae-29dc-41bf-a907-b1412b852cef\") " pod="openshift-ovn-kubernetes/ovnkube-node-75wj4"
Sep 09 20:40:39.938372 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.936695    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/55b4fbae-29dc-41bf-a907-b1412b852cef-host-run-ovn-kubernetes\") pod \"ovnkube-node-75wj4\" (UID: \"55b4fbae-29dc-41bf-a907-b1412b852cef\") " pod="openshift-ovn-kubernetes/ovnkube-node-75wj4"
Sep 09 20:40:39.938372 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.936738    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c45d31bc-82ca-49b3-9bfa-9bdae327bef7-etc-sysconfig\") pod \"tuned-v6pmz\" (UID: \"c45d31bc-82ca-49b3-9bfa-9bdae327bef7\") " pod="openshift-cluster-node-tuning-operator/tuned-v6pmz"
Sep 09 20:40:39.938880 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.937424    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/55b4fbae-29dc-41bf-a907-b1412b852cef-ovn-node-metrics-cert\") pod \"ovnkube-node-75wj4\" (UID: \"55b4fbae-29dc-41bf-a907-b1412b852cef\") " pod="openshift-ovn-kubernetes/ovnkube-node-75wj4"
Sep 09 20:40:39.938880 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.937816    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0bd19a8a-4d11-4e06-b2f9-7f04ccd35574-agent-certs\") pod \"konnectivity-agent-lx79f\" (UID: \"0bd19a8a-4d11-4e06-b2f9-7f04ccd35574\") " pod="kube-system/konnectivity-agent-lx79f"
Sep 09 20:40:39.938880 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.937867    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c45d31bc-82ca-49b3-9bfa-9bdae327bef7-etc-tuned\") pod \"tuned-v6pmz\" (UID: \"c45d31bc-82ca-49b3-9bfa-9bdae327bef7\") " pod="openshift-cluster-node-tuning-operator/tuned-v6pmz"
Sep 09 20:40:39.938880 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.938388    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c45d31bc-82ca-49b3-9bfa-9bdae327bef7-tmp\") pod \"tuned-v6pmz\" (UID: \"c45d31bc-82ca-49b3-9bfa-9bdae327bef7\") " pod="openshift-cluster-node-tuning-operator/tuned-v6pmz"
Sep 09 20:40:39.942659 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.942637    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzbcs\" (UniqueName: \"kubernetes.io/projected/c45d31bc-82ca-49b3-9bfa-9bdae327bef7-kube-api-access-xzbcs\") pod \"tuned-v6pmz\" (UID: \"c45d31bc-82ca-49b3-9bfa-9bdae327bef7\") " pod="openshift-cluster-node-tuning-operator/tuned-v6pmz"
Sep 09 20:40:39.947437 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:39.947411    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 20:40:39.947437 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:39.947430    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 20:40:39.947437 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:39.947443    2581 projected.go:196] Error preparing data for projected volume kube-api-access-x9gvv for pod openshift-network-diagnostics/network-check-target-2c5hn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 20:40:39.947649 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:39.947495    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f3c30c5-3a66-4a51-ba58-a125dce436ab-kube-api-access-x9gvv podName:2f3c30c5-3a66-4a51-ba58-a125dce436ab nodeName:}" failed. No retries permitted until 2026-09-09 20:40:40.447481414 +0000 UTC m=+2.075109269 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-x9gvv" (UniqueName: "kubernetes.io/projected/2f3c30c5-3a66-4a51-ba58-a125dce436ab-kube-api-access-x9gvv") pod "network-check-target-2c5hn" (UID: "2f3c30c5-3a66-4a51-ba58-a125dce436ab") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 20:40:39.948944 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.948919    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgmj6\" (UniqueName: \"kubernetes.io/projected/82c490e6-edea-4f5c-9fe5-111fe55ccee9-kube-api-access-zgmj6\") pod \"multus-additional-cni-plugins-p6642\" (UID: \"82c490e6-edea-4f5c-9fe5-111fe55ccee9\") " pod="openshift-multus/multus-additional-cni-plugins-p6642"
Sep 09 20:40:39.949326 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.949310    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqgzr\" (UniqueName: \"kubernetes.io/projected/1105e43b-4cb7-4b23-89ba-cf6c5370f185-kube-api-access-zqgzr\") pod \"network-metrics-daemon-2fzgg\" (UID: \"1105e43b-4cb7-4b23-89ba-cf6c5370f185\") " pod="openshift-multus/network-metrics-daemon-2fzgg"
Sep 09 20:40:39.950754 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.950719    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4ljn\" (UniqueName: \"kubernetes.io/projected/5096efd2-6fbf-45a2-930b-acdc3970eabd-kube-api-access-z4ljn\") pod \"node-resolver-cfg6s\" (UID: \"5096efd2-6fbf-45a2-930b-acdc3970eabd\") " pod="openshift-dns/node-resolver-cfg6s"
Sep 09 20:40:39.951177 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.951161    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2xwb\" (UniqueName: \"kubernetes.io/projected/10602efd-a422-4354-9f8e-bc58dd9ec7dd-kube-api-access-z2xwb\") pod \"aws-ebs-csi-driver-node-jq8ls\" (UID: \"10602efd-a422-4354-9f8e-bc58dd9ec7dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jq8ls"
Sep 09 20:40:39.951424 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.951405    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4vmc\" (UniqueName: \"kubernetes.io/projected/55b4fbae-29dc-41bf-a907-b1412b852cef-kube-api-access-d4vmc\") pod \"ovnkube-node-75wj4\" (UID: \"55b4fbae-29dc-41bf-a907-b1412b852cef\") " pod="openshift-ovn-kubernetes/ovnkube-node-75wj4"
Sep 09 20:40:39.951690 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.951674    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7bdq\" (UniqueName: \"kubernetes.io/projected/0104bf49-c8d9-493a-b189-e570ebf93e62-kube-api-access-s7bdq\") pod \"node-ca-f8mqt\" (UID: \"0104bf49-c8d9-493a-b189-e570ebf93e62\") " pod="openshift-image-registry/node-ca-f8mqt"
Sep 09 20:40:39.952377 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:39.952363    2581 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 20:40:40.012959 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:40.012926    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77a4e0e482d02ec9e357614bf125373e.slice/crio-5815c984d9f22f18ae112334dc6ec7c87886dde68af090189451b8802826900e WatchSource:0}: Error finding container 5815c984d9f22f18ae112334dc6ec7c87886dde68af090189451b8802826900e: Status 404 returned error can't find the container with id 5815c984d9f22f18ae112334dc6ec7c87886dde68af090189451b8802826900e
Sep 09 20:40:40.013269 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:40.013249    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75cd1de7188eb1602ddc4211bd282ec4.slice/crio-c34b35ca258650111dc00985d61ee3232714eb87f51b5eba1b54939200c24dbe WatchSource:0}: Error finding container c34b35ca258650111dc00985d61ee3232714eb87f51b5eba1b54939200c24dbe: Status 404 returned error can't find the container with id c34b35ca258650111dc00985d61ee3232714eb87f51b5eba1b54939200c24dbe
Sep 09 20:40:40.017950 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:40.017936    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 20:40:40.060587 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:40.060556    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rj5rk"
Sep 09 20:40:40.061825 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:40.061804    2581 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 20:40:40.070238 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:40.070214    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod964fa8a8_b4c3_45c5_b2bd_6a6d32a6536f.slice/crio-a61b33bc0fb26c73fd3ae3e33186bb440965ccc10ea8092af714c75f34e382ab WatchSource:0}: Error finding container a61b33bc0fb26c73fd3ae3e33186bb440965ccc10ea8092af714c75f34e382ab: Status 404 returned error can't find the container with id a61b33bc0fb26c73fd3ae3e33186bb440965ccc10ea8092af714c75f34e382ab
Sep 09 20:40:40.073425 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:40.073408    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-ft4fb"
Sep 09 20:40:40.081791 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:40.081770    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1d25b48_1c88_4b60_93a1_665b90df19da.slice/crio-f2023a254dfc9c044801df4337d9cf73a501bf5063dfa09a80925e832454b34a WatchSource:0}: Error finding container f2023a254dfc9c044801df4337d9cf73a501bf5063dfa09a80925e832454b34a: Status 404 returned error can't find the container with id f2023a254dfc9c044801df4337d9cf73a501bf5063dfa09a80925e832454b34a
Sep 09 20:40:40.094028 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:40.094000    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-75wj4"
Sep 09 20:40:40.102011 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:40.101984    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55b4fbae_29dc_41bf_a907_b1412b852cef.slice/crio-a6f6a2869c59555670d91589ff7a7fc4e35ed39493ab3217c88ae47d65a6bdfd WatchSource:0}: Error finding container a6f6a2869c59555670d91589ff7a7fc4e35ed39493ab3217c88ae47d65a6bdfd: Status 404 returned error can't find the container with id a6f6a2869c59555670d91589ff7a7fc4e35ed39493ab3217c88ae47d65a6bdfd
Sep 09 20:40:40.109832 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:40.109812    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-lx79f"
Sep 09 20:40:40.117293 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:40.117272    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bd19a8a_4d11_4e06_b2f9_7f04ccd35574.slice/crio-b8d40031c64f32dd54572472fa446b0de86697b220a3d1c42bf2a493d325fc9d WatchSource:0}: Error finding container b8d40031c64f32dd54572472fa446b0de86697b220a3d1c42bf2a493d325fc9d: Status 404 returned error can't find the container with id b8d40031c64f32dd54572472fa446b0de86697b220a3d1c42bf2a493d325fc9d
Sep 09 20:40:40.123195 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:40.123177    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-v6pmz"
Sep 09 20:40:40.130813 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:40.130789    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc45d31bc_82ca_49b3_9bfa_9bdae327bef7.slice/crio-4a74309361563f02052102df9d477b7dc5efdd049498a85750d5597abe52cdac WatchSource:0}: Error finding container 4a74309361563f02052102df9d477b7dc5efdd049498a85750d5597abe52cdac: Status 404 returned error can't find the container with id 4a74309361563f02052102df9d477b7dc5efdd049498a85750d5597abe52cdac
Sep 09 20:40:40.140691 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:40.140673    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-f8mqt"
Sep 09 20:40:40.148388 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:40.148368    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0104bf49_c8d9_493a_b189_e570ebf93e62.slice/crio-b47710205c23347d8ecf049ced354495959e5bff71b761771efa0de73d78b4dd WatchSource:0}: Error finding container b47710205c23347d8ecf049ced354495959e5bff71b761771efa0de73d78b4dd: Status 404 returned error can't find the container with id b47710205c23347d8ecf049ced354495959e5bff71b761771efa0de73d78b4dd
Sep 09 20:40:40.157026 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:40.157008    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-cfg6s"
Sep 09 20:40:40.164985 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:40.164963    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5096efd2_6fbf_45a2_930b_acdc3970eabd.slice/crio-fc5040f435722e88900aea8dd82ba83f29dc019a51b9659c3a414ad748702022 WatchSource:0}: Error finding container fc5040f435722e88900aea8dd82ba83f29dc019a51b9659c3a414ad748702022: Status 404 returned error can't find the container with id fc5040f435722e88900aea8dd82ba83f29dc019a51b9659c3a414ad748702022
Sep 09 20:40:40.173948 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:40.173932    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-p6642"
Sep 09 20:40:40.180619 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:40.180605    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jq8ls"
Sep 09 20:40:40.182609 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:40.182590    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82c490e6_edea_4f5c_9fe5_111fe55ccee9.slice/crio-ad7b9465a7c4f46f5550b6eed548778dff64b5401b5252ccb0a5c746574f83a7 WatchSource:0}: Error finding container ad7b9465a7c4f46f5550b6eed548778dff64b5401b5252ccb0a5c746574f83a7: Status 404 returned error can't find the container with id ad7b9465a7c4f46f5550b6eed548778dff64b5401b5252ccb0a5c746574f83a7
Sep 09 20:40:40.189444 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:40:40.189424    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10602efd_a422_4354_9f8e_bc58dd9ec7dd.slice/crio-d28e95a66637c142358e07faa72eaa50a75786e2fdd9b997ccd4fe8f3085fc0d WatchSource:0}: Error finding container d28e95a66637c142358e07faa72eaa50a75786e2fdd9b997ccd4fe8f3085fc0d: Status 404 returned error can't find the container with id d28e95a66637c142358e07faa72eaa50a75786e2fdd9b997ccd4fe8f3085fc0d
Sep 09 20:40:40.191007 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:40.190989    2581 kuberuntime_manager.go:1459] "Unhandled Error" err="container csi-driver start failed in pod aws-ebs-csi-driver-node-jq8ls_openshift-cluster-csi-drivers(10602efd-a422-4354-9f8e-bc58dd9ec7dd): ErrImagePull: pull QPS exceeded" logger="UnhandledError"
Sep 09 20:40:40.191757 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:40.191738    2581 kuberuntime_manager.go:1459] "Unhandled Error" err="container csi-node-driver-registrar start failed in pod aws-ebs-csi-driver-node-jq8ls_openshift-cluster-csi-drivers(10602efd-a422-4354-9f8e-bc58dd9ec7dd): ErrImagePull: pull QPS exceeded" logger="UnhandledError"
Sep 09 20:40:40.193480 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:40.193464    2581 kuberuntime_manager.go:1459] "Unhandled Error" err="container csi-liveness-probe start failed in pod aws-ebs-csi-driver-node-jq8ls_openshift-cluster-csi-drivers(10602efd-a422-4354-9f8e-bc58dd9ec7dd): ErrImagePull: pull QPS exceeded" logger="UnhandledError"
Sep 09 20:40:40.193533 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:40.193507    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="[failed to \"StartContainer\" for \"csi-driver\" with ErrImagePull: \"pull QPS exceeded\", failed to \"StartContainer\" for \"csi-node-driver-registrar\" with ErrImagePull: \"pull QPS exceeded\", failed to \"StartContainer\" for \"csi-liveness-probe\" with ErrImagePull: \"pull QPS exceeded\"]" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jq8ls" podUID="10602efd-a422-4354-9f8e-bc58dd9ec7dd"
Sep 09 20:40:40.440379 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:40.439925    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1105e43b-4cb7-4b23-89ba-cf6c5370f185-metrics-certs\") pod \"network-metrics-daemon-2fzgg\" (UID: \"1105e43b-4cb7-4b23-89ba-cf6c5370f185\") " pod="openshift-multus/network-metrics-daemon-2fzgg"
Sep 09 20:40:40.440379 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:40.440199    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ad64d253-c155-4e3c-b872-cdebb0616a0e-original-pull-secret\") pod \"global-pull-secret-syncer-bjwzq\" (UID: \"ad64d253-c155-4e3c-b872-cdebb0616a0e\") " pod="kube-system/global-pull-secret-syncer-bjwzq"
Sep 09 20:40:40.440379 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:40.440059    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 20:40:40.440633 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:40.440387    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1105e43b-4cb7-4b23-89ba-cf6c5370f185-metrics-certs podName:1105e43b-4cb7-4b23-89ba-cf6c5370f185 nodeName:}" failed. No retries permitted until 2026-09-09 20:40:41.440369476 +0000 UTC m=+3.067997343 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1105e43b-4cb7-4b23-89ba-cf6c5370f185-metrics-certs") pod "network-metrics-daemon-2fzgg" (UID: "1105e43b-4cb7-4b23-89ba-cf6c5370f185") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 20:40:40.440739 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:40.440327    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 20:40:40.440869 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:40.440817    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad64d253-c155-4e3c-b872-cdebb0616a0e-original-pull-secret podName:ad64d253-c155-4e3c-b872-cdebb0616a0e nodeName:}" failed. No retries permitted until 2026-09-09 20:40:41.440792483 +0000 UTC m=+3.068420347 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ad64d253-c155-4e3c-b872-cdebb0616a0e-original-pull-secret") pod "global-pull-secret-syncer-bjwzq" (UID: "ad64d253-c155-4e3c-b872-cdebb0616a0e") : object "kube-system"/"original-pull-secret" not registered
Sep 09 20:40:40.542030 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:40.541407    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9gvv\" (UniqueName: \"kubernetes.io/projected/2f3c30c5-3a66-4a51-ba58-a125dce436ab-kube-api-access-x9gvv\") pod \"network-check-target-2c5hn\" (UID: \"2f3c30c5-3a66-4a51-ba58-a125dce436ab\") " pod="openshift-network-diagnostics/network-check-target-2c5hn"
Sep 09 20:40:40.542030 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:40.541606    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 20:40:40.542030 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:40.541624    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 20:40:40.542030 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:40.541636    2581 projected.go:196] Error preparing data for projected volume kube-api-access-x9gvv for pod openshift-network-diagnostics/network-check-target-2c5hn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 20:40:40.542030 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:40.541687    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f3c30c5-3a66-4a51-ba58-a125dce436ab-kube-api-access-x9gvv podName:2f3c30c5-3a66-4a51-ba58-a125dce436ab nodeName:}" failed. No retries permitted until 2026-09-09 20:40:41.541670311 +0000 UTC m=+3.169298188 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-x9gvv" (UniqueName: "kubernetes.io/projected/2f3c30c5-3a66-4a51-ba58-a125dce436ab-kube-api-access-x9gvv") pod "network-check-target-2c5hn" (UID: "2f3c30c5-3a66-4a51-ba58-a125dce436ab") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 20:40:40.590317 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:40.590286    2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 20:40:40.777281 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:40.777085    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-08 20:35:39 +0000 UTC" deadline="2028-05-03 14:08:16.518122985 +0000 UTC"
Sep 09 20:40:40.777281 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:40.777155    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14441h27m35.740972721s"
Sep 09 20:40:40.829718 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:40.828987    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2fzgg"
Sep 09 20:40:40.829718 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:40.829135    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2fzgg" podUID="1105e43b-4cb7-4b23-89ba-cf6c5370f185"
Sep 09 20:40:40.834262 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:40.834145    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-cfg6s" event={"ID":"5096efd2-6fbf-45a2-930b-acdc3970eabd","Type":"ContainerStarted","Data":"fc5040f435722e88900aea8dd82ba83f29dc019a51b9659c3a414ad748702022"}
Sep 09 20:40:40.840391 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:40.840355    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-f8mqt" event={"ID":"0104bf49-c8d9-493a-b189-e570ebf93e62","Type":"ContainerStarted","Data":"b47710205c23347d8ecf049ced354495959e5bff71b761771efa0de73d78b4dd"}
Sep 09 20:40:40.846993 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:40.846962    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-v6pmz" event={"ID":"c45d31bc-82ca-49b3-9bfa-9bdae327bef7","Type":"ContainerStarted","Data":"4a74309361563f02052102df9d477b7dc5efdd049498a85750d5597abe52cdac"}
Sep 09 20:40:40.872995 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:40.872957    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-lx79f" event={"ID":"0bd19a8a-4d11-4e06-b2f9-7f04ccd35574","Type":"ContainerStarted","Data":"b8d40031c64f32dd54572472fa446b0de86697b220a3d1c42bf2a493d325fc9d"}
Sep 09 20:40:40.877702 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:40.877670    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rj5rk" event={"ID":"964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f","Type":"ContainerStarted","Data":"a61b33bc0fb26c73fd3ae3e33186bb440965ccc10ea8092af714c75f34e382ab"}
Sep 09 20:40:40.886010 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:40.885966    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-99.ec2.internal" event={"ID":"77a4e0e482d02ec9e357614bf125373e","Type":"ContainerStarted","Data":"5815c984d9f22f18ae112334dc6ec7c87886dde68af090189451b8802826900e"}
Sep 09 20:40:40.897506 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:40.897457    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-75wj4" event={"ID":"55b4fbae-29dc-41bf-a907-b1412b852cef","Type":"ContainerStarted","Data":"a6f6a2869c59555670d91589ff7a7fc4e35ed39493ab3217c88ae47d65a6bdfd"}
Sep 09 20:40:40.899435 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:40.899409    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-ft4fb" event={"ID":"d1d25b48-1c88-4b60-93a1-665b90df19da","Type":"ContainerStarted","Data":"f2023a254dfc9c044801df4337d9cf73a501bf5063dfa09a80925e832454b34a"}
Sep 09 20:40:40.916633 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:40.916578    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-99.ec2.internal" event={"ID":"75cd1de7188eb1602ddc4211bd282ec4","Type":"ContainerStarted","Data":"c34b35ca258650111dc00985d61ee3232714eb87f51b5eba1b54939200c24dbe"}
Sep 09 20:40:40.931119 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:40.931068    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jq8ls" event={"ID":"10602efd-a422-4354-9f8e-bc58dd9ec7dd","Type":"ContainerStarted","Data":"d28e95a66637c142358e07faa72eaa50a75786e2fdd9b997ccd4fe8f3085fc0d"}
Sep 09 20:40:40.944501 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:40.944471    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p6642" event={"ID":"82c490e6-edea-4f5c-9fe5-111fe55ccee9","Type":"ContainerStarted","Data":"ad7b9465a7c4f46f5550b6eed548778dff64b5401b5252ccb0a5c746574f83a7"}
Sep 09 20:40:40.945032 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:40.944992    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="[failed to \"StartContainer\" for \"csi-driver\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:442f6f711db8a6f8fae4dfff48a29e5fb0d154b2c6779c5e2e46d649f99ccfec\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"csi-node-driver-registrar\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:909d94acdf16bd26857849698608cb17db0c62ca3eab7d670ee858a6e839390e\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"csi-liveness-probe\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4e3a2f3c9f8d7a459ccd0719643fba2d050d158d387ade828c3d58813bc3b6e4\\\": ErrImagePull: pull QPS exceeded\"]" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jq8ls" podUID="10602efd-a422-4354-9f8e-bc58dd9ec7dd"
Sep 09 20:40:41.448362 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:41.448325    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1105e43b-4cb7-4b23-89ba-cf6c5370f185-metrics-certs\") pod \"network-metrics-daemon-2fzgg\" (UID: \"1105e43b-4cb7-4b23-89ba-cf6c5370f185\") " pod="openshift-multus/network-metrics-daemon-2fzgg"
Sep 09 20:40:41.448547 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:41.448380    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ad64d253-c155-4e3c-b872-cdebb0616a0e-original-pull-secret\") pod \"global-pull-secret-syncer-bjwzq\" (UID: \"ad64d253-c155-4e3c-b872-cdebb0616a0e\") " pod="kube-system/global-pull-secret-syncer-bjwzq"
Sep 09 20:40:41.448607 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:41.448544    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 20:40:41.448607 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:41.448602    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad64d253-c155-4e3c-b872-cdebb0616a0e-original-pull-secret podName:ad64d253-c155-4e3c-b872-cdebb0616a0e nodeName:}" failed. No retries permitted until 2026-09-09 20:40:43.448584045 +0000 UTC m=+5.076211897 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ad64d253-c155-4e3c-b872-cdebb0616a0e-original-pull-secret") pod "global-pull-secret-syncer-bjwzq" (UID: "ad64d253-c155-4e3c-b872-cdebb0616a0e") : object "kube-system"/"original-pull-secret" not registered
Sep 09 20:40:41.448985 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:41.448968    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 20:40:41.449090 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:41.449018    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1105e43b-4cb7-4b23-89ba-cf6c5370f185-metrics-certs podName:1105e43b-4cb7-4b23-89ba-cf6c5370f185 nodeName:}" failed. No retries permitted until 2026-09-09 20:40:43.449003752 +0000 UTC m=+5.076631609 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1105e43b-4cb7-4b23-89ba-cf6c5370f185-metrics-certs") pod "network-metrics-daemon-2fzgg" (UID: "1105e43b-4cb7-4b23-89ba-cf6c5370f185") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 20:40:41.550418 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:41.549713    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9gvv\" (UniqueName: \"kubernetes.io/projected/2f3c30c5-3a66-4a51-ba58-a125dce436ab-kube-api-access-x9gvv\") pod \"network-check-target-2c5hn\" (UID: \"2f3c30c5-3a66-4a51-ba58-a125dce436ab\") " pod="openshift-network-diagnostics/network-check-target-2c5hn"
Sep 09 20:40:41.550418 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:41.549922    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 20:40:41.550418 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:41.549948    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 20:40:41.550418 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:41.549960    2581 projected.go:196] Error preparing data for projected volume kube-api-access-x9gvv for pod openshift-network-diagnostics/network-check-target-2c5hn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 20:40:41.550418 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:41.550053    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f3c30c5-3a66-4a51-ba58-a125dce436ab-kube-api-access-x9gvv podName:2f3c30c5-3a66-4a51-ba58-a125dce436ab nodeName:}" failed. No retries permitted until 2026-09-09 20:40:43.550033755 +0000 UTC m=+5.177661620 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-x9gvv" (UniqueName: "kubernetes.io/projected/2f3c30c5-3a66-4a51-ba58-a125dce436ab-kube-api-access-x9gvv") pod "network-check-target-2c5hn" (UID: "2f3c30c5-3a66-4a51-ba58-a125dce436ab") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 20:40:41.778212 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:41.778116    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-08 20:35:39 +0000 UTC" deadline="2028-04-30 03:19:53.36632255 +0000 UTC"
Sep 09 20:40:41.778212 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:41.778153    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14358h39m11.588173386s"
Sep 09 20:40:41.814495 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:41.814459    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bjwzq"
Sep 09 20:40:41.814680 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:41.814603    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bjwzq" podUID="ad64d253-c155-4e3c-b872-cdebb0616a0e"
Sep 09 20:40:41.815014 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:41.814989    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2c5hn"
Sep 09 20:40:41.815154 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:41.815084    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2c5hn" podUID="2f3c30c5-3a66-4a51-ba58-a125dce436ab"
Sep 09 20:40:41.937966 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:41.937938    2581 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 20:40:42.819035 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:42.819007    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2fzgg"
Sep 09 20:40:42.819512 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:42.819155    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2fzgg" podUID="1105e43b-4cb7-4b23-89ba-cf6c5370f185"
Sep 09 20:40:43.466408 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:43.466373    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1105e43b-4cb7-4b23-89ba-cf6c5370f185-metrics-certs\") pod \"network-metrics-daemon-2fzgg\" (UID: \"1105e43b-4cb7-4b23-89ba-cf6c5370f185\") " pod="openshift-multus/network-metrics-daemon-2fzgg"
Sep 09 20:40:43.466607 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:43.466423    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ad64d253-c155-4e3c-b872-cdebb0616a0e-original-pull-secret\") pod \"global-pull-secret-syncer-bjwzq\" (UID: \"ad64d253-c155-4e3c-b872-cdebb0616a0e\") " pod="kube-system/global-pull-secret-syncer-bjwzq"
Sep 09 20:40:43.466607 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:43.466540    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 20:40:43.466607 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:43.466562    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 20:40:43.466826 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:43.466621    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1105e43b-4cb7-4b23-89ba-cf6c5370f185-metrics-certs podName:1105e43b-4cb7-4b23-89ba-cf6c5370f185 nodeName:}" failed. No retries permitted until 2026-09-09 20:40:47.466600389 +0000 UTC m=+9.094228262 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1105e43b-4cb7-4b23-89ba-cf6c5370f185-metrics-certs") pod "network-metrics-daemon-2fzgg" (UID: "1105e43b-4cb7-4b23-89ba-cf6c5370f185") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 20:40:43.466826 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:43.466644    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad64d253-c155-4e3c-b872-cdebb0616a0e-original-pull-secret podName:ad64d253-c155-4e3c-b872-cdebb0616a0e nodeName:}" failed. No retries permitted until 2026-09-09 20:40:47.466634295 +0000 UTC m=+9.094262147 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ad64d253-c155-4e3c-b872-cdebb0616a0e-original-pull-secret") pod "global-pull-secret-syncer-bjwzq" (UID: "ad64d253-c155-4e3c-b872-cdebb0616a0e") : object "kube-system"/"original-pull-secret" not registered
Sep 09 20:40:43.567080 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:43.567041    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9gvv\" (UniqueName: \"kubernetes.io/projected/2f3c30c5-3a66-4a51-ba58-a125dce436ab-kube-api-access-x9gvv\") pod \"network-check-target-2c5hn\" (UID: \"2f3c30c5-3a66-4a51-ba58-a125dce436ab\") " pod="openshift-network-diagnostics/network-check-target-2c5hn"
Sep 09 20:40:43.567298 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:43.567278    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 20:40:43.567371 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:43.567308    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 20:40:43.567371 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:43.567324    2581 projected.go:196] Error preparing data for projected volume kube-api-access-x9gvv for pod openshift-network-diagnostics/network-check-target-2c5hn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 20:40:43.567490 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:43.567386    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f3c30c5-3a66-4a51-ba58-a125dce436ab-kube-api-access-x9gvv podName:2f3c30c5-3a66-4a51-ba58-a125dce436ab nodeName:}" failed. No retries permitted until 2026-09-09 20:40:47.567366184 +0000 UTC m=+9.194994058 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-x9gvv" (UniqueName: "kubernetes.io/projected/2f3c30c5-3a66-4a51-ba58-a125dce436ab-kube-api-access-x9gvv") pod "network-check-target-2c5hn" (UID: "2f3c30c5-3a66-4a51-ba58-a125dce436ab") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 20:40:43.814131 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:43.814037    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bjwzq"
Sep 09 20:40:43.814131 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:43.814050    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2c5hn"
Sep 09 20:40:43.814345 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:43.814215    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bjwzq" podUID="ad64d253-c155-4e3c-b872-cdebb0616a0e"
Sep 09 20:40:43.814345 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:43.814308    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2c5hn" podUID="2f3c30c5-3a66-4a51-ba58-a125dce436ab"
Sep 09 20:40:44.818792 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:44.818762    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2fzgg"
Sep 09 20:40:44.819258 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:44.818890    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2fzgg" podUID="1105e43b-4cb7-4b23-89ba-cf6c5370f185"
Sep 09 20:40:45.815168 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:45.814468    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2c5hn"
Sep 09 20:40:45.815168 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:45.814597    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2c5hn" podUID="2f3c30c5-3a66-4a51-ba58-a125dce436ab"
Sep 09 20:40:45.815168 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:45.815016    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bjwzq"
Sep 09 20:40:45.815168 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:45.815120    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bjwzq" podUID="ad64d253-c155-4e3c-b872-cdebb0616a0e"
Sep 09 20:40:46.814389 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:46.814357    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2fzgg"
Sep 09 20:40:46.814838 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:46.814525    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2fzgg" podUID="1105e43b-4cb7-4b23-89ba-cf6c5370f185"
Sep 09 20:40:47.502450 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:47.502406    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1105e43b-4cb7-4b23-89ba-cf6c5370f185-metrics-certs\") pod \"network-metrics-daemon-2fzgg\" (UID: \"1105e43b-4cb7-4b23-89ba-cf6c5370f185\") " pod="openshift-multus/network-metrics-daemon-2fzgg"
Sep 09 20:40:47.502617 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:47.502456    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ad64d253-c155-4e3c-b872-cdebb0616a0e-original-pull-secret\") pod \"global-pull-secret-syncer-bjwzq\" (UID: \"ad64d253-c155-4e3c-b872-cdebb0616a0e\") " pod="kube-system/global-pull-secret-syncer-bjwzq"
Sep 09 20:40:47.502617 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:47.502597    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 20:40:47.502724 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:47.502623    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 20:40:47.502724 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:47.502663    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad64d253-c155-4e3c-b872-cdebb0616a0e-original-pull-secret podName:ad64d253-c155-4e3c-b872-cdebb0616a0e nodeName:}" failed. No retries permitted until 2026-09-09 20:40:55.502645048 +0000 UTC m=+17.130272914 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ad64d253-c155-4e3c-b872-cdebb0616a0e-original-pull-secret") pod "global-pull-secret-syncer-bjwzq" (UID: "ad64d253-c155-4e3c-b872-cdebb0616a0e") : object "kube-system"/"original-pull-secret" not registered
Sep 09 20:40:47.502724 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:47.502682    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1105e43b-4cb7-4b23-89ba-cf6c5370f185-metrics-certs podName:1105e43b-4cb7-4b23-89ba-cf6c5370f185 nodeName:}" failed. No retries permitted until 2026-09-09 20:40:55.502672462 +0000 UTC m=+17.130300316 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1105e43b-4cb7-4b23-89ba-cf6c5370f185-metrics-certs") pod "network-metrics-daemon-2fzgg" (UID: "1105e43b-4cb7-4b23-89ba-cf6c5370f185") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 20:40:47.603368 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:47.603329    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9gvv\" (UniqueName: \"kubernetes.io/projected/2f3c30c5-3a66-4a51-ba58-a125dce436ab-kube-api-access-x9gvv\") pod \"network-check-target-2c5hn\" (UID: \"2f3c30c5-3a66-4a51-ba58-a125dce436ab\") " pod="openshift-network-diagnostics/network-check-target-2c5hn"
Sep 09 20:40:47.603550 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:47.603525    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 20:40:47.603614 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:47.603552    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 20:40:47.603614 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:47.603568    2581 projected.go:196] Error preparing data for projected volume kube-api-access-x9gvv for pod openshift-network-diagnostics/network-check-target-2c5hn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 20:40:47.603722 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:47.603633    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f3c30c5-3a66-4a51-ba58-a125dce436ab-kube-api-access-x9gvv podName:2f3c30c5-3a66-4a51-ba58-a125dce436ab nodeName:}" failed. No retries permitted until 2026-09-09 20:40:55.603613518 +0000 UTC m=+17.231241392 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-x9gvv" (UniqueName: "kubernetes.io/projected/2f3c30c5-3a66-4a51-ba58-a125dce436ab-kube-api-access-x9gvv") pod "network-check-target-2c5hn" (UID: "2f3c30c5-3a66-4a51-ba58-a125dce436ab") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 20:40:47.814707 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:47.814635    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2c5hn"
Sep 09 20:40:47.814707 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:47.814660    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bjwzq"
Sep 09 20:40:47.815203 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:47.814771    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2c5hn" podUID="2f3c30c5-3a66-4a51-ba58-a125dce436ab"
Sep 09 20:40:47.815203 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:47.814894    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bjwzq" podUID="ad64d253-c155-4e3c-b872-cdebb0616a0e"
Sep 09 20:40:48.814749 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:48.814716    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2fzgg"
Sep 09 20:40:48.815230 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:48.814842    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2fzgg" podUID="1105e43b-4cb7-4b23-89ba-cf6c5370f185"
Sep 09 20:40:49.814060 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:49.814028    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bjwzq"
Sep 09 20:40:49.814060 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:49.814052    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2c5hn"
Sep 09 20:40:49.814260 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:49.814157    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bjwzq" podUID="ad64d253-c155-4e3c-b872-cdebb0616a0e"
Sep 09 20:40:49.814350 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:49.814300    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2c5hn" podUID="2f3c30c5-3a66-4a51-ba58-a125dce436ab"
Sep 09 20:40:50.817035 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:50.817001    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2fzgg"
Sep 09 20:40:50.817470 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:50.817133    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2fzgg" podUID="1105e43b-4cb7-4b23-89ba-cf6c5370f185"
Sep 09 20:40:51.814114 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:51.814068    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bjwzq"
Sep 09 20:40:51.814282 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:51.814214    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bjwzq" podUID="ad64d253-c155-4e3c-b872-cdebb0616a0e"
Sep 09 20:40:51.814282 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:51.814258    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2c5hn"
Sep 09 20:40:51.814406 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:51.814384    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2c5hn" podUID="2f3c30c5-3a66-4a51-ba58-a125dce436ab"
Sep 09 20:40:52.813991 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:52.813955    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2fzgg"
Sep 09 20:40:52.814506 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:52.814121    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2fzgg" podUID="1105e43b-4cb7-4b23-89ba-cf6c5370f185"
Sep 09 20:40:53.814783 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:53.814753    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bjwzq"
Sep 09 20:40:53.814783 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:53.814780    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2c5hn"
Sep 09 20:40:53.815246 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:53.814856    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bjwzq" podUID="ad64d253-c155-4e3c-b872-cdebb0616a0e"
Sep 09 20:40:53.815246 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:53.814993    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2c5hn" podUID="2f3c30c5-3a66-4a51-ba58-a125dce436ab"
Sep 09 20:40:54.814381 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:54.814342    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2fzgg"
Sep 09 20:40:54.814567 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:54.814479    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2fzgg" podUID="1105e43b-4cb7-4b23-89ba-cf6c5370f185"
Sep 09 20:40:55.561545 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:55.561512    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1105e43b-4cb7-4b23-89ba-cf6c5370f185-metrics-certs\") pod \"network-metrics-daemon-2fzgg\" (UID: \"1105e43b-4cb7-4b23-89ba-cf6c5370f185\") " pod="openshift-multus/network-metrics-daemon-2fzgg"
Sep 09 20:40:55.561545 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:55.561549    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ad64d253-c155-4e3c-b872-cdebb0616a0e-original-pull-secret\") pod \"global-pull-secret-syncer-bjwzq\" (UID: \"ad64d253-c155-4e3c-b872-cdebb0616a0e\") " pod="kube-system/global-pull-secret-syncer-bjwzq"
Sep 09 20:40:55.562128 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:55.561661    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 20:40:55.562128 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:55.561714    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad64d253-c155-4e3c-b872-cdebb0616a0e-original-pull-secret podName:ad64d253-c155-4e3c-b872-cdebb0616a0e nodeName:}" failed. No retries permitted until 2026-09-09 20:41:11.561700672 +0000 UTC m=+33.189328524 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ad64d253-c155-4e3c-b872-cdebb0616a0e-original-pull-secret") pod "global-pull-secret-syncer-bjwzq" (UID: "ad64d253-c155-4e3c-b872-cdebb0616a0e") : object "kube-system"/"original-pull-secret" not registered
Sep 09 20:40:55.562128 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:55.561661    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 20:40:55.562128 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:55.561833    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1105e43b-4cb7-4b23-89ba-cf6c5370f185-metrics-certs podName:1105e43b-4cb7-4b23-89ba-cf6c5370f185 nodeName:}" failed. No retries permitted until 2026-09-09 20:41:11.561812112 +0000 UTC m=+33.189439965 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1105e43b-4cb7-4b23-89ba-cf6c5370f185-metrics-certs") pod "network-metrics-daemon-2fzgg" (UID: "1105e43b-4cb7-4b23-89ba-cf6c5370f185") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 20:40:55.662615 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:55.662579    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9gvv\" (UniqueName: \"kubernetes.io/projected/2f3c30c5-3a66-4a51-ba58-a125dce436ab-kube-api-access-x9gvv\") pod \"network-check-target-2c5hn\" (UID: \"2f3c30c5-3a66-4a51-ba58-a125dce436ab\") " pod="openshift-network-diagnostics/network-check-target-2c5hn"
Sep 09 20:40:55.662798 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:55.662745    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 20:40:55.662798 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:55.662778    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 20:40:55.662798 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:55.662791    2581 projected.go:196] Error preparing data for projected volume kube-api-access-x9gvv for pod openshift-network-diagnostics/network-check-target-2c5hn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 20:40:55.662956 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:55.662858    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f3c30c5-3a66-4a51-ba58-a125dce436ab-kube-api-access-x9gvv podName:2f3c30c5-3a66-4a51-ba58-a125dce436ab nodeName:}" failed. No retries permitted until 2026-09-09 20:41:11.662838769 +0000 UTC m=+33.290466639 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-x9gvv" (UniqueName: "kubernetes.io/projected/2f3c30c5-3a66-4a51-ba58-a125dce436ab-kube-api-access-x9gvv") pod "network-check-target-2c5hn" (UID: "2f3c30c5-3a66-4a51-ba58-a125dce436ab") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 20:40:55.814624 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:55.814538    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bjwzq"
Sep 09 20:40:55.814624 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:55.814559    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2c5hn"
Sep 09 20:40:55.814862 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:55.814669    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bjwzq" podUID="ad64d253-c155-4e3c-b872-cdebb0616a0e"
Sep 09 20:40:55.814862 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:55.814812    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2c5hn" podUID="2f3c30c5-3a66-4a51-ba58-a125dce436ab"
Sep 09 20:40:56.814500 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:56.814468    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2fzgg"
Sep 09 20:40:56.815009 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:56.814609    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2fzgg" podUID="1105e43b-4cb7-4b23-89ba-cf6c5370f185"
Sep 09 20:40:57.814735 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:57.814710    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2c5hn"
Sep 09 20:40:57.815145 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:57.814823    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2c5hn" podUID="2f3c30c5-3a66-4a51-ba58-a125dce436ab"
Sep 09 20:40:57.815145 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:57.814849    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bjwzq"
Sep 09 20:40:57.815145 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:57.814982    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bjwzq" podUID="ad64d253-c155-4e3c-b872-cdebb0616a0e"
Sep 09 20:40:58.815200 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:58.814962    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2fzgg"
Sep 09 20:40:58.816017 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:58.815299    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2fzgg" podUID="1105e43b-4cb7-4b23-89ba-cf6c5370f185"
Sep 09 20:40:59.001747 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:59.001637    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-99.ec2.internal" event={"ID":"75cd1de7188eb1602ddc4211bd282ec4","Type":"ContainerStarted","Data":"fc1d9e3f75d1d57bd3b028700d514956af60d397674351f1b92a2f4e67667d41"}
Sep 09 20:40:59.003422 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:59.003391    2581 generic.go:356] "Generic (PLEG): container finished" podID="82c490e6-edea-4f5c-9fe5-111fe55ccee9" containerID="c903181d3afea46ff2a329f81123bce87c90242106c8e952e029bdb2813818b3" exitCode=0
Sep 09 20:40:59.003548 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:59.003464    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p6642" event={"ID":"82c490e6-edea-4f5c-9fe5-111fe55ccee9","Type":"ContainerDied","Data":"c903181d3afea46ff2a329f81123bce87c90242106c8e952e029bdb2813818b3"}
Sep 09 20:40:59.005352 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:59.005329    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-cfg6s" event={"ID":"5096efd2-6fbf-45a2-930b-acdc3970eabd","Type":"ContainerStarted","Data":"b32c4f1c5b0ff7c7328309a9c7c26117a6966e25762d9822ba156afde6723c43"}
Sep 09 20:40:59.007146 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:59.007059    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-f8mqt" event={"ID":"0104bf49-c8d9-493a-b189-e570ebf93e62","Type":"ContainerStarted","Data":"4fda43e0648d4d06e6b5b0ad15afb8a32e777a634705624c02d67c89507166db"}
Sep 09 20:40:59.008728 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:59.008680    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-v6pmz" event={"ID":"c45d31bc-82ca-49b3-9bfa-9bdae327bef7","Type":"ContainerStarted","Data":"c43bdf00dafca1083f0bf0f18216704f5708911417aa6ddb58fe8625ed25125e"}
Sep 09 20:40:59.010196 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:59.010176    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-lx79f" event={"ID":"0bd19a8a-4d11-4e06-b2f9-7f04ccd35574","Type":"ContainerStarted","Data":"206e2107cb8be30d46c81953b8e76c74ce9e817c705dd60f8100d534e45785f5"}
Sep 09 20:40:59.011889 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:59.011864    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rj5rk" event={"ID":"964fa8a8-b4c3-45c5-b2bd-6a6d32a6536f","Type":"ContainerStarted","Data":"f5bf2a97c47c19bb470eeb6ee57a2d8e615918e5428dbc0dbaea6dbffb8e30cf"}
Sep 09 20:40:59.013639 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:59.013618    2581 generic.go:356] "Generic (PLEG): container finished" podID="77a4e0e482d02ec9e357614bf125373e" containerID="f853135be7de136fc2d5b7028e29930143c2115ade214ba7f9cd88eb12393c7a" exitCode=0
Sep 09 20:40:59.013723 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:59.013645    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-99.ec2.internal" event={"ID":"77a4e0e482d02ec9e357614bf125373e","Type":"ContainerDied","Data":"f853135be7de136fc2d5b7028e29930143c2115ade214ba7f9cd88eb12393c7a"}
Sep 09 20:40:59.016782 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:59.016758    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-75wj4_55b4fbae-29dc-41bf-a907-b1412b852cef/ovn-acl-logging/0.log"
Sep 09 20:40:59.017173 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:59.017154    2581 generic.go:356] "Generic (PLEG): container finished" podID="55b4fbae-29dc-41bf-a907-b1412b852cef" containerID="46bd6ce1d7471906c7c48154f2e38161dec1610c0d53bdb1f711c29c8527447f" exitCode=1
Sep 09 20:40:59.017276 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:59.017205    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-75wj4" event={"ID":"55b4fbae-29dc-41bf-a907-b1412b852cef","Type":"ContainerStarted","Data":"64d709c9daed1c9032837bbff27bfc10cd63a4e1a564cfa064180917cec130d1"}
Sep 09 20:40:59.017276 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:59.017251    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-75wj4" event={"ID":"55b4fbae-29dc-41bf-a907-b1412b852cef","Type":"ContainerStarted","Data":"596bdddb8232887a3fd8e602d0bb2e17d43a76cc1b678470ea6e607233162780"}
Sep 09 20:40:59.017391 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:59.017295    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-75wj4" event={"ID":"55b4fbae-29dc-41bf-a907-b1412b852cef","Type":"ContainerStarted","Data":"998e970b3084bb178d8023f0f1735afa952f59ab185fc89259d36ac18e7367c7"}
Sep 09 20:40:59.017391 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:59.017331    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-75wj4" event={"ID":"55b4fbae-29dc-41bf-a907-b1412b852cef","Type":"ContainerStarted","Data":"4169e7345a254f43c3f4c2506105a5468cc59e258c1b894dcd182e7498e2f83d"}
Sep 09 20:40:59.017391 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:59.017376    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-75wj4" event={"ID":"55b4fbae-29dc-41bf-a907-b1412b852cef","Type":"ContainerDied","Data":"46bd6ce1d7471906c7c48154f2e38161dec1610c0d53bdb1f711c29c8527447f"}
Sep 09 20:40:59.017527 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:59.017407    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-75wj4" event={"ID":"55b4fbae-29dc-41bf-a907-b1412b852cef","Type":"ContainerStarted","Data":"13245830cf517931e3817dd23d69d919a80724ca596bb5b4237c2dd52a92c873"}
Sep 09 20:40:59.050417 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:59.050358    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-99.ec2.internal" podStartSLOduration=20.050338848 podStartE2EDuration="20.050338848s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:40:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:40:59.022983349 +0000 UTC m=+20.650611221" watchObservedRunningTime="2026-09-09 20:40:59.050338848 +0000 UTC m=+20.677966722"
Sep 09 20:40:59.089494 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:59.089440    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-v6pmz" podStartSLOduration=3.415192628 podStartE2EDuration="21.089426413s" totalImagesPullingTime="17.674233785s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:40:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:40:59.071375353 +0000 UTC m=+20.699003227" watchObservedRunningTime="2026-09-09 20:40:59.089426413 +0000 UTC m=+20.717054286"
Sep 09 20:40:59.089674 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:59.089578    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-f8mqt" podStartSLOduration=3.458892371 podStartE2EDuration="21.089572515s" totalImagesPullingTime="17.630680144s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:40:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:40:59.089567184 +0000 UTC m=+20.717195043" watchObservedRunningTime="2026-09-09 20:40:59.089572515 +0000 UTC m=+20.717200389"
Sep 09 20:40:59.127548 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:59.127490    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-rj5rk" podStartSLOduration=3.002977859 podStartE2EDuration="21.127475695s" totalImagesPullingTime="18.124497836s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:40:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:40:59.12742344 +0000 UTC m=+20.755051318" watchObservedRunningTime="2026-09-09 20:40:59.127475695 +0000 UTC m=+20.755103568"
Sep 09 20:40:59.144160 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:59.144114    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-lx79f" podStartSLOduration=7.2766195830000004 podStartE2EDuration="21.144080137s" totalImagesPullingTime="13.867460554s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:40:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:40:59.143783667 +0000 UTC m=+20.771411540" watchObservedRunningTime="2026-09-09 20:40:59.144080137 +0000 UTC m=+20.771708011"
Sep 09 20:40:59.157716 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:59.157672    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-cfg6s" podStartSLOduration=2.542503288 podStartE2EDuration="20.157659295s" totalImagesPullingTime="17.615156007s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:40:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:40:59.15750752 +0000 UTC m=+20.785135396" watchObservedRunningTime="2026-09-09 20:40:59.157659295 +0000 UTC m=+20.785287168"
Sep 09 20:40:59.814806 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:59.814724    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bjwzq"
Sep 09 20:40:59.814952 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:40:59.814728    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2c5hn"
Sep 09 20:40:59.814952 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:59.814838    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bjwzq" podUID="ad64d253-c155-4e3c-b872-cdebb0616a0e"
Sep 09 20:40:59.814952 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:40:59.814925    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2c5hn" podUID="2f3c30c5-3a66-4a51-ba58-a125dce436ab"
Sep 09 20:41:00.022044 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:00.021998    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jq8ls" event={"ID":"10602efd-a422-4354-9f8e-bc58dd9ec7dd","Type":"ContainerStarted","Data":"87d89632a604fa740700627be896dfe3d8b2ffd7943ba30df0475b93d3878a60"}
Sep 09 20:41:00.024262 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:00.024236    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-99.ec2.internal" event={"ID":"77a4e0e482d02ec9e357614bf125373e","Type":"ContainerStarted","Data":"f63d71a7bf3f3755d839fea5dfce737af205a757428e241c6cdf8c1b8f9556f1"}
Sep 09 20:41:00.026448 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:00.026416    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-ft4fb" event={"ID":"d1d25b48-1c88-4b60-93a1-665b90df19da","Type":"ContainerStarted","Data":"067b8889059ae2c391aded1cccb7826343313b8057203b7b9c8213c77fca7470"}
Sep 09 20:41:00.040138 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:00.040069    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-99.ec2.internal" podStartSLOduration=21.040048792 podStartE2EDuration="21.040048792s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:40:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:41:00.039694886 +0000 UTC m=+21.667322767" watchObservedRunningTime="2026-09-09 20:41:00.040048792 +0000 UTC m=+21.667676669"
Sep 09 20:41:00.601186 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:00.600951    2581 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 09 20:41:00.787922 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:00.787718    2581 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-09T20:41:00.60116886Z","UUID":"aacbb9cf-bcd4-4904-ad91-861e1e92e162","Handler":null,"Name":"","Endpoint":""}
Sep 09 20:41:00.789426 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:00.789405    2581 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 09 20:41:00.789542 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:00.789439    2581 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 09 20:41:00.813782 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:00.813749    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2fzgg"
Sep 09 20:41:00.813933 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:00.813848    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2fzgg" podUID="1105e43b-4cb7-4b23-89ba-cf6c5370f185"
Sep 09 20:41:01.031504 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:01.031462    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jq8ls" event={"ID":"10602efd-a422-4354-9f8e-bc58dd9ec7dd","Type":"ContainerStarted","Data":"d70bce3dcf0b89ba0a70e9d193a0265f410bff20f087a6ea4c8a6bd7c6f52a31"}
Sep 09 20:41:01.035217 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:01.035193    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-75wj4_55b4fbae-29dc-41bf-a907-b1412b852cef/ovn-acl-logging/0.log"
Sep 09 20:41:01.035624 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:01.035582    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-75wj4" event={"ID":"55b4fbae-29dc-41bf-a907-b1412b852cef","Type":"ContainerStarted","Data":"764b81c35750ed87ac5f5179e52506b11a3cb9b3136e5378ae3285473b40ab71"}
Sep 09 20:41:01.814121 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:01.814030    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bjwzq"
Sep 09 20:41:01.814121 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:01.814076    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2c5hn"
Sep 09 20:41:01.814340 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:01.814185    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bjwzq" podUID="ad64d253-c155-4e3c-b872-cdebb0616a0e"
Sep 09 20:41:01.814400 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:01.814330    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2c5hn" podUID="2f3c30c5-3a66-4a51-ba58-a125dce436ab"
Sep 09 20:41:02.040958 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:02.040918    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jq8ls" event={"ID":"10602efd-a422-4354-9f8e-bc58dd9ec7dd","Type":"ContainerStarted","Data":"df448a407663e19be96decb0dc0fb7080af7e094d625424724a1bc5c78a389e8"}
Sep 09 20:41:02.060867 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:02.060813    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jq8ls" podStartSLOduration=1.7353801309999999 podStartE2EDuration="23.060796047s" totalImagesPullingTime="21.325415916s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:40:39 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=3 observedRunningTime="2026-09-09 20:41:02.060734752 +0000 UTC m=+23.688362625" watchObservedRunningTime="2026-09-09 20:41:02.060796047 +0000 UTC m=+23.688423920"
Sep 09 20:41:02.061151 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:02.060968    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-ft4fb" podStartSLOduration=6.364457177 podStartE2EDuration="24.060962715s" totalImagesPullingTime="17.696505538s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:40:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:41:00.054182726 +0000 UTC m=+21.681810601" watchObservedRunningTime="2026-09-09 20:41:02.060962715 +0000 UTC m=+23.688590630"
Sep 09 20:41:02.814309 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:02.814277    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2fzgg"
Sep 09 20:41:02.814496 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:02.814398    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2fzgg" podUID="1105e43b-4cb7-4b23-89ba-cf6c5370f185"
Sep 09 20:41:03.688869 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:03.688847    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-lx79f"
Sep 09 20:41:03.689455 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:03.689436    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-lx79f"
Sep 09 20:41:03.814260 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:03.814234    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2c5hn"
Sep 09 20:41:03.814363 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:03.814248    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bjwzq"
Sep 09 20:41:03.814363 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:03.814348    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2c5hn" podUID="2f3c30c5-3a66-4a51-ba58-a125dce436ab"
Sep 09 20:41:03.814446 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:03.814388    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bjwzq" podUID="ad64d253-c155-4e3c-b872-cdebb0616a0e"
Sep 09 20:41:04.050783 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:04.050624    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-75wj4_55b4fbae-29dc-41bf-a907-b1412b852cef/ovn-acl-logging/0.log"
Sep 09 20:41:04.051157 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:04.051128    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-75wj4" event={"ID":"55b4fbae-29dc-41bf-a907-b1412b852cef","Type":"ContainerStarted","Data":"7b8e4673490934354017630596ed28973b43caa55e00704adbe970d42e9e68d9"}
Sep 09 20:41:04.051485 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:04.051459    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-75wj4"
Sep 09 20:41:04.051589 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:04.051550    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-75wj4"
Sep 09 20:41:04.051699 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:04.051679    2581 scope.go:117] "RemoveContainer" containerID="46bd6ce1d7471906c7c48154f2e38161dec1610c0d53bdb1f711c29c8527447f"
Sep 09 20:41:04.053389 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:04.053366    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p6642" event={"ID":"82c490e6-edea-4f5c-9fe5-111fe55ccee9","Type":"ContainerStarted","Data":"3ca250cad18a72fd2076f195dced8a10602eaf7540c8bcdf4783158ef826fe8e"}
Sep 09 20:41:04.067230 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:04.067209    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-75wj4"
Sep 09 20:41:04.814574 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:04.814545    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2fzgg"
Sep 09 20:41:04.814929 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:04.814655    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2fzgg" podUID="1105e43b-4cb7-4b23-89ba-cf6c5370f185"
Sep 09 20:41:05.059905 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:05.059820    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-75wj4_55b4fbae-29dc-41bf-a907-b1412b852cef/ovn-acl-logging/0.log"
Sep 09 20:41:05.060398 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:05.060367    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-75wj4" event={"ID":"55b4fbae-29dc-41bf-a907-b1412b852cef","Type":"ContainerStarted","Data":"c525e1c7b084949cf9399d8250bba39021c7269f9c2e941c6e969d56b7ae6541"}
Sep 09 20:41:05.060714 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:05.060694    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-75wj4"
Sep 09 20:41:05.062392 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:05.062353    2581 generic.go:356] "Generic (PLEG): container finished" podID="82c490e6-edea-4f5c-9fe5-111fe55ccee9" containerID="3ca250cad18a72fd2076f195dced8a10602eaf7540c8bcdf4783158ef826fe8e" exitCode=0
Sep 09 20:41:05.062496 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:05.062402    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p6642" event={"ID":"82c490e6-edea-4f5c-9fe5-111fe55ccee9","Type":"ContainerDied","Data":"3ca250cad18a72fd2076f195dced8a10602eaf7540c8bcdf4783158ef826fe8e"}
Sep 09 20:41:05.075917 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:05.075897    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-75wj4"
Sep 09 20:41:05.089724 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:05.089682    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-75wj4" podStartSLOduration=9.349084098 podStartE2EDuration="27.089667229s" totalImagesPullingTime="17.740583131s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:40:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:41:05.088334796 +0000 UTC m=+26.715962669" watchObservedRunningTime="2026-09-09 20:41:05.089667229 +0000 UTC m=+26.717295102"
Sep 09 20:41:05.610757 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:05.609543    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-bjwzq"]
Sep 09 20:41:05.610757 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:05.609768    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bjwzq"
Sep 09 20:41:05.610757 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:05.609892    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bjwzq" podUID="ad64d253-c155-4e3c-b872-cdebb0616a0e"
Sep 09 20:41:05.610757 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:05.610521    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2c5hn"]
Sep 09 20:41:05.610757 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:05.610609    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2c5hn"
Sep 09 20:41:05.610757 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:05.610679    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2c5hn" podUID="2f3c30c5-3a66-4a51-ba58-a125dce436ab"
Sep 09 20:41:05.611986 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:05.611963    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2fzgg"]
Sep 09 20:41:05.612090 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:05.612062    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2fzgg"
Sep 09 20:41:05.612205 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:05.612179    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2fzgg" podUID="1105e43b-4cb7-4b23-89ba-cf6c5370f185"
Sep 09 20:41:06.066438 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:06.066346    2581 generic.go:356] "Generic (PLEG): container finished" podID="82c490e6-edea-4f5c-9fe5-111fe55ccee9" containerID="ac8967d0399f6819e1003020bfaf9649ae86cfd486a26cfb9297f7cb72f46770" exitCode=0
Sep 09 20:41:06.067129 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:06.066432    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p6642" event={"ID":"82c490e6-edea-4f5c-9fe5-111fe55ccee9","Type":"ContainerDied","Data":"ac8967d0399f6819e1003020bfaf9649ae86cfd486a26cfb9297f7cb72f46770"}
Sep 09 20:41:06.814441 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:06.814411    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bjwzq"
Sep 09 20:41:06.814592 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:06.814516    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bjwzq" podUID="ad64d253-c155-4e3c-b872-cdebb0616a0e"
Sep 09 20:41:07.071150 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:07.071057    2581 generic.go:356] "Generic (PLEG): container finished" podID="82c490e6-edea-4f5c-9fe5-111fe55ccee9" containerID="970ad61d47c868a987ac99ac3c14582932177559b6529e7f556f6a4a3621ef0e" exitCode=0
Sep 09 20:41:07.071563 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:07.071194    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p6642" event={"ID":"82c490e6-edea-4f5c-9fe5-111fe55ccee9","Type":"ContainerDied","Data":"970ad61d47c868a987ac99ac3c14582932177559b6529e7f556f6a4a3621ef0e"}
Sep 09 20:41:07.243299 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:07.243110    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-lx79f"
Sep 09 20:41:07.243432 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:07.243412    2581 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 09 20:41:07.243781 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:07.243755    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-lx79f"
Sep 09 20:41:07.814751 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:07.814713    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2c5hn"
Sep 09 20:41:07.814933 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:07.814840    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2c5hn" podUID="2f3c30c5-3a66-4a51-ba58-a125dce436ab"
Sep 09 20:41:07.814933 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:07.814891    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2fzgg"
Sep 09 20:41:07.815055 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:07.815003    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2fzgg" podUID="1105e43b-4cb7-4b23-89ba-cf6c5370f185"
Sep 09 20:41:08.814411 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:08.814381    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bjwzq"
Sep 09 20:41:08.814795 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:08.814466    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bjwzq" podUID="ad64d253-c155-4e3c-b872-cdebb0616a0e"
Sep 09 20:41:09.813963 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:09.813872    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2c5hn"
Sep 09 20:41:09.814165 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:09.814016    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2c5hn" podUID="2f3c30c5-3a66-4a51-ba58-a125dce436ab"
Sep 09 20:41:09.814165 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:09.814050    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2fzgg"
Sep 09 20:41:09.814273 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:09.814204    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2fzgg" podUID="1105e43b-4cb7-4b23-89ba-cf6c5370f185"
Sep 09 20:41:10.814815 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:10.814781    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bjwzq"
Sep 09 20:41:10.815273 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:10.814902    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bjwzq" podUID="ad64d253-c155-4e3c-b872-cdebb0616a0e"
Sep 09 20:41:11.199353 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:11.199323    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-99.ec2.internal" event="NodeReady"
Sep 09 20:41:11.199515 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:11.199464    2581 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 09 20:41:11.242244 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:11.242211    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-g2tsp"]
Sep 09 20:41:11.276985 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:11.276949    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-4zgnb"]
Sep 09 20:41:11.277163 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:11.277119    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-g2tsp"
Sep 09 20:41:11.279653 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:11.279627    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 09 20:41:11.279785 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:11.279666    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-vs5mb\""
Sep 09 20:41:11.279930 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:11.279916    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 09 20:41:11.301075 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:11.301051    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4zgnb"]
Sep 09 20:41:11.301199 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:11.301087    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-g2tsp"]
Sep 09 20:41:11.301199 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:11.301119    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4zgnb"
Sep 09 20:41:11.303431 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:11.303411    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 09 20:41:11.303543 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:11.303463    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-87t46\""
Sep 09 20:41:11.303543 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:11.303489    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 09 20:41:11.303661 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:11.303579    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 09 20:41:11.387748 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:11.387715    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/48a52e3f-68fb-496f-b974-320a8539e7f7-config-volume\") pod \"dns-default-g2tsp\" (UID: \"48a52e3f-68fb-496f-b974-320a8539e7f7\") " pod="openshift-dns/dns-default-g2tsp"
Sep 09 20:41:11.387942 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:11.387757    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r25cn\" (UniqueName: \"kubernetes.io/projected/933d2649-0bcc-4c52-a0ac-65f33ba49741-kube-api-access-r25cn\") pod \"ingress-canary-4zgnb\" (UID: \"933d2649-0bcc-4c52-a0ac-65f33ba49741\") " pod="openshift-ingress-canary/ingress-canary-4zgnb"
Sep 09 20:41:11.387942 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:11.387792    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/48a52e3f-68fb-496f-b974-320a8539e7f7-metrics-tls\") pod \"dns-default-g2tsp\" (UID: \"48a52e3f-68fb-496f-b974-320a8539e7f7\") " pod="openshift-dns/dns-default-g2tsp"
Sep 09 20:41:11.387942 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:11.387818    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgbxw\" (UniqueName: \"kubernetes.io/projected/48a52e3f-68fb-496f-b974-320a8539e7f7-kube-api-access-qgbxw\") pod \"dns-default-g2tsp\" (UID: \"48a52e3f-68fb-496f-b974-320a8539e7f7\") " pod="openshift-dns/dns-default-g2tsp"
Sep 09 20:41:11.387942 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:11.387865    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/933d2649-0bcc-4c52-a0ac-65f33ba49741-cert\") pod \"ingress-canary-4zgnb\" (UID: \"933d2649-0bcc-4c52-a0ac-65f33ba49741\") " pod="openshift-ingress-canary/ingress-canary-4zgnb"
Sep 09 20:41:11.387942 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:11.387897    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48a52e3f-68fb-496f-b974-320a8539e7f7-tmp-dir\") pod \"dns-default-g2tsp\" (UID: \"48a52e3f-68fb-496f-b974-320a8539e7f7\") " pod="openshift-dns/dns-default-g2tsp"
Sep 09 20:41:11.488528 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:11.488448    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/48a52e3f-68fb-496f-b974-320a8539e7f7-config-volume\") pod \"dns-default-g2tsp\" (UID: \"48a52e3f-68fb-496f-b974-320a8539e7f7\") " pod="openshift-dns/dns-default-g2tsp"
Sep 09 20:41:11.488528 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:11.488488    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r25cn\" (UniqueName: \"kubernetes.io/projected/933d2649-0bcc-4c52-a0ac-65f33ba49741-kube-api-access-r25cn\") pod \"ingress-canary-4zgnb\" (UID: \"933d2649-0bcc-4c52-a0ac-65f33ba49741\") " pod="openshift-ingress-canary/ingress-canary-4zgnb"
Sep 09 20:41:11.488528 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:11.488517    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/48a52e3f-68fb-496f-b974-320a8539e7f7-metrics-tls\") pod \"dns-default-g2tsp\" (UID: \"48a52e3f-68fb-496f-b974-320a8539e7f7\") " pod="openshift-dns/dns-default-g2tsp"
Sep 09 20:41:11.488806 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:11.488540    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgbxw\" (UniqueName: \"kubernetes.io/projected/48a52e3f-68fb-496f-b974-320a8539e7f7-kube-api-access-qgbxw\") pod \"dns-default-g2tsp\" (UID: \"48a52e3f-68fb-496f-b974-320a8539e7f7\") " pod="openshift-dns/dns-default-g2tsp"
Sep 09 20:41:11.488806 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:11.488644    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 20:41:11.488806 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:11.488695    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/933d2649-0bcc-4c52-a0ac-65f33ba49741-cert\") pod \"ingress-canary-4zgnb\" (UID: \"933d2649-0bcc-4c52-a0ac-65f33ba49741\") " pod="openshift-ingress-canary/ingress-canary-4zgnb"
Sep 09 20:41:11.488806 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:11.488716    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48a52e3f-68fb-496f-b974-320a8539e7f7-metrics-tls podName:48a52e3f-68fb-496f-b974-320a8539e7f7 nodeName:}" failed. No retries permitted until 2026-09-09 20:41:11.988693999 +0000 UTC m=+33.616321865 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/48a52e3f-68fb-496f-b974-320a8539e7f7-metrics-tls") pod "dns-default-g2tsp" (UID: "48a52e3f-68fb-496f-b974-320a8539e7f7") : secret "dns-default-metrics-tls" not found
Sep 09 20:41:11.488806 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:11.488756    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48a52e3f-68fb-496f-b974-320a8539e7f7-tmp-dir\") pod \"dns-default-g2tsp\" (UID: \"48a52e3f-68fb-496f-b974-320a8539e7f7\") " pod="openshift-dns/dns-default-g2tsp"
Sep 09 20:41:11.489151 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:11.489133    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/48a52e3f-68fb-496f-b974-320a8539e7f7-config-volume\") pod \"dns-default-g2tsp\" (UID: \"48a52e3f-68fb-496f-b974-320a8539e7f7\") " pod="openshift-dns/dns-default-g2tsp"
Sep 09 20:41:11.489202 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:11.489183    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48a52e3f-68fb-496f-b974-320a8539e7f7-tmp-dir\") pod \"dns-default-g2tsp\" (UID: \"48a52e3f-68fb-496f-b974-320a8539e7f7\") " pod="openshift-dns/dns-default-g2tsp"
Sep 09 20:41:11.493566 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:11.493520    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/933d2649-0bcc-4c52-a0ac-65f33ba49741-cert\") pod \"ingress-canary-4zgnb\" (UID: \"933d2649-0bcc-4c52-a0ac-65f33ba49741\") " pod="openshift-ingress-canary/ingress-canary-4zgnb"
Sep 09 20:41:11.496989 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:11.496962    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r25cn\" (UniqueName: \"kubernetes.io/projected/933d2649-0bcc-4c52-a0ac-65f33ba49741-kube-api-access-r25cn\") pod \"ingress-canary-4zgnb\" (UID: \"933d2649-0bcc-4c52-a0ac-65f33ba49741\") " pod="openshift-ingress-canary/ingress-canary-4zgnb"
Sep 09 20:41:11.497722 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:11.497679    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgbxw\" (UniqueName: \"kubernetes.io/projected/48a52e3f-68fb-496f-b974-320a8539e7f7-kube-api-access-qgbxw\") pod \"dns-default-g2tsp\" (UID: \"48a52e3f-68fb-496f-b974-320a8539e7f7\") " pod="openshift-dns/dns-default-g2tsp"
Sep 09 20:41:11.589917 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:11.589881    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1105e43b-4cb7-4b23-89ba-cf6c5370f185-metrics-certs\") pod \"network-metrics-daemon-2fzgg\" (UID: \"1105e43b-4cb7-4b23-89ba-cf6c5370f185\") " pod="openshift-multus/network-metrics-daemon-2fzgg"
Sep 09 20:41:11.590157 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:11.589928    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ad64d253-c155-4e3c-b872-cdebb0616a0e-original-pull-secret\") pod \"global-pull-secret-syncer-bjwzq\" (UID: \"ad64d253-c155-4e3c-b872-cdebb0616a0e\") " pod="kube-system/global-pull-secret-syncer-bjwzq"
Sep 09 20:41:11.590157 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:11.590049    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 20:41:11.590157 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:11.590131    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad64d253-c155-4e3c-b872-cdebb0616a0e-original-pull-secret podName:ad64d253-c155-4e3c-b872-cdebb0616a0e nodeName:}" failed. No retries permitted until 2026-09-09 20:41:43.590096056 +0000 UTC m=+65.217723921 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ad64d253-c155-4e3c-b872-cdebb0616a0e-original-pull-secret") pod "global-pull-secret-syncer-bjwzq" (UID: "ad64d253-c155-4e3c-b872-cdebb0616a0e") : object "kube-system"/"original-pull-secret" not registered
Sep 09 20:41:11.590157 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:11.590050    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 20:41:11.590377 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:11.590209    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1105e43b-4cb7-4b23-89ba-cf6c5370f185-metrics-certs podName:1105e43b-4cb7-4b23-89ba-cf6c5370f185 nodeName:}" failed. No retries permitted until 2026-09-09 20:41:43.590189762 +0000 UTC m=+65.217817637 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1105e43b-4cb7-4b23-89ba-cf6c5370f185-metrics-certs") pod "network-metrics-daemon-2fzgg" (UID: "1105e43b-4cb7-4b23-89ba-cf6c5370f185") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 20:41:11.610725 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:11.610695    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4zgnb"
Sep 09 20:41:11.691213 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:11.691147    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9gvv\" (UniqueName: \"kubernetes.io/projected/2f3c30c5-3a66-4a51-ba58-a125dce436ab-kube-api-access-x9gvv\") pod \"network-check-target-2c5hn\" (UID: \"2f3c30c5-3a66-4a51-ba58-a125dce436ab\") " pod="openshift-network-diagnostics/network-check-target-2c5hn"
Sep 09 20:41:11.691388 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:11.691326    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 20:41:11.691388 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:11.691349    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 20:41:11.691388 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:11.691359    2581 projected.go:196] Error preparing data for projected volume kube-api-access-x9gvv for pod openshift-network-diagnostics/network-check-target-2c5hn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 20:41:11.691497 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:11.691409    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f3c30c5-3a66-4a51-ba58-a125dce436ab-kube-api-access-x9gvv podName:2f3c30c5-3a66-4a51-ba58-a125dce436ab nodeName:}" failed. No retries permitted until 2026-09-09 20:41:43.691395982 +0000 UTC m=+65.319023838 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-x9gvv" (UniqueName: "kubernetes.io/projected/2f3c30c5-3a66-4a51-ba58-a125dce436ab-kube-api-access-x9gvv") pod "network-check-target-2c5hn" (UID: "2f3c30c5-3a66-4a51-ba58-a125dce436ab") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 20:41:11.814476 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:11.814447    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2fzgg"
Sep 09 20:41:11.814619 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:11.814457    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2c5hn"
Sep 09 20:41:11.817118 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:11.817083    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 09 20:41:11.817462 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:11.817084    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 09 20:41:11.817462 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:11.817230    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 09 20:41:11.817462 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:11.817321    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-sl49q\""
Sep 09 20:41:11.817462 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:11.817426    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-q6qnd\""
Sep 09 20:41:11.993503 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:11.993418    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/48a52e3f-68fb-496f-b974-320a8539e7f7-metrics-tls\") pod \"dns-default-g2tsp\" (UID: \"48a52e3f-68fb-496f-b974-320a8539e7f7\") " pod="openshift-dns/dns-default-g2tsp"
Sep 09 20:41:11.993657 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:11.993560    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 20:41:11.993657 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:11.993622    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48a52e3f-68fb-496f-b974-320a8539e7f7-metrics-tls podName:48a52e3f-68fb-496f-b974-320a8539e7f7 nodeName:}" failed. No retries permitted until 2026-09-09 20:41:12.993606247 +0000 UTC m=+34.621234103 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/48a52e3f-68fb-496f-b974-320a8539e7f7-metrics-tls") pod "dns-default-g2tsp" (UID: "48a52e3f-68fb-496f-b974-320a8539e7f7") : secret "dns-default-metrics-tls" not found
Sep 09 20:41:12.016702 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:12.016672    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4zgnb"]
Sep 09 20:41:12.034911 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:41:12.034887    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod933d2649_0bcc_4c52_a0ac_65f33ba49741.slice/crio-903b35e6b381055bf56a4e87de401067c393efc41ce9b9dea32481ff776f16f9 WatchSource:0}: Error finding container 903b35e6b381055bf56a4e87de401067c393efc41ce9b9dea32481ff776f16f9: Status 404 returned error can't find the container with id 903b35e6b381055bf56a4e87de401067c393efc41ce9b9dea32481ff776f16f9
Sep 09 20:41:12.084312 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:12.084289    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4zgnb" event={"ID":"933d2649-0bcc-4c52-a0ac-65f33ba49741","Type":"ContainerStarted","Data":"903b35e6b381055bf56a4e87de401067c393efc41ce9b9dea32481ff776f16f9"}
Sep 09 20:41:12.813998 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:12.813828    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bjwzq"
Sep 09 20:41:12.816599 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:12.816575    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 09 20:41:13.001190 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:13.001153    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/48a52e3f-68fb-496f-b974-320a8539e7f7-metrics-tls\") pod \"dns-default-g2tsp\" (UID: \"48a52e3f-68fb-496f-b974-320a8539e7f7\") " pod="openshift-dns/dns-default-g2tsp"
Sep 09 20:41:13.001700 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:13.001333    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 20:41:13.001700 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:13.001434    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48a52e3f-68fb-496f-b974-320a8539e7f7-metrics-tls podName:48a52e3f-68fb-496f-b974-320a8539e7f7 nodeName:}" failed. No retries permitted until 2026-09-09 20:41:15.001410051 +0000 UTC m=+36.629037918 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/48a52e3f-68fb-496f-b974-320a8539e7f7-metrics-tls") pod "dns-default-g2tsp" (UID: "48a52e3f-68fb-496f-b974-320a8539e7f7") : secret "dns-default-metrics-tls" not found
Sep 09 20:41:13.090784 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:13.090709    2581 generic.go:356] "Generic (PLEG): container finished" podID="82c490e6-edea-4f5c-9fe5-111fe55ccee9" containerID="16c66ab75d48a90622bc1ad93c2c0f2673dcfb86d56fad5efc95ad39298f2bdb" exitCode=0
Sep 09 20:41:13.090988 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:13.090818    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p6642" event={"ID":"82c490e6-edea-4f5c-9fe5-111fe55ccee9","Type":"ContainerDied","Data":"16c66ab75d48a90622bc1ad93c2c0f2673dcfb86d56fad5efc95ad39298f2bdb"}
Sep 09 20:41:14.097003 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:14.096964    2581 generic.go:356] "Generic (PLEG): container finished" podID="82c490e6-edea-4f5c-9fe5-111fe55ccee9" containerID="9e3171ec5100dbf10f02227e9a126438301dde3899b6af5ed899c5eeef7aa91d" exitCode=0
Sep 09 20:41:14.097762 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:14.097044    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p6642" event={"ID":"82c490e6-edea-4f5c-9fe5-111fe55ccee9","Type":"ContainerDied","Data":"9e3171ec5100dbf10f02227e9a126438301dde3899b6af5ed899c5eeef7aa91d"}
Sep 09 20:41:15.015529 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:15.015448    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/48a52e3f-68fb-496f-b974-320a8539e7f7-metrics-tls\") pod \"dns-default-g2tsp\" (UID: \"48a52e3f-68fb-496f-b974-320a8539e7f7\") " pod="openshift-dns/dns-default-g2tsp"
Sep 09 20:41:15.015671 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:15.015556    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 20:41:15.015671 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:15.015611    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48a52e3f-68fb-496f-b974-320a8539e7f7-metrics-tls podName:48a52e3f-68fb-496f-b974-320a8539e7f7 nodeName:}" failed. No retries permitted until 2026-09-09 20:41:19.015596352 +0000 UTC m=+40.643224202 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/48a52e3f-68fb-496f-b974-320a8539e7f7-metrics-tls") pod "dns-default-g2tsp" (UID: "48a52e3f-68fb-496f-b974-320a8539e7f7") : secret "dns-default-metrics-tls" not found
Sep 09 20:41:15.101247 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:15.101212    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4zgnb" event={"ID":"933d2649-0bcc-4c52-a0ac-65f33ba49741","Type":"ContainerStarted","Data":"b41ff99bc31c35d58d2e1376848911a0d3d2dfe46cdb9e09908a787b03dcb486"}
Sep 09 20:41:15.105200 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:15.105172    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p6642" event={"ID":"82c490e6-edea-4f5c-9fe5-111fe55ccee9","Type":"ContainerStarted","Data":"f2ab51448dbc4fc3a22d7976bde769e64499fb74137d7e01ee3d7fa7561b2e2e"}
Sep 09 20:41:15.116894 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:15.116845    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-4zgnb" podStartSLOduration=1.614657038 podStartE2EDuration="4.116828669s" totalImagesPullingTime="2.502171631s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:41:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:41:15.115947663 +0000 UTC m=+36.743575537" watchObservedRunningTime="2026-09-09 20:41:15.116828669 +0000 UTC m=+36.744456533"
Sep 09 20:41:15.139662 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:15.139600    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-p6642" podStartSLOduration=7.477877554 podStartE2EDuration="36.139580673s" totalImagesPullingTime="28.661703119s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:40:39 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:41:15.137484669 +0000 UTC m=+36.765112542" watchObservedRunningTime="2026-09-09 20:41:15.139580673 +0000 UTC m=+36.767208546"
Sep 09 20:41:19.044119 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:19.044041    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/48a52e3f-68fb-496f-b974-320a8539e7f7-metrics-tls\") pod \"dns-default-g2tsp\" (UID: \"48a52e3f-68fb-496f-b974-320a8539e7f7\") " pod="openshift-dns/dns-default-g2tsp"
Sep 09 20:41:19.044496 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:19.044206    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 20:41:19.044496 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:19.044269    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48a52e3f-68fb-496f-b974-320a8539e7f7-metrics-tls podName:48a52e3f-68fb-496f-b974-320a8539e7f7 nodeName:}" failed. No retries permitted until 2026-09-09 20:41:27.044254427 +0000 UTC m=+48.671882283 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/48a52e3f-68fb-496f-b974-320a8539e7f7-metrics-tls") pod "dns-default-g2tsp" (UID: "48a52e3f-68fb-496f-b974-320a8539e7f7") : secret "dns-default-metrics-tls" not found
Sep 09 20:41:20.983094 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:20.983061    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-r5kxh"]
Sep 09 20:41:21.032938 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.032900    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-gwc6n"]
Sep 09 20:41:21.033094 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.033038    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-r5kxh"
Sep 09 20:41:21.035279 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.035251    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 09 20:41:21.035409 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.035358    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 09 20:41:21.035567 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.035549    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 09 20:41:21.035803 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.035788    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 09 20:41:21.036064 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.036051    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-gw7k6\""
Sep 09 20:41:21.041859 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.041839    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 09 20:41:21.052443 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.052419    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-85b454874b-wsv4v"]
Sep 09 20:41:21.052572 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.052556    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-gwc6n"
Sep 09 20:41:21.054784 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.054767    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 09 20:41:21.054879 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.054793    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 09 20:41:21.054879 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.054806    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 09 20:41:21.054879 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.054806    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-rhljr\""
Sep 09 20:41:21.078311 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.078288    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-gwc6n"]
Sep 09 20:41:21.078444 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.078323    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-wsv4v"]
Sep 09 20:41:21.078444 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.078337    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-r5kxh"]
Sep 09 20:41:21.078444 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.078417    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-wsv4v"
Sep 09 20:41:21.082244 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.082221    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 09 20:41:21.082368 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.082221    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 09 20:41:21.083009 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.082987    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 09 20:41:21.083088 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.083054    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 09 20:41:21.083477 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.083460    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-qk6vc\""
Sep 09 20:41:21.089967 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.089923    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 09 20:41:21.158634 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.158601    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/981c03df-6cd9-4e28-b060-2fd16739d669-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-gwc6n\" (UID: \"981c03df-6cd9-4e28-b060-2fd16739d669\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-gwc6n"
Sep 09 20:41:21.158634 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.158637    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4kqn\" (UniqueName: \"kubernetes.io/projected/981c03df-6cd9-4e28-b060-2fd16739d669-kube-api-access-l4kqn\") pod \"cluster-samples-operator-7cdcfcf76c-gwc6n\" (UID: \"981c03df-6cd9-4e28-b060-2fd16739d669\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-gwc6n"
Sep 09 20:41:21.158824 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.158656    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/4eb46131-2f57-408b-9d0f-a9ee9b55c8ef-snapshots\") pod \"insights-operator-76bcd6ccf5-r5kxh\" (UID: \"4eb46131-2f57-408b-9d0f-a9ee9b55c8ef\") " pod="openshift-insights/insights-operator-76bcd6ccf5-r5kxh"
Sep 09 20:41:21.158824 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.158698    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4eb46131-2f57-408b-9d0f-a9ee9b55c8ef-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-r5kxh\" (UID: \"4eb46131-2f57-408b-9d0f-a9ee9b55c8ef\") " pod="openshift-insights/insights-operator-76bcd6ccf5-r5kxh"
Sep 09 20:41:21.158824 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.158781    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4eb46131-2f57-408b-9d0f-a9ee9b55c8ef-tmp\") pod \"insights-operator-76bcd6ccf5-r5kxh\" (UID: \"4eb46131-2f57-408b-9d0f-a9ee9b55c8ef\") " pod="openshift-insights/insights-operator-76bcd6ccf5-r5kxh"
Sep 09 20:41:21.158824 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.158797    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5q2nh\" (UniqueName: \"kubernetes.io/projected/4eb46131-2f57-408b-9d0f-a9ee9b55c8ef-kube-api-access-5q2nh\") pod \"insights-operator-76bcd6ccf5-r5kxh\" (UID: \"4eb46131-2f57-408b-9d0f-a9ee9b55c8ef\") " pod="openshift-insights/insights-operator-76bcd6ccf5-r5kxh"
Sep 09 20:41:21.158824 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.158816    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4eb46131-2f57-408b-9d0f-a9ee9b55c8ef-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-r5kxh\" (UID: \"4eb46131-2f57-408b-9d0f-a9ee9b55c8ef\") " pod="openshift-insights/insights-operator-76bcd6ccf5-r5kxh"
Sep 09 20:41:21.158972 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.158856    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4eb46131-2f57-408b-9d0f-a9ee9b55c8ef-serving-cert\") pod \"insights-operator-76bcd6ccf5-r5kxh\" (UID: \"4eb46131-2f57-408b-9d0f-a9ee9b55c8ef\") " pod="openshift-insights/insights-operator-76bcd6ccf5-r5kxh"
Sep 09 20:41:21.182376 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.182345    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-jx7vj"]
Sep 09 20:41:21.204408 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.204376    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-7f6c8f5bc8-h8tgh"]
Sep 09 20:41:21.204549 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.204486    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-jx7vj"
Sep 09 20:41:21.207307 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.207285    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 09 20:41:21.207784 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.207765    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 09 20:41:21.207784 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.207782    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-z5x86\""
Sep 09 20:41:21.218260 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.218236    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-jx7vj"]
Sep 09 20:41:21.218360 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.218271    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7f6c8f5bc8-h8tgh"]
Sep 09 20:41:21.218399 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.218353    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7f6c8f5bc8-h8tgh"
Sep 09 20:41:21.221570 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.221551    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 09 20:41:21.221682 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.221581    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 09 20:41:21.221682 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.221628    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 09 20:41:21.221682 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.221582    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 09 20:41:21.221893 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.221878    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 09 20:41:21.222020 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.222004    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-qblcs\""
Sep 09 20:41:21.222177 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.222161    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 09 20:41:21.260091 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.260009    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2b1f9a6e-7a79-4e89-ac3f-4658d6d91550-config\") pod \"console-operator-85b454874b-wsv4v\" (UID: \"2b1f9a6e-7a79-4e89-ac3f-4658d6d91550\") " pod="openshift-console-operator/console-operator-85b454874b-wsv4v"
Sep 09 20:41:21.260091 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.260048    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4eb46131-2f57-408b-9d0f-a9ee9b55c8ef-tmp\") pod \"insights-operator-76bcd6ccf5-r5kxh\" (UID: \"4eb46131-2f57-408b-9d0f-a9ee9b55c8ef\") " pod="openshift-insights/insights-operator-76bcd6ccf5-r5kxh"
Sep 09 20:41:21.260091 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.260075    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5q2nh\" (UniqueName: \"kubernetes.io/projected/4eb46131-2f57-408b-9d0f-a9ee9b55c8ef-kube-api-access-5q2nh\") pod \"insights-operator-76bcd6ccf5-r5kxh\" (UID: \"4eb46131-2f57-408b-9d0f-a9ee9b55c8ef\") " pod="openshift-insights/insights-operator-76bcd6ccf5-r5kxh"
Sep 09 20:41:21.260377 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.260131    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kttnh\" (UniqueName: \"kubernetes.io/projected/2b1f9a6e-7a79-4e89-ac3f-4658d6d91550-kube-api-access-kttnh\") pod \"console-operator-85b454874b-wsv4v\" (UID: \"2b1f9a6e-7a79-4e89-ac3f-4658d6d91550\") " pod="openshift-console-operator/console-operator-85b454874b-wsv4v"
Sep 09 20:41:21.260377 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.260158    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2b1f9a6e-7a79-4e89-ac3f-4658d6d91550-serving-cert\") pod \"console-operator-85b454874b-wsv4v\" (UID: \"2b1f9a6e-7a79-4e89-ac3f-4658d6d91550\") " pod="openshift-console-operator/console-operator-85b454874b-wsv4v"
Sep 09 20:41:21.260377 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.260185    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4eb46131-2f57-408b-9d0f-a9ee9b55c8ef-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-r5kxh\" (UID: \"4eb46131-2f57-408b-9d0f-a9ee9b55c8ef\") " pod="openshift-insights/insights-operator-76bcd6ccf5-r5kxh"
Sep 09 20:41:21.260377 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.260208    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4eb46131-2f57-408b-9d0f-a9ee9b55c8ef-serving-cert\") pod \"insights-operator-76bcd6ccf5-r5kxh\" (UID: \"4eb46131-2f57-408b-9d0f-a9ee9b55c8ef\") " pod="openshift-insights/insights-operator-76bcd6ccf5-r5kxh"
Sep 09 20:41:21.260377 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.260270    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/981c03df-6cd9-4e28-b060-2fd16739d669-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-gwc6n\" (UID: \"981c03df-6cd9-4e28-b060-2fd16739d669\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-gwc6n"
Sep 09 20:41:21.260377 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.260306    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4kqn\" (UniqueName: \"kubernetes.io/projected/981c03df-6cd9-4e28-b060-2fd16739d669-kube-api-access-l4kqn\") pod \"cluster-samples-operator-7cdcfcf76c-gwc6n\" (UID: \"981c03df-6cd9-4e28-b060-2fd16739d669\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-gwc6n"
Sep 09 20:41:21.260377 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.260325    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/4eb46131-2f57-408b-9d0f-a9ee9b55c8ef-snapshots\") pod \"insights-operator-76bcd6ccf5-r5kxh\" (UID: \"4eb46131-2f57-408b-9d0f-a9ee9b55c8ef\") " pod="openshift-insights/insights-operator-76bcd6ccf5-r5kxh"
Sep 09 20:41:21.260377 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.260343    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2b1f9a6e-7a79-4e89-ac3f-4658d6d91550-trusted-ca\") pod \"console-operator-85b454874b-wsv4v\" (UID: \"2b1f9a6e-7a79-4e89-ac3f-4658d6d91550\") " pod="openshift-console-operator/console-operator-85b454874b-wsv4v"
Sep 09 20:41:21.260377 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:21.260364    2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 09 20:41:21.260904 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.260394    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4eb46131-2f57-408b-9d0f-a9ee9b55c8ef-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-r5kxh\" (UID: \"4eb46131-2f57-408b-9d0f-a9ee9b55c8ef\") " pod="openshift-insights/insights-operator-76bcd6ccf5-r5kxh"
Sep 09 20:41:21.260904 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:21.260420    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/981c03df-6cd9-4e28-b060-2fd16739d669-samples-operator-tls podName:981c03df-6cd9-4e28-b060-2fd16739d669 nodeName:}" failed. No retries permitted until 2026-09-09 20:41:21.760402502 +0000 UTC m=+43.388030360 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/981c03df-6cd9-4e28-b060-2fd16739d669-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-gwc6n" (UID: "981c03df-6cd9-4e28-b060-2fd16739d669") : secret "samples-operator-tls" not found
Sep 09 20:41:21.260904 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.260587    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4eb46131-2f57-408b-9d0f-a9ee9b55c8ef-tmp\") pod \"insights-operator-76bcd6ccf5-r5kxh\" (UID: \"4eb46131-2f57-408b-9d0f-a9ee9b55c8ef\") " pod="openshift-insights/insights-operator-76bcd6ccf5-r5kxh"
Sep 09 20:41:21.260904 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.260713    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4eb46131-2f57-408b-9d0f-a9ee9b55c8ef-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-r5kxh\" (UID: \"4eb46131-2f57-408b-9d0f-a9ee9b55c8ef\") " pod="openshift-insights/insights-operator-76bcd6ccf5-r5kxh"
Sep 09 20:41:21.260904 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.260868    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/4eb46131-2f57-408b-9d0f-a9ee9b55c8ef-snapshots\") pod \"insights-operator-76bcd6ccf5-r5kxh\" (UID: \"4eb46131-2f57-408b-9d0f-a9ee9b55c8ef\") " pod="openshift-insights/insights-operator-76bcd6ccf5-r5kxh"
Sep 09 20:41:21.261140 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.261121    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4eb46131-2f57-408b-9d0f-a9ee9b55c8ef-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-r5kxh\" (UID: \"4eb46131-2f57-408b-9d0f-a9ee9b55c8ef\") " pod="openshift-insights/insights-operator-76bcd6ccf5-r5kxh"
Sep 09 20:41:21.263419 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.263401    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4eb46131-2f57-408b-9d0f-a9ee9b55c8ef-serving-cert\") pod \"insights-operator-76bcd6ccf5-r5kxh\" (UID: \"4eb46131-2f57-408b-9d0f-a9ee9b55c8ef\") " pod="openshift-insights/insights-operator-76bcd6ccf5-r5kxh"
Sep 09 20:41:21.273681 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.273660    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5q2nh\" (UniqueName: \"kubernetes.io/projected/4eb46131-2f57-408b-9d0f-a9ee9b55c8ef-kube-api-access-5q2nh\") pod \"insights-operator-76bcd6ccf5-r5kxh\" (UID: \"4eb46131-2f57-408b-9d0f-a9ee9b55c8ef\") " pod="openshift-insights/insights-operator-76bcd6ccf5-r5kxh"
Sep 09 20:41:21.273800 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.273783    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4kqn\" (UniqueName: \"kubernetes.io/projected/981c03df-6cd9-4e28-b060-2fd16739d669-kube-api-access-l4kqn\") pod \"cluster-samples-operator-7cdcfcf76c-gwc6n\" (UID: \"981c03df-6cd9-4e28-b060-2fd16739d669\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-gwc6n"
Sep 09 20:41:21.342751 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.342712    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-r5kxh"
Sep 09 20:41:21.361010 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.360981    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2b1f9a6e-7a79-4e89-ac3f-4658d6d91550-serving-cert\") pod \"console-operator-85b454874b-wsv4v\" (UID: \"2b1f9a6e-7a79-4e89-ac3f-4658d6d91550\") " pod="openshift-console-operator/console-operator-85b454874b-wsv4v"
Sep 09 20:41:21.361142 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.361017    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/d5d449a3-ab1b-4899-842d-fcc9accb0838-default-certificate\") pod \"router-default-7f6c8f5bc8-h8tgh\" (UID: \"d5d449a3-ab1b-4899-842d-fcc9accb0838\") " pod="openshift-ingress/router-default-7f6c8f5bc8-h8tgh"
Sep 09 20:41:21.361142 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.361034    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/d5d449a3-ab1b-4899-842d-fcc9accb0838-stats-auth\") pod \"router-default-7f6c8f5bc8-h8tgh\" (UID: \"d5d449a3-ab1b-4899-842d-fcc9accb0838\") " pod="openshift-ingress/router-default-7f6c8f5bc8-h8tgh"
Sep 09 20:41:21.361142 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.361119    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2b1f9a6e-7a79-4e89-ac3f-4658d6d91550-trusted-ca\") pod \"console-operator-85b454874b-wsv4v\" (UID: \"2b1f9a6e-7a79-4e89-ac3f-4658d6d91550\") " pod="openshift-console-operator/console-operator-85b454874b-wsv4v"
Sep 09 20:41:21.361245 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.361157    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5d449a3-ab1b-4899-842d-fcc9accb0838-metrics-certs\") pod \"router-default-7f6c8f5bc8-h8tgh\" (UID: \"d5d449a3-ab1b-4899-842d-fcc9accb0838\") " pod="openshift-ingress/router-default-7f6c8f5bc8-h8tgh"
Sep 09 20:41:21.361245 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.361200    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7c35fa5e-a4d9-48bb-9fcf-63cca4ed0fd2-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-jx7vj\" (UID: \"7c35fa5e-a4d9-48bb-9fcf-63cca4ed0fd2\") " pod="openshift-network-console/networking-console-plugin-79977f9844-jx7vj"
Sep 09 20:41:21.361245 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.361229    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mlgl7\" (UniqueName: \"kubernetes.io/projected/d5d449a3-ab1b-4899-842d-fcc9accb0838-kube-api-access-mlgl7\") pod \"router-default-7f6c8f5bc8-h8tgh\" (UID: \"d5d449a3-ab1b-4899-842d-fcc9accb0838\") " pod="openshift-ingress/router-default-7f6c8f5bc8-h8tgh"
Sep 09 20:41:21.361376 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.361349    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5d449a3-ab1b-4899-842d-fcc9accb0838-service-ca-bundle\") pod \"router-default-7f6c8f5bc8-h8tgh\" (UID: \"d5d449a3-ab1b-4899-842d-fcc9accb0838\") " pod="openshift-ingress/router-default-7f6c8f5bc8-h8tgh"
Sep 09 20:41:21.361449 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.361430    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2b1f9a6e-7a79-4e89-ac3f-4658d6d91550-config\") pod \"console-operator-85b454874b-wsv4v\" (UID: \"2b1f9a6e-7a79-4e89-ac3f-4658d6d91550\") " pod="openshift-console-operator/console-operator-85b454874b-wsv4v"
Sep 09 20:41:21.361504 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.361471    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kttnh\" (UniqueName: \"kubernetes.io/projected/2b1f9a6e-7a79-4e89-ac3f-4658d6d91550-kube-api-access-kttnh\") pod \"console-operator-85b454874b-wsv4v\" (UID: \"2b1f9a6e-7a79-4e89-ac3f-4658d6d91550\") " pod="openshift-console-operator/console-operator-85b454874b-wsv4v"
Sep 09 20:41:21.361558 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.361502    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7c35fa5e-a4d9-48bb-9fcf-63cca4ed0fd2-nginx-conf\") pod \"networking-console-plugin-79977f9844-jx7vj\" (UID: \"7c35fa5e-a4d9-48bb-9fcf-63cca4ed0fd2\") " pod="openshift-network-console/networking-console-plugin-79977f9844-jx7vj"
Sep 09 20:41:21.361995 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.361973    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2b1f9a6e-7a79-4e89-ac3f-4658d6d91550-config\") pod \"console-operator-85b454874b-wsv4v\" (UID: \"2b1f9a6e-7a79-4e89-ac3f-4658d6d91550\") " pod="openshift-console-operator/console-operator-85b454874b-wsv4v"
Sep 09 20:41:21.362061 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.362040    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2b1f9a6e-7a79-4e89-ac3f-4658d6d91550-trusted-ca\") pod \"console-operator-85b454874b-wsv4v\" (UID: \"2b1f9a6e-7a79-4e89-ac3f-4658d6d91550\") " pod="openshift-console-operator/console-operator-85b454874b-wsv4v"
Sep 09 20:41:21.363328 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.363313    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2b1f9a6e-7a79-4e89-ac3f-4658d6d91550-serving-cert\") pod \"console-operator-85b454874b-wsv4v\" (UID: \"2b1f9a6e-7a79-4e89-ac3f-4658d6d91550\") " pod="openshift-console-operator/console-operator-85b454874b-wsv4v"
Sep 09 20:41:21.369245 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.369228    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kttnh\" (UniqueName: \"kubernetes.io/projected/2b1f9a6e-7a79-4e89-ac3f-4658d6d91550-kube-api-access-kttnh\") pod \"console-operator-85b454874b-wsv4v\" (UID: \"2b1f9a6e-7a79-4e89-ac3f-4658d6d91550\") " pod="openshift-console-operator/console-operator-85b454874b-wsv4v"
Sep 09 20:41:21.387267 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.387246    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-wsv4v"
Sep 09 20:41:21.462957 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.462903    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5d449a3-ab1b-4899-842d-fcc9accb0838-metrics-certs\") pod \"router-default-7f6c8f5bc8-h8tgh\" (UID: \"d5d449a3-ab1b-4899-842d-fcc9accb0838\") " pod="openshift-ingress/router-default-7f6c8f5bc8-h8tgh"
Sep 09 20:41:21.463037 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.463001    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7c35fa5e-a4d9-48bb-9fcf-63cca4ed0fd2-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-jx7vj\" (UID: \"7c35fa5e-a4d9-48bb-9fcf-63cca4ed0fd2\") " pod="openshift-network-console/networking-console-plugin-79977f9844-jx7vj"
Sep 09 20:41:21.463037 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.463029    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mlgl7\" (UniqueName: \"kubernetes.io/projected/d5d449a3-ab1b-4899-842d-fcc9accb0838-kube-api-access-mlgl7\") pod \"router-default-7f6c8f5bc8-h8tgh\" (UID: \"d5d449a3-ab1b-4899-842d-fcc9accb0838\") " pod="openshift-ingress/router-default-7f6c8f5bc8-h8tgh"
Sep 09 20:41:21.463141 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.463083    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5d449a3-ab1b-4899-842d-fcc9accb0838-service-ca-bundle\") pod \"router-default-7f6c8f5bc8-h8tgh\" (UID: \"d5d449a3-ab1b-4899-842d-fcc9accb0838\") " pod="openshift-ingress/router-default-7f6c8f5bc8-h8tgh"
Sep 09 20:41:21.463281 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:21.463221    2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 09 20:41:21.463281 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:21.463268    2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 09 20:41:21.463533 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:21.463294    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c35fa5e-a4d9-48bb-9fcf-63cca4ed0fd2-networking-console-plugin-cert podName:7c35fa5e-a4d9-48bb-9fcf-63cca4ed0fd2 nodeName:}" failed. No retries permitted until 2026-09-09 20:41:21.963272533 +0000 UTC m=+43.590900400 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7c35fa5e-a4d9-48bb-9fcf-63cca4ed0fd2-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-jx7vj" (UID: "7c35fa5e-a4d9-48bb-9fcf-63cca4ed0fd2") : secret "networking-console-plugin-cert" not found
Sep 09 20:41:21.463533 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:21.463333    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5d449a3-ab1b-4899-842d-fcc9accb0838-metrics-certs podName:d5d449a3-ab1b-4899-842d-fcc9accb0838 nodeName:}" failed. No retries permitted until 2026-09-09 20:41:21.963313756 +0000 UTC m=+43.590941610 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d5d449a3-ab1b-4899-842d-fcc9accb0838-metrics-certs") pod "router-default-7f6c8f5bc8-h8tgh" (UID: "d5d449a3-ab1b-4899-842d-fcc9accb0838") : secret "router-metrics-certs-default" not found
Sep 09 20:41:21.463533 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:21.463499    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d5d449a3-ab1b-4899-842d-fcc9accb0838-service-ca-bundle podName:d5d449a3-ab1b-4899-842d-fcc9accb0838 nodeName:}" failed. No retries permitted until 2026-09-09 20:41:21.963484583 +0000 UTC m=+43.591112436 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d5d449a3-ab1b-4899-842d-fcc9accb0838-service-ca-bundle") pod "router-default-7f6c8f5bc8-h8tgh" (UID: "d5d449a3-ab1b-4899-842d-fcc9accb0838") : configmap references non-existent config key: service-ca.crt
Sep 09 20:41:21.463829 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.463653    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7c35fa5e-a4d9-48bb-9fcf-63cca4ed0fd2-nginx-conf\") pod \"networking-console-plugin-79977f9844-jx7vj\" (UID: \"7c35fa5e-a4d9-48bb-9fcf-63cca4ed0fd2\") " pod="openshift-network-console/networking-console-plugin-79977f9844-jx7vj"
Sep 09 20:41:21.463829 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.463695    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/d5d449a3-ab1b-4899-842d-fcc9accb0838-default-certificate\") pod \"router-default-7f6c8f5bc8-h8tgh\" (UID: \"d5d449a3-ab1b-4899-842d-fcc9accb0838\") " pod="openshift-ingress/router-default-7f6c8f5bc8-h8tgh"
Sep 09 20:41:21.463829 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.463716    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/d5d449a3-ab1b-4899-842d-fcc9accb0838-stats-auth\") pod \"router-default-7f6c8f5bc8-h8tgh\" (UID: \"d5d449a3-ab1b-4899-842d-fcc9accb0838\") " pod="openshift-ingress/router-default-7f6c8f5bc8-h8tgh"
Sep 09 20:41:21.464777 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.464753    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7c35fa5e-a4d9-48bb-9fcf-63cca4ed0fd2-nginx-conf\") pod \"networking-console-plugin-79977f9844-jx7vj\" (UID: \"7c35fa5e-a4d9-48bb-9fcf-63cca4ed0fd2\") " pod="openshift-network-console/networking-console-plugin-79977f9844-jx7vj"
Sep 09 20:41:21.466744 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.466702    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/d5d449a3-ab1b-4899-842d-fcc9accb0838-stats-auth\") pod \"router-default-7f6c8f5bc8-h8tgh\" (UID: \"d5d449a3-ab1b-4899-842d-fcc9accb0838\") " pod="openshift-ingress/router-default-7f6c8f5bc8-h8tgh"
Sep 09 20:41:21.467625 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.467602    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/d5d449a3-ab1b-4899-842d-fcc9accb0838-default-certificate\") pod \"router-default-7f6c8f5bc8-h8tgh\" (UID: \"d5d449a3-ab1b-4899-842d-fcc9accb0838\") " pod="openshift-ingress/router-default-7f6c8f5bc8-h8tgh"
Sep 09 20:41:21.472003 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.471950    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mlgl7\" (UniqueName: \"kubernetes.io/projected/d5d449a3-ab1b-4899-842d-fcc9accb0838-kube-api-access-mlgl7\") pod \"router-default-7f6c8f5bc8-h8tgh\" (UID: \"d5d449a3-ab1b-4899-842d-fcc9accb0838\") " pod="openshift-ingress/router-default-7f6c8f5bc8-h8tgh"
Sep 09 20:41:21.645666 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.645579    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-r5kxh"]
Sep 09 20:41:21.648934 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:41:21.648904    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4eb46131_2f57_408b_9d0f_a9ee9b55c8ef.slice/crio-20121e5115e4c46d7e44d7e44600dbd40bf374c17387c66e22dc815261c371ba WatchSource:0}: Error finding container 20121e5115e4c46d7e44d7e44600dbd40bf374c17387c66e22dc815261c371ba: Status 404 returned error can't find the container with id 20121e5115e4c46d7e44d7e44600dbd40bf374c17387c66e22dc815261c371ba
Sep 09 20:41:21.685782 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.685754    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-wsv4v"]
Sep 09 20:41:21.688972 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:41:21.688949    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b1f9a6e_7a79_4e89_ac3f_4658d6d91550.slice/crio-37c8b335a5eaaac9cdf56b55914364687251ff931ef8a8f125076b980580a7a6 WatchSource:0}: Error finding container 37c8b335a5eaaac9cdf56b55914364687251ff931ef8a8f125076b980580a7a6: Status 404 returned error can't find the container with id 37c8b335a5eaaac9cdf56b55914364687251ff931ef8a8f125076b980580a7a6
Sep 09 20:41:21.766060 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.766020    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/981c03df-6cd9-4e28-b060-2fd16739d669-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-gwc6n\" (UID: \"981c03df-6cd9-4e28-b060-2fd16739d669\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-gwc6n"
Sep 09 20:41:21.766282 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:21.766261    2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 09 20:41:21.766344 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:21.766331    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/981c03df-6cd9-4e28-b060-2fd16739d669-samples-operator-tls podName:981c03df-6cd9-4e28-b060-2fd16739d669 nodeName:}" failed. No retries permitted until 2026-09-09 20:41:22.766313067 +0000 UTC m=+44.393940918 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/981c03df-6cd9-4e28-b060-2fd16739d669-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-gwc6n" (UID: "981c03df-6cd9-4e28-b060-2fd16739d669") : secret "samples-operator-tls" not found
Sep 09 20:41:21.968344 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.968263    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5d449a3-ab1b-4899-842d-fcc9accb0838-metrics-certs\") pod \"router-default-7f6c8f5bc8-h8tgh\" (UID: \"d5d449a3-ab1b-4899-842d-fcc9accb0838\") " pod="openshift-ingress/router-default-7f6c8f5bc8-h8tgh"
Sep 09 20:41:21.968344 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.968307    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7c35fa5e-a4d9-48bb-9fcf-63cca4ed0fd2-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-jx7vj\" (UID: \"7c35fa5e-a4d9-48bb-9fcf-63cca4ed0fd2\") " pod="openshift-network-console/networking-console-plugin-79977f9844-jx7vj"
Sep 09 20:41:21.968344 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:21.968341    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5d449a3-ab1b-4899-842d-fcc9accb0838-service-ca-bundle\") pod \"router-default-7f6c8f5bc8-h8tgh\" (UID: \"d5d449a3-ab1b-4899-842d-fcc9accb0838\") " pod="openshift-ingress/router-default-7f6c8f5bc8-h8tgh"
Sep 09 20:41:21.968544 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:21.968415    2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 09 20:41:21.968544 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:21.968455    2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 09 20:41:21.968544 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:21.968482    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d5d449a3-ab1b-4899-842d-fcc9accb0838-service-ca-bundle podName:d5d449a3-ab1b-4899-842d-fcc9accb0838 nodeName:}" failed. No retries permitted until 2026-09-09 20:41:22.968469646 +0000 UTC m=+44.596097497 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d5d449a3-ab1b-4899-842d-fcc9accb0838-service-ca-bundle") pod "router-default-7f6c8f5bc8-h8tgh" (UID: "d5d449a3-ab1b-4899-842d-fcc9accb0838") : configmap references non-existent config key: service-ca.crt
Sep 09 20:41:21.968544 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:21.968501    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c35fa5e-a4d9-48bb-9fcf-63cca4ed0fd2-networking-console-plugin-cert podName:7c35fa5e-a4d9-48bb-9fcf-63cca4ed0fd2 nodeName:}" failed. No retries permitted until 2026-09-09 20:41:22.968488643 +0000 UTC m=+44.596116494 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7c35fa5e-a4d9-48bb-9fcf-63cca4ed0fd2-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-jx7vj" (UID: "7c35fa5e-a4d9-48bb-9fcf-63cca4ed0fd2") : secret "networking-console-plugin-cert" not found
Sep 09 20:41:21.968544 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:21.968516    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5d449a3-ab1b-4899-842d-fcc9accb0838-metrics-certs podName:d5d449a3-ab1b-4899-842d-fcc9accb0838 nodeName:}" failed. No retries permitted until 2026-09-09 20:41:22.968510198 +0000 UTC m=+44.596138049 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d5d449a3-ab1b-4899-842d-fcc9accb0838-metrics-certs") pod "router-default-7f6c8f5bc8-h8tgh" (UID: "d5d449a3-ab1b-4899-842d-fcc9accb0838") : secret "router-metrics-certs-default" not found
Sep 09 20:41:22.122847 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:22.122810    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-wsv4v" event={"ID":"2b1f9a6e-7a79-4e89-ac3f-4658d6d91550","Type":"ContainerStarted","Data":"37c8b335a5eaaac9cdf56b55914364687251ff931ef8a8f125076b980580a7a6"}
Sep 09 20:41:22.123950 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:22.123927    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-r5kxh" event={"ID":"4eb46131-2f57-408b-9d0f-a9ee9b55c8ef","Type":"ContainerStarted","Data":"20121e5115e4c46d7e44d7e44600dbd40bf374c17387c66e22dc815261c371ba"}
Sep 09 20:41:22.774508 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:22.774470    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/981c03df-6cd9-4e28-b060-2fd16739d669-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-gwc6n\" (UID: \"981c03df-6cd9-4e28-b060-2fd16739d669\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-gwc6n"
Sep 09 20:41:22.774761 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:22.774736    2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 09 20:41:22.774895 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:22.774813    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/981c03df-6cd9-4e28-b060-2fd16739d669-samples-operator-tls podName:981c03df-6cd9-4e28-b060-2fd16739d669 nodeName:}" failed. No retries permitted until 2026-09-09 20:41:24.774792259 +0000 UTC m=+46.402420125 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/981c03df-6cd9-4e28-b060-2fd16739d669-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-gwc6n" (UID: "981c03df-6cd9-4e28-b060-2fd16739d669") : secret "samples-operator-tls" not found
Sep 09 20:41:22.975837 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:22.975793    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5d449a3-ab1b-4899-842d-fcc9accb0838-service-ca-bundle\") pod \"router-default-7f6c8f5bc8-h8tgh\" (UID: \"d5d449a3-ab1b-4899-842d-fcc9accb0838\") " pod="openshift-ingress/router-default-7f6c8f5bc8-h8tgh"
Sep 09 20:41:22.976030 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:22.975917    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5d449a3-ab1b-4899-842d-fcc9accb0838-metrics-certs\") pod \"router-default-7f6c8f5bc8-h8tgh\" (UID: \"d5d449a3-ab1b-4899-842d-fcc9accb0838\") " pod="openshift-ingress/router-default-7f6c8f5bc8-h8tgh"
Sep 09 20:41:22.976030 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:22.975935    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d5d449a3-ab1b-4899-842d-fcc9accb0838-service-ca-bundle podName:d5d449a3-ab1b-4899-842d-fcc9accb0838 nodeName:}" failed. No retries permitted until 2026-09-09 20:41:24.975915339 +0000 UTC m=+46.603543208 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d5d449a3-ab1b-4899-842d-fcc9accb0838-service-ca-bundle") pod "router-default-7f6c8f5bc8-h8tgh" (UID: "d5d449a3-ab1b-4899-842d-fcc9accb0838") : configmap references non-existent config key: service-ca.crt
Sep 09 20:41:22.976030 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:22.975982    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7c35fa5e-a4d9-48bb-9fcf-63cca4ed0fd2-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-jx7vj\" (UID: \"7c35fa5e-a4d9-48bb-9fcf-63cca4ed0fd2\") " pod="openshift-network-console/networking-console-plugin-79977f9844-jx7vj"
Sep 09 20:41:22.976030 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:22.976001    2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 09 20:41:22.976281 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:22.976038    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5d449a3-ab1b-4899-842d-fcc9accb0838-metrics-certs podName:d5d449a3-ab1b-4899-842d-fcc9accb0838 nodeName:}" failed. No retries permitted until 2026-09-09 20:41:24.976026126 +0000 UTC m=+46.603653992 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d5d449a3-ab1b-4899-842d-fcc9accb0838-metrics-certs") pod "router-default-7f6c8f5bc8-h8tgh" (UID: "d5d449a3-ab1b-4899-842d-fcc9accb0838") : secret "router-metrics-certs-default" not found
Sep 09 20:41:22.976281 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:22.976061    2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 09 20:41:22.976281 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:22.976115    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c35fa5e-a4d9-48bb-9fcf-63cca4ed0fd2-networking-console-plugin-cert podName:7c35fa5e-a4d9-48bb-9fcf-63cca4ed0fd2 nodeName:}" failed. No retries permitted until 2026-09-09 20:41:24.976084725 +0000 UTC m=+46.603712575 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7c35fa5e-a4d9-48bb-9fcf-63cca4ed0fd2-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-jx7vj" (UID: "7c35fa5e-a4d9-48bb-9fcf-63cca4ed0fd2") : secret "networking-console-plugin-cert" not found
Sep 09 20:41:24.790038 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:24.789986    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/981c03df-6cd9-4e28-b060-2fd16739d669-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-gwc6n\" (UID: \"981c03df-6cd9-4e28-b060-2fd16739d669\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-gwc6n"
Sep 09 20:41:24.790422 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:24.790167    2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 09 20:41:24.790422 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:24.790271    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/981c03df-6cd9-4e28-b060-2fd16739d669-samples-operator-tls podName:981c03df-6cd9-4e28-b060-2fd16739d669 nodeName:}" failed. No retries permitted until 2026-09-09 20:41:28.790249412 +0000 UTC m=+50.417877262 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/981c03df-6cd9-4e28-b060-2fd16739d669-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-gwc6n" (UID: "981c03df-6cd9-4e28-b060-2fd16739d669") : secret "samples-operator-tls" not found
Sep 09 20:41:24.992002 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:24.991967    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5d449a3-ab1b-4899-842d-fcc9accb0838-service-ca-bundle\") pod \"router-default-7f6c8f5bc8-h8tgh\" (UID: \"d5d449a3-ab1b-4899-842d-fcc9accb0838\") " pod="openshift-ingress/router-default-7f6c8f5bc8-h8tgh"
Sep 09 20:41:24.992144 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:24.992123    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5d449a3-ab1b-4899-842d-fcc9accb0838-metrics-certs\") pod \"router-default-7f6c8f5bc8-h8tgh\" (UID: \"d5d449a3-ab1b-4899-842d-fcc9accb0838\") " pod="openshift-ingress/router-default-7f6c8f5bc8-h8tgh"
Sep 09 20:41:24.992220 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:24.992149    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d5d449a3-ab1b-4899-842d-fcc9accb0838-service-ca-bundle podName:d5d449a3-ab1b-4899-842d-fcc9accb0838 nodeName:}" failed. No retries permitted until 2026-09-09 20:41:28.992132284 +0000 UTC m=+50.619760150 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d5d449a3-ab1b-4899-842d-fcc9accb0838-service-ca-bundle") pod "router-default-7f6c8f5bc8-h8tgh" (UID: "d5d449a3-ab1b-4899-842d-fcc9accb0838") : configmap references non-existent config key: service-ca.crt
Sep 09 20:41:24.992220 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:24.992206    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7c35fa5e-a4d9-48bb-9fcf-63cca4ed0fd2-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-jx7vj\" (UID: \"7c35fa5e-a4d9-48bb-9fcf-63cca4ed0fd2\") " pod="openshift-network-console/networking-console-plugin-79977f9844-jx7vj"
Sep 09 20:41:24.992328 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:24.992230    2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 09 20:41:24.992328 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:24.992280    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5d449a3-ab1b-4899-842d-fcc9accb0838-metrics-certs podName:d5d449a3-ab1b-4899-842d-fcc9accb0838 nodeName:}" failed. No retries permitted until 2026-09-09 20:41:28.992264087 +0000 UTC m=+50.619891958 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d5d449a3-ab1b-4899-842d-fcc9accb0838-metrics-certs") pod "router-default-7f6c8f5bc8-h8tgh" (UID: "d5d449a3-ab1b-4899-842d-fcc9accb0838") : secret "router-metrics-certs-default" not found
Sep 09 20:41:24.992328 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:24.992301    2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 09 20:41:24.992468 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:24.992334    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c35fa5e-a4d9-48bb-9fcf-63cca4ed0fd2-networking-console-plugin-cert podName:7c35fa5e-a4d9-48bb-9fcf-63cca4ed0fd2 nodeName:}" failed. No retries permitted until 2026-09-09 20:41:28.992324876 +0000 UTC m=+50.619952727 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7c35fa5e-a4d9-48bb-9fcf-63cca4ed0fd2-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-jx7vj" (UID: "7c35fa5e-a4d9-48bb-9fcf-63cca4ed0fd2") : secret "networking-console-plugin-cert" not found
Sep 09 20:41:25.132617 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:25.132588    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-wsv4v_2b1f9a6e-7a79-4e89-ac3f-4658d6d91550/console-operator/0.log"
Sep 09 20:41:25.132791 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:25.132628    2581 generic.go:356] "Generic (PLEG): container finished" podID="2b1f9a6e-7a79-4e89-ac3f-4658d6d91550" containerID="ede3567086b0cd2fc442421c87c44aebb4a654571f517fe6714a18e7e380c87b" exitCode=255
Sep 09 20:41:25.132791 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:25.132658    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-wsv4v" event={"ID":"2b1f9a6e-7a79-4e89-ac3f-4658d6d91550","Type":"ContainerDied","Data":"ede3567086b0cd2fc442421c87c44aebb4a654571f517fe6714a18e7e380c87b"}
Sep 09 20:41:25.132907 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:25.132880    2581 scope.go:117] "RemoveContainer" containerID="ede3567086b0cd2fc442421c87c44aebb4a654571f517fe6714a18e7e380c87b"
Sep 09 20:41:25.134254 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:25.134231    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-r5kxh" event={"ID":"4eb46131-2f57-408b-9d0f-a9ee9b55c8ef","Type":"ContainerStarted","Data":"e1402a01c29e9c1f6d000bf974ccaa2182d83bec3a8e9c4f33f1c337b944ed3c"}
Sep 09 20:41:25.165122 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:25.165028    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-76bcd6ccf5-r5kxh" podStartSLOduration=1.997405976 podStartE2EDuration="5.165013521s" totalImagesPullingTime="3.167607545s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:41:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:41:25.164645975 +0000 UTC m=+46.792273857" watchObservedRunningTime="2026-09-09 20:41:25.165013521 +0000 UTC m=+46.792641393"
Sep 09 20:41:26.138360 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:26.138331    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-wsv4v_2b1f9a6e-7a79-4e89-ac3f-4658d6d91550/console-operator/1.log"
Sep 09 20:41:26.138839 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:26.138787    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-wsv4v_2b1f9a6e-7a79-4e89-ac3f-4658d6d91550/console-operator/0.log"
Sep 09 20:41:26.138912 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:26.138832    2581 generic.go:356] "Generic (PLEG): container finished" podID="2b1f9a6e-7a79-4e89-ac3f-4658d6d91550" containerID="84ee389d25fbe5e33a540a69b56e2c9bd0a9ff1336e61f918cd1bf6414645c75" exitCode=255
Sep 09 20:41:26.138960 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:26.138907    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-wsv4v" event={"ID":"2b1f9a6e-7a79-4e89-ac3f-4658d6d91550","Type":"ContainerDied","Data":"84ee389d25fbe5e33a540a69b56e2c9bd0a9ff1336e61f918cd1bf6414645c75"}
Sep 09 20:41:26.139007 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:26.138974    2581 scope.go:117] "RemoveContainer" containerID="ede3567086b0cd2fc442421c87c44aebb4a654571f517fe6714a18e7e380c87b"
Sep 09 20:41:26.139224 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:26.139199    2581 scope.go:117] "RemoveContainer" containerID="84ee389d25fbe5e33a540a69b56e2c9bd0a9ff1336e61f918cd1bf6414645c75"
Sep 09 20:41:26.139431 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:26.139408    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-wsv4v_openshift-console-operator(2b1f9a6e-7a79-4e89-ac3f-4658d6d91550)\"" pod="openshift-console-operator/console-operator-85b454874b-wsv4v" podUID="2b1f9a6e-7a79-4e89-ac3f-4658d6d91550"
Sep 09 20:41:26.549736 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:26.549702    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-2ptr7"]
Sep 09 20:41:26.583209 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:26.583182    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2ptr7"]
Sep 09 20:41:26.583356 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:26.583291    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2ptr7"
Sep 09 20:41:26.585578 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:26.585554    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 09 20:41:26.585723 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:26.585648    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 09 20:41:26.585723 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:26.585697    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-r8hx4\""
Sep 09 20:41:26.706904 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:26.706869    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/64dbb975-388d-48c4-bd16-7b33658225d0-data-volume\") pod \"insights-runtime-extractor-2ptr7\" (UID: \"64dbb975-388d-48c4-bd16-7b33658225d0\") " pod="openshift-insights/insights-runtime-extractor-2ptr7"
Sep 09 20:41:26.706904 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:26.706913    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/64dbb975-388d-48c4-bd16-7b33658225d0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2ptr7\" (UID: \"64dbb975-388d-48c4-bd16-7b33658225d0\") " pod="openshift-insights/insights-runtime-extractor-2ptr7"
Sep 09 20:41:26.707195 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:26.706976    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/64dbb975-388d-48c4-bd16-7b33658225d0-crio-socket\") pod \"insights-runtime-extractor-2ptr7\" (UID: \"64dbb975-388d-48c4-bd16-7b33658225d0\") " pod="openshift-insights/insights-runtime-extractor-2ptr7"
Sep 09 20:41:26.707195 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:26.706999    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lzzg\" (UniqueName: \"kubernetes.io/projected/64dbb975-388d-48c4-bd16-7b33658225d0-kube-api-access-5lzzg\") pod \"insights-runtime-extractor-2ptr7\" (UID: \"64dbb975-388d-48c4-bd16-7b33658225d0\") " pod="openshift-insights/insights-runtime-extractor-2ptr7"
Sep 09 20:41:26.707195 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:26.707123    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/64dbb975-388d-48c4-bd16-7b33658225d0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2ptr7\" (UID: \"64dbb975-388d-48c4-bd16-7b33658225d0\") " pod="openshift-insights/insights-runtime-extractor-2ptr7"
Sep 09 20:41:26.808173 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:26.808051    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/64dbb975-388d-48c4-bd16-7b33658225d0-data-volume\") pod \"insights-runtime-extractor-2ptr7\" (UID: \"64dbb975-388d-48c4-bd16-7b33658225d0\") " pod="openshift-insights/insights-runtime-extractor-2ptr7"
Sep 09 20:41:26.808173 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:26.808094    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/64dbb975-388d-48c4-bd16-7b33658225d0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2ptr7\" (UID: \"64dbb975-388d-48c4-bd16-7b33658225d0\") " pod="openshift-insights/insights-runtime-extractor-2ptr7"
Sep 09 20:41:26.808379 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:26.808176    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/64dbb975-388d-48c4-bd16-7b33658225d0-crio-socket\") pod \"insights-runtime-extractor-2ptr7\" (UID: \"64dbb975-388d-48c4-bd16-7b33658225d0\") " pod="openshift-insights/insights-runtime-extractor-2ptr7"
Sep 09 20:41:26.808379 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:26.808307    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 20:41:26.808379 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:26.808360    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lzzg\" (UniqueName: \"kubernetes.io/projected/64dbb975-388d-48c4-bd16-7b33658225d0-kube-api-access-5lzzg\") pod \"insights-runtime-extractor-2ptr7\" (UID: \"64dbb975-388d-48c4-bd16-7b33658225d0\") " pod="openshift-insights/insights-runtime-extractor-2ptr7"
Sep 09 20:41:26.808379 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:26.808374    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/64dbb975-388d-48c4-bd16-7b33658225d0-crio-socket\") pod \"insights-runtime-extractor-2ptr7\" (UID: \"64dbb975-388d-48c4-bd16-7b33658225d0\") " pod="openshift-insights/insights-runtime-extractor-2ptr7"
Sep 09 20:41:26.811661 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:26.808540    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64dbb975-388d-48c4-bd16-7b33658225d0-insights-runtime-extractor-tls podName:64dbb975-388d-48c4-bd16-7b33658225d0 nodeName:}" failed. No retries permitted until 2026-09-09 20:41:27.308350545 +0000 UTC m=+48.935978396 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/64dbb975-388d-48c4-bd16-7b33658225d0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2ptr7" (UID: "64dbb975-388d-48c4-bd16-7b33658225d0") : secret "insights-runtime-extractor-tls" not found
Sep 09 20:41:26.811661 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:26.808735    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/64dbb975-388d-48c4-bd16-7b33658225d0-data-volume\") pod \"insights-runtime-extractor-2ptr7\" (UID: \"64dbb975-388d-48c4-bd16-7b33658225d0\") " pod="openshift-insights/insights-runtime-extractor-2ptr7"
Sep 09 20:41:26.811661 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:26.809064    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/64dbb975-388d-48c4-bd16-7b33658225d0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2ptr7\" (UID: \"64dbb975-388d-48c4-bd16-7b33658225d0\") " pod="openshift-insights/insights-runtime-extractor-2ptr7"
Sep 09 20:41:26.811661 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:26.809259    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 20:41:26.811661 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:26.809323    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/64dbb975-388d-48c4-bd16-7b33658225d0-kube-rbac-proxy-cm podName:64dbb975-388d-48c4-bd16-7b33658225d0 nodeName:}" failed. No retries permitted until 2026-09-09 20:41:27.309302603 +0000 UTC m=+48.936930466 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/64dbb975-388d-48c4-bd16-7b33658225d0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2ptr7" (UID: "64dbb975-388d-48c4-bd16-7b33658225d0") : configmap "kube-rbac-proxy" not found
Sep 09 20:41:26.816864 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:26.816837    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lzzg\" (UniqueName: \"kubernetes.io/projected/64dbb975-388d-48c4-bd16-7b33658225d0-kube-api-access-5lzzg\") pod \"insights-runtime-extractor-2ptr7\" (UID: \"64dbb975-388d-48c4-bd16-7b33658225d0\") " pod="openshift-insights/insights-runtime-extractor-2ptr7"
Sep 09 20:41:27.112555 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:27.112466    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/48a52e3f-68fb-496f-b974-320a8539e7f7-metrics-tls\") pod \"dns-default-g2tsp\" (UID: \"48a52e3f-68fb-496f-b974-320a8539e7f7\") " pod="openshift-dns/dns-default-g2tsp"
Sep 09 20:41:27.112725 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:27.112645    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 20:41:27.112725 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:27.112721    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48a52e3f-68fb-496f-b974-320a8539e7f7-metrics-tls podName:48a52e3f-68fb-496f-b974-320a8539e7f7 nodeName:}" failed. No retries permitted until 2026-09-09 20:41:43.112701066 +0000 UTC m=+64.740328938 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/48a52e3f-68fb-496f-b974-320a8539e7f7-metrics-tls") pod "dns-default-g2tsp" (UID: "48a52e3f-68fb-496f-b974-320a8539e7f7") : secret "dns-default-metrics-tls" not found
Sep 09 20:41:27.143300 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:27.143274    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-wsv4v_2b1f9a6e-7a79-4e89-ac3f-4658d6d91550/console-operator/1.log"
Sep 09 20:41:27.143716 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:27.143583    2581 scope.go:117] "RemoveContainer" containerID="84ee389d25fbe5e33a540a69b56e2c9bd0a9ff1336e61f918cd1bf6414645c75"
Sep 09 20:41:27.143758 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:27.143727    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-wsv4v_openshift-console-operator(2b1f9a6e-7a79-4e89-ac3f-4658d6d91550)\"" pod="openshift-console-operator/console-operator-85b454874b-wsv4v" podUID="2b1f9a6e-7a79-4e89-ac3f-4658d6d91550"
Sep 09 20:41:27.314600 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:27.314567    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/64dbb975-388d-48c4-bd16-7b33658225d0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2ptr7\" (UID: \"64dbb975-388d-48c4-bd16-7b33658225d0\") " pod="openshift-insights/insights-runtime-extractor-2ptr7"
Sep 09 20:41:27.314766 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:27.314682    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 20:41:27.314766 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:27.314692    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/64dbb975-388d-48c4-bd16-7b33658225d0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2ptr7\" (UID: \"64dbb975-388d-48c4-bd16-7b33658225d0\") " pod="openshift-insights/insights-runtime-extractor-2ptr7"
Sep 09 20:41:27.314766 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:27.314736    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/64dbb975-388d-48c4-bd16-7b33658225d0-kube-rbac-proxy-cm podName:64dbb975-388d-48c4-bd16-7b33658225d0 nodeName:}" failed. No retries permitted until 2026-09-09 20:41:28.314721982 +0000 UTC m=+49.942349832 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/64dbb975-388d-48c4-bd16-7b33658225d0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2ptr7" (UID: "64dbb975-388d-48c4-bd16-7b33658225d0") : configmap "kube-rbac-proxy" not found
Sep 09 20:41:27.314938 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:27.314855    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 20:41:27.314988 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:27.314945    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64dbb975-388d-48c4-bd16-7b33658225d0-insights-runtime-extractor-tls podName:64dbb975-388d-48c4-bd16-7b33658225d0 nodeName:}" failed. No retries permitted until 2026-09-09 20:41:28.314926199 +0000 UTC m=+49.942554066 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/64dbb975-388d-48c4-bd16-7b33658225d0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2ptr7" (UID: "64dbb975-388d-48c4-bd16-7b33658225d0") : secret "insights-runtime-extractor-tls" not found
Sep 09 20:41:28.322927 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:28.322887    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/64dbb975-388d-48c4-bd16-7b33658225d0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2ptr7\" (UID: \"64dbb975-388d-48c4-bd16-7b33658225d0\") " pod="openshift-insights/insights-runtime-extractor-2ptr7"
Sep 09 20:41:28.323343 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:28.323013    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 20:41:28.323343 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:28.323080    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/64dbb975-388d-48c4-bd16-7b33658225d0-kube-rbac-proxy-cm podName:64dbb975-388d-48c4-bd16-7b33658225d0 nodeName:}" failed. No retries permitted until 2026-09-09 20:41:30.323063909 +0000 UTC m=+51.950691760 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/64dbb975-388d-48c4-bd16-7b33658225d0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2ptr7" (UID: "64dbb975-388d-48c4-bd16-7b33658225d0") : configmap "kube-rbac-proxy" not found
Sep 09 20:41:28.323343 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:28.323141    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/64dbb975-388d-48c4-bd16-7b33658225d0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2ptr7\" (UID: \"64dbb975-388d-48c4-bd16-7b33658225d0\") " pod="openshift-insights/insights-runtime-extractor-2ptr7"
Sep 09 20:41:28.323343 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:28.323254    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 20:41:28.323343 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:28.323284    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64dbb975-388d-48c4-bd16-7b33658225d0-insights-runtime-extractor-tls podName:64dbb975-388d-48c4-bd16-7b33658225d0 nodeName:}" failed. No retries permitted until 2026-09-09 20:41:30.323277514 +0000 UTC m=+51.950905364 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/64dbb975-388d-48c4-bd16-7b33658225d0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2ptr7" (UID: "64dbb975-388d-48c4-bd16-7b33658225d0") : secret "insights-runtime-extractor-tls" not found
Sep 09 20:41:28.496321 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:28.496287    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-wg2t6"]
Sep 09 20:41:28.498307 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:28.498291    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-wg2t6"
Sep 09 20:41:28.501230 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:28.501206    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 09 20:41:28.501230 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:28.501222    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-ngfb9\""
Sep 09 20:41:28.502265 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:28.502245    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 09 20:41:28.514341 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:28.514314    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-wg2t6"]
Sep 09 20:41:28.517095 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:28.517077    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-cfg6s_5096efd2-6fbf-45a2-930b-acdc3970eabd/dns-node-resolver/0.log"
Sep 09 20:41:28.524518 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:28.524497    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbrjv\" (UniqueName: \"kubernetes.io/projected/ccb53e29-73b5-4a60-a39e-51354976c399-kube-api-access-cbrjv\") pod \"migrator-fdbb8fdff-wg2t6\" (UID: \"ccb53e29-73b5-4a60-a39e-51354976c399\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-wg2t6"
Sep 09 20:41:28.625047 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:28.624960    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbrjv\" (UniqueName: \"kubernetes.io/projected/ccb53e29-73b5-4a60-a39e-51354976c399-kube-api-access-cbrjv\") pod \"migrator-fdbb8fdff-wg2t6\" (UID: \"ccb53e29-73b5-4a60-a39e-51354976c399\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-wg2t6"
Sep 09 20:41:28.633884 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:28.633860    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbrjv\" (UniqueName: \"kubernetes.io/projected/ccb53e29-73b5-4a60-a39e-51354976c399-kube-api-access-cbrjv\") pod \"migrator-fdbb8fdff-wg2t6\" (UID: \"ccb53e29-73b5-4a60-a39e-51354976c399\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-wg2t6"
Sep 09 20:41:28.806977 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:28.806952    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-wg2t6"
Sep 09 20:41:28.826596 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:28.826567    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/981c03df-6cd9-4e28-b060-2fd16739d669-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-gwc6n\" (UID: \"981c03df-6cd9-4e28-b060-2fd16739d669\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-gwc6n"
Sep 09 20:41:28.826709 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:28.826692    2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 09 20:41:28.826760 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:28.826751    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/981c03df-6cd9-4e28-b060-2fd16739d669-samples-operator-tls podName:981c03df-6cd9-4e28-b060-2fd16739d669 nodeName:}" failed. No retries permitted until 2026-09-09 20:41:36.82673586 +0000 UTC m=+58.454363717 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/981c03df-6cd9-4e28-b060-2fd16739d669-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-gwc6n" (UID: "981c03df-6cd9-4e28-b060-2fd16739d669") : secret "samples-operator-tls" not found
Sep 09 20:41:29.028024 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:29.027987    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5d449a3-ab1b-4899-842d-fcc9accb0838-metrics-certs\") pod \"router-default-7f6c8f5bc8-h8tgh\" (UID: \"d5d449a3-ab1b-4899-842d-fcc9accb0838\") " pod="openshift-ingress/router-default-7f6c8f5bc8-h8tgh"
Sep 09 20:41:29.028202 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:29.028041    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7c35fa5e-a4d9-48bb-9fcf-63cca4ed0fd2-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-jx7vj\" (UID: \"7c35fa5e-a4d9-48bb-9fcf-63cca4ed0fd2\") " pod="openshift-network-console/networking-console-plugin-79977f9844-jx7vj"
Sep 09 20:41:29.028202 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:29.028141    2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 09 20:41:29.028202 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:29.028200    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5d449a3-ab1b-4899-842d-fcc9accb0838-metrics-certs podName:d5d449a3-ab1b-4899-842d-fcc9accb0838 nodeName:}" failed. No retries permitted until 2026-09-09 20:41:37.028185809 +0000 UTC m=+58.655813660 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d5d449a3-ab1b-4899-842d-fcc9accb0838-metrics-certs") pod "router-default-7f6c8f5bc8-h8tgh" (UID: "d5d449a3-ab1b-4899-842d-fcc9accb0838") : secret "router-metrics-certs-default" not found
Sep 09 20:41:29.028334 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:29.028142    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5d449a3-ab1b-4899-842d-fcc9accb0838-service-ca-bundle\") pod \"router-default-7f6c8f5bc8-h8tgh\" (UID: \"d5d449a3-ab1b-4899-842d-fcc9accb0838\") " pod="openshift-ingress/router-default-7f6c8f5bc8-h8tgh"
Sep 09 20:41:29.028334 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:29.028205    2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 09 20:41:29.028334 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:29.028231    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d5d449a3-ab1b-4899-842d-fcc9accb0838-service-ca-bundle podName:d5d449a3-ab1b-4899-842d-fcc9accb0838 nodeName:}" failed. No retries permitted until 2026-09-09 20:41:37.028218082 +0000 UTC m=+58.655845953 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d5d449a3-ab1b-4899-842d-fcc9accb0838-service-ca-bundle") pod "router-default-7f6c8f5bc8-h8tgh" (UID: "d5d449a3-ab1b-4899-842d-fcc9accb0838") : configmap references non-existent config key: service-ca.crt
Sep 09 20:41:29.028334 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:29.028265    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c35fa5e-a4d9-48bb-9fcf-63cca4ed0fd2-networking-console-plugin-cert podName:7c35fa5e-a4d9-48bb-9fcf-63cca4ed0fd2 nodeName:}" failed. No retries permitted until 2026-09-09 20:41:37.028251577 +0000 UTC m=+58.655879432 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7c35fa5e-a4d9-48bb-9fcf-63cca4ed0fd2-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-jx7vj" (UID: "7c35fa5e-a4d9-48bb-9fcf-63cca4ed0fd2") : secret "networking-console-plugin-cert" not found
Sep 09 20:41:29.071543 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:29.071484    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-wg2t6"]
Sep 09 20:41:29.074844 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:41:29.074813    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccb53e29_73b5_4a60_a39e_51354976c399.slice/crio-ee2efb1fe592892fde97141544abf9be2395cb136969a120a04849c1bc26a663 WatchSource:0}: Error finding container ee2efb1fe592892fde97141544abf9be2395cb136969a120a04849c1bc26a663: Status 404 returned error can't find the container with id ee2efb1fe592892fde97141544abf9be2395cb136969a120a04849c1bc26a663
Sep 09 20:41:29.149262 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:29.149228    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-wg2t6" event={"ID":"ccb53e29-73b5-4a60-a39e-51354976c399","Type":"ContainerStarted","Data":"ee2efb1fe592892fde97141544abf9be2395cb136969a120a04849c1bc26a663"}
Sep 09 20:41:29.511685 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:29.511661    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-f8mqt_0104bf49-c8d9-493a-b189-e570ebf93e62/node-ca/0.log"
Sep 09 20:41:30.113496 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:30.113459    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4zgnb_933d2649-0bcc-4c52-a0ac-65f33ba49741/serve-healthcheck-canary/0.log"
Sep 09 20:41:30.337717 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:30.337680    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/64dbb975-388d-48c4-bd16-7b33658225d0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2ptr7\" (UID: \"64dbb975-388d-48c4-bd16-7b33658225d0\") " pod="openshift-insights/insights-runtime-extractor-2ptr7"
Sep 09 20:41:30.337916 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:30.337782    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/64dbb975-388d-48c4-bd16-7b33658225d0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2ptr7\" (UID: \"64dbb975-388d-48c4-bd16-7b33658225d0\") " pod="openshift-insights/insights-runtime-extractor-2ptr7"
Sep 09 20:41:30.337916 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:30.337848    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 20:41:30.338038 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:30.337929    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64dbb975-388d-48c4-bd16-7b33658225d0-insights-runtime-extractor-tls podName:64dbb975-388d-48c4-bd16-7b33658225d0 nodeName:}" failed. No retries permitted until 2026-09-09 20:41:34.337906963 +0000 UTC m=+55.965534824 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/64dbb975-388d-48c4-bd16-7b33658225d0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2ptr7" (UID: "64dbb975-388d-48c4-bd16-7b33658225d0") : secret "insights-runtime-extractor-tls" not found
Sep 09 20:41:30.338038 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:30.337935    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 20:41:30.338038 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:30.337976    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/64dbb975-388d-48c4-bd16-7b33658225d0-kube-rbac-proxy-cm podName:64dbb975-388d-48c4-bd16-7b33658225d0 nodeName:}" failed. No retries permitted until 2026-09-09 20:41:34.337965292 +0000 UTC m=+55.965593143 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/64dbb975-388d-48c4-bd16-7b33658225d0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2ptr7" (UID: "64dbb975-388d-48c4-bd16-7b33658225d0") : configmap "kube-rbac-proxy" not found
Sep 09 20:41:31.157239 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:31.157200    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-wg2t6" event={"ID":"ccb53e29-73b5-4a60-a39e-51354976c399","Type":"ContainerStarted","Data":"69234f4b8a3462b228d91e28912e5020b5c7086878f1757432003caa66c6169b"}
Sep 09 20:41:31.157633 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:31.157258    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-wg2t6" event={"ID":"ccb53e29-73b5-4a60-a39e-51354976c399","Type":"ContainerStarted","Data":"4e96298254ac9ed49c3d0ade6ea206fbf8c06038f8dc3c1d749f1fed49dc57ec"}
Sep 09 20:41:31.178550 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:31.178496    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-wg2t6" podStartSLOduration=1.820766807 podStartE2EDuration="3.178481919s" totalImagesPullingTime="1.357715112s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:41:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:41:31.176722328 +0000 UTC m=+52.804350201" watchObservedRunningTime="2026-09-09 20:41:31.178481919 +0000 UTC m=+52.806109814"
Sep 09 20:41:31.388419 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:31.388374    2581 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-wsv4v"
Sep 09 20:41:31.388612 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:31.388442    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-wsv4v"
Sep 09 20:41:31.388832 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:31.388815    2581 scope.go:117] "RemoveContainer" containerID="84ee389d25fbe5e33a540a69b56e2c9bd0a9ff1336e61f918cd1bf6414645c75"
Sep 09 20:41:31.389010 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:31.388982    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-wsv4v_openshift-console-operator(2b1f9a6e-7a79-4e89-ac3f-4658d6d91550)\"" pod="openshift-console-operator/console-operator-85b454874b-wsv4v" podUID="2b1f9a6e-7a79-4e89-ac3f-4658d6d91550"
Sep 09 20:41:34.369502 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:34.369454    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/64dbb975-388d-48c4-bd16-7b33658225d0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2ptr7\" (UID: \"64dbb975-388d-48c4-bd16-7b33658225d0\") " pod="openshift-insights/insights-runtime-extractor-2ptr7"
Sep 09 20:41:34.369961 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:34.369588    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 20:41:34.369961 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:34.369658    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/64dbb975-388d-48c4-bd16-7b33658225d0-kube-rbac-proxy-cm podName:64dbb975-388d-48c4-bd16-7b33658225d0 nodeName:}" failed. No retries permitted until 2026-09-09 20:41:42.369642657 +0000 UTC m=+63.997270514 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/64dbb975-388d-48c4-bd16-7b33658225d0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2ptr7" (UID: "64dbb975-388d-48c4-bd16-7b33658225d0") : configmap "kube-rbac-proxy" not found
Sep 09 20:41:34.369961 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:34.369691    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/64dbb975-388d-48c4-bd16-7b33658225d0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2ptr7\" (UID: \"64dbb975-388d-48c4-bd16-7b33658225d0\") " pod="openshift-insights/insights-runtime-extractor-2ptr7"
Sep 09 20:41:34.369961 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:34.369796    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 20:41:34.369961 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:34.369825    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64dbb975-388d-48c4-bd16-7b33658225d0-insights-runtime-extractor-tls podName:64dbb975-388d-48c4-bd16-7b33658225d0 nodeName:}" failed. No retries permitted until 2026-09-09 20:41:42.369817999 +0000 UTC m=+63.997445849 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/64dbb975-388d-48c4-bd16-7b33658225d0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2ptr7" (UID: "64dbb975-388d-48c4-bd16-7b33658225d0") : secret "insights-runtime-extractor-tls" not found
Sep 09 20:41:36.887898 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:36.887857    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/981c03df-6cd9-4e28-b060-2fd16739d669-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-gwc6n\" (UID: \"981c03df-6cd9-4e28-b060-2fd16739d669\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-gwc6n"
Sep 09 20:41:36.890354 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:36.890326    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/981c03df-6cd9-4e28-b060-2fd16739d669-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-gwc6n\" (UID: \"981c03df-6cd9-4e28-b060-2fd16739d669\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-gwc6n"
Sep 09 20:41:36.960259 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:36.960210    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-gwc6n"
Sep 09 20:41:37.081392 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:37.081362    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-75wj4"
Sep 09 20:41:37.089535 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:37.089508    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5d449a3-ab1b-4899-842d-fcc9accb0838-service-ca-bundle\") pod \"router-default-7f6c8f5bc8-h8tgh\" (UID: \"d5d449a3-ab1b-4899-842d-fcc9accb0838\") " pod="openshift-ingress/router-default-7f6c8f5bc8-h8tgh"
Sep 09 20:41:37.089683 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:37.089559    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5d449a3-ab1b-4899-842d-fcc9accb0838-metrics-certs\") pod \"router-default-7f6c8f5bc8-h8tgh\" (UID: \"d5d449a3-ab1b-4899-842d-fcc9accb0838\") " pod="openshift-ingress/router-default-7f6c8f5bc8-h8tgh"
Sep 09 20:41:37.089683 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:37.089662    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7c35fa5e-a4d9-48bb-9fcf-63cca4ed0fd2-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-jx7vj\" (UID: \"7c35fa5e-a4d9-48bb-9fcf-63cca4ed0fd2\") " pod="openshift-network-console/networking-console-plugin-79977f9844-jx7vj"
Sep 09 20:41:37.089807 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:37.089734    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d5d449a3-ab1b-4899-842d-fcc9accb0838-service-ca-bundle podName:d5d449a3-ab1b-4899-842d-fcc9accb0838 nodeName:}" failed. No retries permitted until 2026-09-09 20:41:53.08970414 +0000 UTC m=+74.717332008 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d5d449a3-ab1b-4899-842d-fcc9accb0838-service-ca-bundle") pod "router-default-7f6c8f5bc8-h8tgh" (UID: "d5d449a3-ab1b-4899-842d-fcc9accb0838") : configmap references non-existent config key: service-ca.crt
Sep 09 20:41:37.089807 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:37.089797    2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 09 20:41:37.089914 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:41:37.089853    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c35fa5e-a4d9-48bb-9fcf-63cca4ed0fd2-networking-console-plugin-cert podName:7c35fa5e-a4d9-48bb-9fcf-63cca4ed0fd2 nodeName:}" failed. No retries permitted until 2026-09-09 20:41:53.089838353 +0000 UTC m=+74.717466207 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7c35fa5e-a4d9-48bb-9fcf-63cca4ed0fd2-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-jx7vj" (UID: "7c35fa5e-a4d9-48bb-9fcf-63cca4ed0fd2") : secret "networking-console-plugin-cert" not found
Sep 09 20:41:37.091846 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:37.091826    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5d449a3-ab1b-4899-842d-fcc9accb0838-metrics-certs\") pod \"router-default-7f6c8f5bc8-h8tgh\" (UID: \"d5d449a3-ab1b-4899-842d-fcc9accb0838\") " pod="openshift-ingress/router-default-7f6c8f5bc8-h8tgh"
Sep 09 20:41:37.227427 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:37.227346    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-gwc6n"]
Sep 09 20:41:38.179040 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:38.179000    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-gwc6n" event={"ID":"981c03df-6cd9-4e28-b060-2fd16739d669","Type":"ContainerStarted","Data":"966d236f630de8ea65dd9c442609918a9ef1a0ba488793c97bef414adddda95c"}
Sep 09 20:41:39.183800 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:39.183768    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-gwc6n" event={"ID":"981c03df-6cd9-4e28-b060-2fd16739d669","Type":"ContainerStarted","Data":"5d3bdfb7881b64dc22963eea0d2721f2fa0630f27eeec6049b64d25eb4732ddd"}
Sep 09 20:41:40.188880 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:40.188842    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-gwc6n" event={"ID":"981c03df-6cd9-4e28-b060-2fd16739d669","Type":"ContainerStarted","Data":"a4a4cf1fba46a51bfabc41adafac2c53c2d22c1bb663b37f4ce0ed75adfd4f9a"}
Sep 09 20:41:40.203707 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:40.203658    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-gwc6n" podStartSLOduration=18.365542858 podStartE2EDuration="20.203642884s" totalImagesPullingTime="1.838100026s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:41:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:41:40.203352228 +0000 UTC m=+61.830980101" watchObservedRunningTime="2026-09-09 20:41:40.203642884 +0000 UTC m=+61.831270757"
Sep 09 20:41:42.433732 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:42.433688    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/64dbb975-388d-48c4-bd16-7b33658225d0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2ptr7\" (UID: \"64dbb975-388d-48c4-bd16-7b33658225d0\") " pod="openshift-insights/insights-runtime-extractor-2ptr7"
Sep 09 20:41:42.434163 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:42.433760    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/64dbb975-388d-48c4-bd16-7b33658225d0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2ptr7\" (UID: \"64dbb975-388d-48c4-bd16-7b33658225d0\") " pod="openshift-insights/insights-runtime-extractor-2ptr7"
Sep 09 20:41:42.434382 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:42.434360    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/64dbb975-388d-48c4-bd16-7b33658225d0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2ptr7\" (UID: \"64dbb975-388d-48c4-bd16-7b33658225d0\") " pod="openshift-insights/insights-runtime-extractor-2ptr7"
Sep 09 20:41:42.435976 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:42.435946    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/64dbb975-388d-48c4-bd16-7b33658225d0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2ptr7\" (UID: \"64dbb975-388d-48c4-bd16-7b33658225d0\") " pod="openshift-insights/insights-runtime-extractor-2ptr7"
Sep 09 20:41:42.494869 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:42.494840    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-r8hx4\""
Sep 09 20:41:42.502968 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:42.502946    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2ptr7"
Sep 09 20:41:42.804762 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:42.804724    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2ptr7"]
Sep 09 20:41:42.808375 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:41:42.808349    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64dbb975_388d_48c4_bd16_7b33658225d0.slice/crio-bec243a10424e1f0abe7753877f6947d5fc3b7a060dfe81e2090afd08f2ed132 WatchSource:0}: Error finding container bec243a10424e1f0abe7753877f6947d5fc3b7a060dfe81e2090afd08f2ed132: Status 404 returned error can't find the container with id bec243a10424e1f0abe7753877f6947d5fc3b7a060dfe81e2090afd08f2ed132
Sep 09 20:41:43.139460 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:43.139347    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/48a52e3f-68fb-496f-b974-320a8539e7f7-metrics-tls\") pod \"dns-default-g2tsp\" (UID: \"48a52e3f-68fb-496f-b974-320a8539e7f7\") " pod="openshift-dns/dns-default-g2tsp"
Sep 09 20:41:43.141641 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:43.141622    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/48a52e3f-68fb-496f-b974-320a8539e7f7-metrics-tls\") pod \"dns-default-g2tsp\" (UID: \"48a52e3f-68fb-496f-b974-320a8539e7f7\") " pod="openshift-dns/dns-default-g2tsp"
Sep 09 20:41:43.199361 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:43.199327    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2ptr7" event={"ID":"64dbb975-388d-48c4-bd16-7b33658225d0","Type":"ContainerStarted","Data":"26a769e100a090cc6637374a0d77daeff9ee72925c2483b01e2082e9db36bbd1"}
Sep 09 20:41:43.199505 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:43.199383    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2ptr7" event={"ID":"64dbb975-388d-48c4-bd16-7b33658225d0","Type":"ContainerStarted","Data":"bec243a10424e1f0abe7753877f6947d5fc3b7a060dfe81e2090afd08f2ed132"}
Sep 09 20:41:43.391122 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:43.391027    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-vs5mb\""
Sep 09 20:41:43.399416 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:43.399385    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-g2tsp"
Sep 09 20:41:43.643452 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:43.643367    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1105e43b-4cb7-4b23-89ba-cf6c5370f185-metrics-certs\") pod \"network-metrics-daemon-2fzgg\" (UID: \"1105e43b-4cb7-4b23-89ba-cf6c5370f185\") " pod="openshift-multus/network-metrics-daemon-2fzgg"
Sep 09 20:41:43.643790 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:43.643462    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ad64d253-c155-4e3c-b872-cdebb0616a0e-original-pull-secret\") pod \"global-pull-secret-syncer-bjwzq\" (UID: \"ad64d253-c155-4e3c-b872-cdebb0616a0e\") " pod="kube-system/global-pull-secret-syncer-bjwzq"
Sep 09 20:41:43.646716 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:43.646697    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 09 20:41:43.646819 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:43.646802    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 09 20:41:43.656060 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:43.656036    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ad64d253-c155-4e3c-b872-cdebb0616a0e-original-pull-secret\") pod \"global-pull-secret-syncer-bjwzq\" (UID: \"ad64d253-c155-4e3c-b872-cdebb0616a0e\") " pod="kube-system/global-pull-secret-syncer-bjwzq"
Sep 09 20:41:43.656236 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:43.656180    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1105e43b-4cb7-4b23-89ba-cf6c5370f185-metrics-certs\") pod \"network-metrics-daemon-2fzgg\" (UID: \"1105e43b-4cb7-4b23-89ba-cf6c5370f185\") " pod="openshift-multus/network-metrics-daemon-2fzgg"
Sep 09 20:41:43.698193 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:43.698166    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-g2tsp"]
Sep 09 20:41:43.701455 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:41:43.701433    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48a52e3f_68fb_496f_b974_320a8539e7f7.slice/crio-992495008ac5fdfe301baed71558d87f6024a7ce0857ba056f33f19a16a00558 WatchSource:0}: Error finding container 992495008ac5fdfe301baed71558d87f6024a7ce0857ba056f33f19a16a00558: Status 404 returned error can't find the container with id 992495008ac5fdfe301baed71558d87f6024a7ce0857ba056f33f19a16a00558
Sep 09 20:41:43.722651 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:43.722616    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bjwzq"
Sep 09 20:41:43.744202 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:43.744174    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9gvv\" (UniqueName: \"kubernetes.io/projected/2f3c30c5-3a66-4a51-ba58-a125dce436ab-kube-api-access-x9gvv\") pod \"network-check-target-2c5hn\" (UID: \"2f3c30c5-3a66-4a51-ba58-a125dce436ab\") " pod="openshift-network-diagnostics/network-check-target-2c5hn"
Sep 09 20:41:43.751203 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:43.751180    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 09 20:41:43.761492 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:43.761473    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 09 20:41:43.774970 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:43.774906    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9gvv\" (UniqueName: \"kubernetes.io/projected/2f3c30c5-3a66-4a51-ba58-a125dce436ab-kube-api-access-x9gvv\") pod \"network-check-target-2c5hn\" (UID: \"2f3c30c5-3a66-4a51-ba58-a125dce436ab\") " pod="openshift-network-diagnostics/network-check-target-2c5hn"
Sep 09 20:41:43.928028 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:43.927948    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-sl49q\""
Sep 09 20:41:43.931943 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:43.931918    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-q6qnd\""
Sep 09 20:41:43.935837 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:43.935818    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2fzgg"
Sep 09 20:41:43.940112 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:43.940082    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2c5hn"
Sep 09 20:41:43.991987 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:43.991952    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-bjwzq"]
Sep 09 20:41:43.999650 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:41:43.999617    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad64d253_c155_4e3c_b872_cdebb0616a0e.slice/crio-f28fa12fba758c281f0a46c4f63aecb6d2b2cb5eb10a48d5ab154d08f69cddf4 WatchSource:0}: Error finding container f28fa12fba758c281f0a46c4f63aecb6d2b2cb5eb10a48d5ab154d08f69cddf4: Status 404 returned error can't find the container with id f28fa12fba758c281f0a46c4f63aecb6d2b2cb5eb10a48d5ab154d08f69cddf4
Sep 09 20:41:44.203347 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:44.203270    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-bjwzq" event={"ID":"ad64d253-c155-4e3c-b872-cdebb0616a0e","Type":"ContainerStarted","Data":"f28fa12fba758c281f0a46c4f63aecb6d2b2cb5eb10a48d5ab154d08f69cddf4"}
Sep 09 20:41:44.205039 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:44.205006    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2ptr7" event={"ID":"64dbb975-388d-48c4-bd16-7b33658225d0","Type":"ContainerStarted","Data":"90d985b34745072caf3f2cac62e292948489d91cf8e136464a8128eea22fd19d"}
Sep 09 20:41:44.206462 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:44.206435    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-g2tsp" event={"ID":"48a52e3f-68fb-496f-b974-320a8539e7f7","Type":"ContainerStarted","Data":"992495008ac5fdfe301baed71558d87f6024a7ce0857ba056f33f19a16a00558"}
Sep 09 20:41:44.206681 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:44.206665    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2fzgg"]
Sep 09 20:41:44.209271 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:41:44.209243    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1105e43b_4cb7_4b23_89ba_cf6c5370f185.slice/crio-d4b66ffce9afd9c8f16e977217c035afd4ffa228e03d5d22526c4764df083168 WatchSource:0}: Error finding container d4b66ffce9afd9c8f16e977217c035afd4ffa228e03d5d22526c4764df083168: Status 404 returned error can't find the container with id d4b66ffce9afd9c8f16e977217c035afd4ffa228e03d5d22526c4764df083168
Sep 09 20:41:44.210459 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:44.210439    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2c5hn"]
Sep 09 20:41:44.213048 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:41:44.213024    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f3c30c5_3a66_4a51_ba58_a125dce436ab.slice/crio-793d12a77f0890c478aa15e0a4d8071a1f27d82fb28bec790943fc4a6b25fa02 WatchSource:0}: Error finding container 793d12a77f0890c478aa15e0a4d8071a1f27d82fb28bec790943fc4a6b25fa02: Status 404 returned error can't find the container with id 793d12a77f0890c478aa15e0a4d8071a1f27d82fb28bec790943fc4a6b25fa02
Sep 09 20:41:44.816190 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:44.816162    2581 scope.go:117] "RemoveContainer" containerID="84ee389d25fbe5e33a540a69b56e2c9bd0a9ff1336e61f918cd1bf6414645c75"
Sep 09 20:41:45.214972 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:45.214751    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2c5hn" event={"ID":"2f3c30c5-3a66-4a51-ba58-a125dce436ab","Type":"ContainerStarted","Data":"793d12a77f0890c478aa15e0a4d8071a1f27d82fb28bec790943fc4a6b25fa02"}
Sep 09 20:41:45.216467 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:45.216404    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2fzgg" event={"ID":"1105e43b-4cb7-4b23-89ba-cf6c5370f185","Type":"ContainerStarted","Data":"d4b66ffce9afd9c8f16e977217c035afd4ffa228e03d5d22526c4764df083168"}
Sep 09 20:41:46.231916 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:46.231887    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-wsv4v_2b1f9a6e-7a79-4e89-ac3f-4658d6d91550/console-operator/1.log"
Sep 09 20:41:46.232544 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:46.232514    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-wsv4v" event={"ID":"2b1f9a6e-7a79-4e89-ac3f-4658d6d91550","Type":"ContainerStarted","Data":"9e74ea40a2044e8b35b8cc34ca472169d3bb68a723281be65090fec2ceaf6daa"}
Sep 09 20:41:46.232874 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:46.232818    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-wsv4v"
Sep 09 20:41:46.237055 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:46.237011    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-g2tsp" event={"ID":"48a52e3f-68fb-496f-b974-320a8539e7f7","Type":"ContainerStarted","Data":"8b3b7acafb1b15e8f79d427c881fd21a435d1f2303c5fafbfd6a0050f61f5287"}
Sep 09 20:41:46.237178 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:46.237081    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-g2tsp" event={"ID":"48a52e3f-68fb-496f-b974-320a8539e7f7","Type":"ContainerStarted","Data":"e900ae8c23a99862f96b05fed6c8c0eda331f353cfc7a422e428d1b665c8910d"}
Sep 09 20:41:46.237763 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:46.237747    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-g2tsp"
Sep 09 20:41:46.240317 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:46.240292    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2ptr7" event={"ID":"64dbb975-388d-48c4-bd16-7b33658225d0","Type":"ContainerStarted","Data":"84450dbf6b092c085a7246f8ba46fddcc37fb6b4206c27c437df6bc51d0406c5"}
Sep 09 20:41:46.265651 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:46.265598    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-g2tsp" podStartSLOduration=33.050170715 podStartE2EDuration="35.265579713s" totalImagesPullingTime="2.215408998s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:41:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:41:46.265482041 +0000 UTC m=+67.893109914" watchObservedRunningTime="2026-09-09 20:41:46.265579713 +0000 UTC m=+67.893207587"
Sep 09 20:41:46.266714 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:46.266643    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-85b454874b-wsv4v" podStartSLOduration=23.133165488 podStartE2EDuration="26.266627414s" totalImagesPullingTime="3.133461926s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:41:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:41:46.249585937 +0000 UTC m=+67.877213817" watchObservedRunningTime="2026-09-09 20:41:46.266627414 +0000 UTC m=+67.894255288"
Sep 09 20:41:46.283152 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:46.283084    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-2ptr7" podStartSLOduration=17.353860164 podStartE2EDuration="20.283063552s" totalImagesPullingTime="2.929203388s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:41:26 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:41:46.2816189 +0000 UTC m=+67.909246813" watchObservedRunningTime="2026-09-09 20:41:46.283063552 +0000 UTC m=+67.910691428"
Sep 09 20:41:47.233905 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:47.233852    2581 patch_prober.go:28] interesting pod/console-operator-85b454874b-wsv4v container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.133.0.10:8443/readyz\": context deadline exceeded" start-of-body=
Sep 09 20:41:47.234417 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:47.233940    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-85b454874b-wsv4v" podUID="2b1f9a6e-7a79-4e89-ac3f-4658d6d91550" containerName="console-operator" probeResult="failure" output="Get \"https://10.133.0.10:8443/readyz\": context deadline exceeded"
Sep 09 20:41:47.246271 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:47.246231    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2fzgg" event={"ID":"1105e43b-4cb7-4b23-89ba-cf6c5370f185","Type":"ContainerStarted","Data":"461b12c0d918f579b1f7f2f1ee8367b07240d07e4d40d07798e13f54b42a8f78"}
Sep 09 20:41:47.259860 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:47.259828    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-85b454874b-wsv4v"
Sep 09 20:41:50.258451 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:50.258412    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-bjwzq" event={"ID":"ad64d253-c155-4e3c-b872-cdebb0616a0e","Type":"ContainerStarted","Data":"36573d201ebb5a3a81a07db9a11d42ce3c0c00b3937d0ad2aa442b615fbd324c"}
Sep 09 20:41:50.259919 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:50.259886    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2c5hn" event={"ID":"2f3c30c5-3a66-4a51-ba58-a125dce436ab","Type":"ContainerStarted","Data":"1d5498b3a241e02f5a6ae74653ec95c8778433d45c7f3cd930208e2d7e81e2d4"}
Sep 09 20:41:50.260059 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:50.259936    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-2c5hn"
Sep 09 20:41:50.261736 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:50.261714    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2fzgg" event={"ID":"1105e43b-4cb7-4b23-89ba-cf6c5370f185","Type":"ContainerStarted","Data":"c24146a8bb5d3af093094987fdbf7e817e486a10b75fef695866af743136238e"}
Sep 09 20:41:50.273545 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:50.273496    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-bjwzq" podStartSLOduration=65.87657264 podStartE2EDuration="1m11.273480789s" totalImagesPullingTime="5.396908149s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:40:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:41:50.272318095 +0000 UTC m=+71.899945968" watchObservedRunningTime="2026-09-09 20:41:50.273480789 +0000 UTC m=+71.901108682"
Sep 09 20:41:50.288064 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:50.288019    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-2fzgg" podStartSLOduration=69.690838292 podStartE2EDuration="1m12.288004303s" totalImagesPullingTime="2.597166011s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:40:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:41:50.286989582 +0000 UTC m=+71.914617455" watchObservedRunningTime="2026-09-09 20:41:50.288004303 +0000 UTC m=+71.915632176"
Sep 09 20:41:50.302792 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:50.302744    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-2c5hn" podStartSLOduration=67.123734965 podStartE2EDuration="1m12.302728016s" totalImagesPullingTime="5.178993051s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:40:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:41:50.302393823 +0000 UTC m=+71.930021710" watchObservedRunningTime="2026-09-09 20:41:50.302728016 +0000 UTC m=+71.930355890"
Sep 09 20:41:51.414798 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:51.414763    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-69887ff495-l9r6j"]
Sep 09 20:41:51.417919 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:51.417897    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69887ff495-l9r6j"
Sep 09 20:41:51.420400 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:51.420380    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 09 20:41:51.420856 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:51.420840    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 09 20:41:51.421743 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:51.421720    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 09 20:41:51.421880 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:51.421808    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-tj77j\""
Sep 09 20:41:51.425597 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:51.425577    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 09 20:41:51.429392 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:51.429371    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-69887ff495-l9r6j"]
Sep 09 20:41:51.515357 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:51.515316    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/45d6b919-92b0-4092-8522-039bcb692260-registry-tls\") pod \"image-registry-69887ff495-l9r6j\" (UID: \"45d6b919-92b0-4092-8522-039bcb692260\") " pod="openshift-image-registry/image-registry-69887ff495-l9r6j"
Sep 09 20:41:51.515357 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:51.515357    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/45d6b919-92b0-4092-8522-039bcb692260-registry-certificates\") pod \"image-registry-69887ff495-l9r6j\" (UID: \"45d6b919-92b0-4092-8522-039bcb692260\") " pod="openshift-image-registry/image-registry-69887ff495-l9r6j"
Sep 09 20:41:51.515577 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:51.515380    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7k5l9\" (UniqueName: \"kubernetes.io/projected/45d6b919-92b0-4092-8522-039bcb692260-kube-api-access-7k5l9\") pod \"image-registry-69887ff495-l9r6j\" (UID: \"45d6b919-92b0-4092-8522-039bcb692260\") " pod="openshift-image-registry/image-registry-69887ff495-l9r6j"
Sep 09 20:41:51.515577 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:51.515424    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/45d6b919-92b0-4092-8522-039bcb692260-image-registry-private-configuration\") pod \"image-registry-69887ff495-l9r6j\" (UID: \"45d6b919-92b0-4092-8522-039bcb692260\") " pod="openshift-image-registry/image-registry-69887ff495-l9r6j"
Sep 09 20:41:51.515577 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:51.515447    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/45d6b919-92b0-4092-8522-039bcb692260-ca-trust-extracted\") pod \"image-registry-69887ff495-l9r6j\" (UID: \"45d6b919-92b0-4092-8522-039bcb692260\") " pod="openshift-image-registry/image-registry-69887ff495-l9r6j"
Sep 09 20:41:51.515577 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:51.515469    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/45d6b919-92b0-4092-8522-039bcb692260-installation-pull-secrets\") pod \"image-registry-69887ff495-l9r6j\" (UID: \"45d6b919-92b0-4092-8522-039bcb692260\") " pod="openshift-image-registry/image-registry-69887ff495-l9r6j"
Sep 09 20:41:51.515577 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:51.515507    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/45d6b919-92b0-4092-8522-039bcb692260-bound-sa-token\") pod \"image-registry-69887ff495-l9r6j\" (UID: \"45d6b919-92b0-4092-8522-039bcb692260\") " pod="openshift-image-registry/image-registry-69887ff495-l9r6j"
Sep 09 20:41:51.515726 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:51.515575    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/45d6b919-92b0-4092-8522-039bcb692260-trusted-ca\") pod \"image-registry-69887ff495-l9r6j\" (UID: \"45d6b919-92b0-4092-8522-039bcb692260\") " pod="openshift-image-registry/image-registry-69887ff495-l9r6j"
Sep 09 20:41:51.616399 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:51.616364    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/45d6b919-92b0-4092-8522-039bcb692260-registry-tls\") pod \"image-registry-69887ff495-l9r6j\" (UID: \"45d6b919-92b0-4092-8522-039bcb692260\") " pod="openshift-image-registry/image-registry-69887ff495-l9r6j"
Sep 09 20:41:51.616399 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:51.616403    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/45d6b919-92b0-4092-8522-039bcb692260-registry-certificates\") pod \"image-registry-69887ff495-l9r6j\" (UID: \"45d6b919-92b0-4092-8522-039bcb692260\") " pod="openshift-image-registry/image-registry-69887ff495-l9r6j"
Sep 09 20:41:51.616634 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:51.616421    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7k5l9\" (UniqueName: \"kubernetes.io/projected/45d6b919-92b0-4092-8522-039bcb692260-kube-api-access-7k5l9\") pod \"image-registry-69887ff495-l9r6j\" (UID: \"45d6b919-92b0-4092-8522-039bcb692260\") " pod="openshift-image-registry/image-registry-69887ff495-l9r6j"
Sep 09 20:41:51.616634 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:51.616446    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/45d6b919-92b0-4092-8522-039bcb692260-image-registry-private-configuration\") pod \"image-registry-69887ff495-l9r6j\" (UID: \"45d6b919-92b0-4092-8522-039bcb692260\") " pod="openshift-image-registry/image-registry-69887ff495-l9r6j"
Sep 09 20:41:51.616634 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:51.616475    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/45d6b919-92b0-4092-8522-039bcb692260-ca-trust-extracted\") pod \"image-registry-69887ff495-l9r6j\" (UID: \"45d6b919-92b0-4092-8522-039bcb692260\") " pod="openshift-image-registry/image-registry-69887ff495-l9r6j"
Sep 09 20:41:51.616634 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:51.616505    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/45d6b919-92b0-4092-8522-039bcb692260-installation-pull-secrets\") pod \"image-registry-69887ff495-l9r6j\" (UID: \"45d6b919-92b0-4092-8522-039bcb692260\") " pod="openshift-image-registry/image-registry-69887ff495-l9r6j"
Sep 09 20:41:51.616634 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:51.616531    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/45d6b919-92b0-4092-8522-039bcb692260-bound-sa-token\") pod \"image-registry-69887ff495-l9r6j\" (UID: \"45d6b919-92b0-4092-8522-039bcb692260\") " pod="openshift-image-registry/image-registry-69887ff495-l9r6j"
Sep 09 20:41:51.616894 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:51.616704    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/45d6b919-92b0-4092-8522-039bcb692260-trusted-ca\") pod \"image-registry-69887ff495-l9r6j\" (UID: \"45d6b919-92b0-4092-8522-039bcb692260\") " pod="openshift-image-registry/image-registry-69887ff495-l9r6j"
Sep 09 20:41:51.616992 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:51.616972    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/45d6b919-92b0-4092-8522-039bcb692260-ca-trust-extracted\") pod \"image-registry-69887ff495-l9r6j\" (UID: \"45d6b919-92b0-4092-8522-039bcb692260\") " pod="openshift-image-registry/image-registry-69887ff495-l9r6j"
Sep 09 20:41:51.617363 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:51.617333    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/45d6b919-92b0-4092-8522-039bcb692260-registry-certificates\") pod \"image-registry-69887ff495-l9r6j\" (UID: \"45d6b919-92b0-4092-8522-039bcb692260\") " pod="openshift-image-registry/image-registry-69887ff495-l9r6j"
Sep 09 20:41:51.617588 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:51.617568    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/45d6b919-92b0-4092-8522-039bcb692260-trusted-ca\") pod \"image-registry-69887ff495-l9r6j\" (UID: \"45d6b919-92b0-4092-8522-039bcb692260\") " pod="openshift-image-registry/image-registry-69887ff495-l9r6j"
Sep 09 20:41:51.618938 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:51.618918    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/45d6b919-92b0-4092-8522-039bcb692260-image-registry-private-configuration\") pod \"image-registry-69887ff495-l9r6j\" (UID: \"45d6b919-92b0-4092-8522-039bcb692260\") " pod="openshift-image-registry/image-registry-69887ff495-l9r6j"
Sep 09 20:41:51.619019 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:51.619002    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/45d6b919-92b0-4092-8522-039bcb692260-registry-tls\") pod \"image-registry-69887ff495-l9r6j\" (UID: \"45d6b919-92b0-4092-8522-039bcb692260\") " pod="openshift-image-registry/image-registry-69887ff495-l9r6j"
Sep 09 20:41:51.619137 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:51.619118    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/45d6b919-92b0-4092-8522-039bcb692260-installation-pull-secrets\") pod \"image-registry-69887ff495-l9r6j\" (UID: \"45d6b919-92b0-4092-8522-039bcb692260\") " pod="openshift-image-registry/image-registry-69887ff495-l9r6j"
Sep 09 20:41:51.627268 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:51.627242    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/45d6b919-92b0-4092-8522-039bcb692260-bound-sa-token\") pod \"image-registry-69887ff495-l9r6j\" (UID: \"45d6b919-92b0-4092-8522-039bcb692260\") " pod="openshift-image-registry/image-registry-69887ff495-l9r6j"
Sep 09 20:41:51.627577 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:51.627559    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7k5l9\" (UniqueName: \"kubernetes.io/projected/45d6b919-92b0-4092-8522-039bcb692260-kube-api-access-7k5l9\") pod \"image-registry-69887ff495-l9r6j\" (UID: \"45d6b919-92b0-4092-8522-039bcb692260\") " pod="openshift-image-registry/image-registry-69887ff495-l9r6j"
Sep 09 20:41:51.726667 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:51.726573    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69887ff495-l9r6j"
Sep 09 20:41:51.999860 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:51.999827    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-69887ff495-l9r6j"]
Sep 09 20:41:52.271348 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:52.271266    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-69887ff495-l9r6j" event={"ID":"45d6b919-92b0-4092-8522-039bcb692260","Type":"ContainerStarted","Data":"c901fc4c82995531f09212fa58b3489e89e466cfe37f1396d6b88324eee50a87"}
Sep 09 20:41:52.271348 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:52.271340    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-69887ff495-l9r6j" event={"ID":"45d6b919-92b0-4092-8522-039bcb692260","Type":"ContainerStarted","Data":"22ad65d23f40b1433902e50efd590575c21b07cef928f6417b176aa45f363c0d"}
Sep 09 20:41:52.271528 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:52.271374    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-69887ff495-l9r6j"
Sep 09 20:41:52.290327 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:52.290271    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-69887ff495-l9r6j" podStartSLOduration=1.290254574 podStartE2EDuration="1.290254574s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:41:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:41:52.288829749 +0000 UTC m=+73.916457623" watchObservedRunningTime="2026-09-09 20:41:52.290254574 +0000 UTC m=+73.917882446"
Sep 09 20:41:53.129075 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:53.129034    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5d449a3-ab1b-4899-842d-fcc9accb0838-service-ca-bundle\") pod \"router-default-7f6c8f5bc8-h8tgh\" (UID: \"d5d449a3-ab1b-4899-842d-fcc9accb0838\") " pod="openshift-ingress/router-default-7f6c8f5bc8-h8tgh"
Sep 09 20:41:53.129474 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:53.129118    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7c35fa5e-a4d9-48bb-9fcf-63cca4ed0fd2-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-jx7vj\" (UID: \"7c35fa5e-a4d9-48bb-9fcf-63cca4ed0fd2\") " pod="openshift-network-console/networking-console-plugin-79977f9844-jx7vj"
Sep 09 20:41:53.129774 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:53.129751    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5d449a3-ab1b-4899-842d-fcc9accb0838-service-ca-bundle\") pod \"router-default-7f6c8f5bc8-h8tgh\" (UID: \"d5d449a3-ab1b-4899-842d-fcc9accb0838\") " pod="openshift-ingress/router-default-7f6c8f5bc8-h8tgh"
Sep 09 20:41:53.131512 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:53.131493    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7c35fa5e-a4d9-48bb-9fcf-63cca4ed0fd2-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-jx7vj\" (UID: \"7c35fa5e-a4d9-48bb-9fcf-63cca4ed0fd2\") " pod="openshift-network-console/networking-console-plugin-79977f9844-jx7vj"
Sep 09 20:41:53.316045 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:53.316016    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-z5x86\""
Sep 09 20:41:53.324960 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:53.324937    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-jx7vj"
Sep 09 20:41:53.329602 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:53.329281    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-qblcs\""
Sep 09 20:41:53.338265 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:53.338243    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7f6c8f5bc8-h8tgh"
Sep 09 20:41:53.600019 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:53.599985    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-jx7vj"]
Sep 09 20:41:53.603380 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:41:53.603351    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c35fa5e_a4d9_48bb_9fcf_63cca4ed0fd2.slice/crio-5b95fe78007922e8c41286b4e37c50a0037c7a2cff8a6c8dca3ed6df4f91c1b5 WatchSource:0}: Error finding container 5b95fe78007922e8c41286b4e37c50a0037c7a2cff8a6c8dca3ed6df4f91c1b5: Status 404 returned error can't find the container with id 5b95fe78007922e8c41286b4e37c50a0037c7a2cff8a6c8dca3ed6df4f91c1b5
Sep 09 20:41:53.623393 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:53.623364    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7f6c8f5bc8-h8tgh"]
Sep 09 20:41:53.628085 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:41:53.628062    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5d449a3_ab1b_4899_842d_fcc9accb0838.slice/crio-8c986f8b3e5c9a72a129bf25f832aa016b230c591f81a054b83a71b7388ade1e WatchSource:0}: Error finding container 8c986f8b3e5c9a72a129bf25f832aa016b230c591f81a054b83a71b7388ade1e: Status 404 returned error can't find the container with id 8c986f8b3e5c9a72a129bf25f832aa016b230c591f81a054b83a71b7388ade1e
Sep 09 20:41:54.281126 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:54.281066    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7f6c8f5bc8-h8tgh" event={"ID":"d5d449a3-ab1b-4899-842d-fcc9accb0838","Type":"ContainerStarted","Data":"6a492b9291272596f1b163e7f1fe6f63bdd70e5b3555449ff4198665d3aa2b3e"}
Sep 09 20:41:54.281571 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:54.281148    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7f6c8f5bc8-h8tgh" event={"ID":"d5d449a3-ab1b-4899-842d-fcc9accb0838","Type":"ContainerStarted","Data":"8c986f8b3e5c9a72a129bf25f832aa016b230c591f81a054b83a71b7388ade1e"}
Sep 09 20:41:54.282922 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:54.282900    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-jx7vj" event={"ID":"7c35fa5e-a4d9-48bb-9fcf-63cca4ed0fd2","Type":"ContainerStarted","Data":"5b95fe78007922e8c41286b4e37c50a0037c7a2cff8a6c8dca3ed6df4f91c1b5"}
Sep 09 20:41:54.300491 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:54.300439    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-7f6c8f5bc8-h8tgh" podStartSLOduration=33.300421996 podStartE2EDuration="33.300421996s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:41:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:41:54.298913174 +0000 UTC m=+75.926541051" watchObservedRunningTime="2026-09-09 20:41:54.300421996 +0000 UTC m=+75.928049860"
Sep 09 20:41:54.338458 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:54.338419    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-7f6c8f5bc8-h8tgh"
Sep 09 20:41:54.340904 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:54.340880    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-7f6c8f5bc8-h8tgh"
Sep 09 20:41:55.288542 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:55.288500    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-jx7vj" event={"ID":"7c35fa5e-a4d9-48bb-9fcf-63cca4ed0fd2","Type":"ContainerStarted","Data":"6795acff9e2ee5c874445037a89cf8f9e341e6652b08878bdaa104eba3e628c0"}
Sep 09 20:41:55.289026 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:55.288784    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-7f6c8f5bc8-h8tgh"
Sep 09 20:41:55.289741 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:55.289722    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-7f6c8f5bc8-h8tgh"
Sep 09 20:41:55.302682 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:55.302633    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-79977f9844-jx7vj" podStartSLOduration=33.165278827 podStartE2EDuration="34.302616311s" totalImagesPullingTime="1.137337484s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:41:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:41:55.302346694 +0000 UTC m=+76.929974568" watchObservedRunningTime="2026-09-09 20:41:55.302616311 +0000 UTC m=+76.930244178"
Sep 09 20:41:57.252132 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:41:57.252086    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-g2tsp"
Sep 09 20:42:01.168642 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:01.168500    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-2njn8"]
Sep 09 20:42:01.171981 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:01.171959    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-2njn8"
Sep 09 20:42:01.174407 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:01.174386    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 09 20:42:01.174504 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:01.174402    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 09 20:42:01.175432 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:01.175412    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 09 20:42:01.175544 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:01.175466    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 09 20:42:01.175613 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:01.175552    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 09 20:42:01.175613 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:01.175556    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-m5zh5\""
Sep 09 20:42:01.181194 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:01.181171    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-2njn8"]
Sep 09 20:42:01.298833 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:01.298794    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b94a227a-a76f-4bf9-9a49-91088409cb26-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-2njn8\" (UID: \"b94a227a-a76f-4bf9-9a49-91088409cb26\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-2njn8"
Sep 09 20:42:01.298996 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:01.298850    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b94a227a-a76f-4bf9-9a49-91088409cb26-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-2njn8\" (UID: \"b94a227a-a76f-4bf9-9a49-91088409cb26\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-2njn8"
Sep 09 20:42:01.298996 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:01.298895    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-td4g5\" (UniqueName: \"kubernetes.io/projected/b94a227a-a76f-4bf9-9a49-91088409cb26-kube-api-access-td4g5\") pod \"prometheus-operator-6f4946db56-2njn8\" (UID: \"b94a227a-a76f-4bf9-9a49-91088409cb26\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-2njn8"
Sep 09 20:42:01.298996 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:01.298922    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b94a227a-a76f-4bf9-9a49-91088409cb26-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-2njn8\" (UID: \"b94a227a-a76f-4bf9-9a49-91088409cb26\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-2njn8"
Sep 09 20:42:01.399252 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:01.399220    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b94a227a-a76f-4bf9-9a49-91088409cb26-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-2njn8\" (UID: \"b94a227a-a76f-4bf9-9a49-91088409cb26\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-2njn8"
Sep 09 20:42:01.399426 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:01.399256    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-td4g5\" (UniqueName: \"kubernetes.io/projected/b94a227a-a76f-4bf9-9a49-91088409cb26-kube-api-access-td4g5\") pod \"prometheus-operator-6f4946db56-2njn8\" (UID: \"b94a227a-a76f-4bf9-9a49-91088409cb26\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-2njn8"
Sep 09 20:42:01.399426 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:01.399282    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b94a227a-a76f-4bf9-9a49-91088409cb26-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-2njn8\" (UID: \"b94a227a-a76f-4bf9-9a49-91088409cb26\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-2njn8"
Sep 09 20:42:01.399426 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:01.399310    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b94a227a-a76f-4bf9-9a49-91088409cb26-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-2njn8\" (UID: \"b94a227a-a76f-4bf9-9a49-91088409cb26\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-2njn8"
Sep 09 20:42:01.399426 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:42:01.399392    2581 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Sep 09 20:42:01.399636 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:42:01.399447    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b94a227a-a76f-4bf9-9a49-91088409cb26-prometheus-operator-tls podName:b94a227a-a76f-4bf9-9a49-91088409cb26 nodeName:}" failed. No retries permitted until 2026-09-09 20:42:01.89943177 +0000 UTC m=+83.527059641 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/b94a227a-a76f-4bf9-9a49-91088409cb26-prometheus-operator-tls") pod "prometheus-operator-6f4946db56-2njn8" (UID: "b94a227a-a76f-4bf9-9a49-91088409cb26") : secret "prometheus-operator-tls" not found
Sep 09 20:42:01.399928 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:01.399907    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b94a227a-a76f-4bf9-9a49-91088409cb26-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-2njn8\" (UID: \"b94a227a-a76f-4bf9-9a49-91088409cb26\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-2njn8"
Sep 09 20:42:01.401488 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:01.401466    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b94a227a-a76f-4bf9-9a49-91088409cb26-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-2njn8\" (UID: \"b94a227a-a76f-4bf9-9a49-91088409cb26\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-2njn8"
Sep 09 20:42:01.407995 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:01.407973    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-td4g5\" (UniqueName: \"kubernetes.io/projected/b94a227a-a76f-4bf9-9a49-91088409cb26-kube-api-access-td4g5\") pod \"prometheus-operator-6f4946db56-2njn8\" (UID: \"b94a227a-a76f-4bf9-9a49-91088409cb26\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-2njn8"
Sep 09 20:42:01.901979 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:01.901940    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b94a227a-a76f-4bf9-9a49-91088409cb26-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-2njn8\" (UID: \"b94a227a-a76f-4bf9-9a49-91088409cb26\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-2njn8"
Sep 09 20:42:01.904288 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:01.904259    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b94a227a-a76f-4bf9-9a49-91088409cb26-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-2njn8\" (UID: \"b94a227a-a76f-4bf9-9a49-91088409cb26\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-2njn8"
Sep 09 20:42:02.081645 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:02.081608    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-2njn8"
Sep 09 20:42:02.348969 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:02.348869    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-2njn8"]
Sep 09 20:42:02.351348 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:42:02.351320    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb94a227a_a76f_4bf9_9a49_91088409cb26.slice/crio-720e0a79f5eb10d97909cc39f300fbcf06078da11083f4e5486a220b19a9c4b1 WatchSource:0}: Error finding container 720e0a79f5eb10d97909cc39f300fbcf06078da11083f4e5486a220b19a9c4b1: Status 404 returned error can't find the container with id 720e0a79f5eb10d97909cc39f300fbcf06078da11083f4e5486a220b19a9c4b1
Sep 09 20:42:03.316111 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:03.316063    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-2njn8" event={"ID":"b94a227a-a76f-4bf9-9a49-91088409cb26","Type":"ContainerStarted","Data":"720e0a79f5eb10d97909cc39f300fbcf06078da11083f4e5486a220b19a9c4b1"}
Sep 09 20:42:04.324270 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:04.324173    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-2njn8" event={"ID":"b94a227a-a76f-4bf9-9a49-91088409cb26","Type":"ContainerStarted","Data":"b401237a344765cc726aa48de3efcb92bdf045cd5cb3108c8a694216d1ea3fc9"}
Sep 09 20:42:04.324270 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:04.324234    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-2njn8" event={"ID":"b94a227a-a76f-4bf9-9a49-91088409cb26","Type":"ContainerStarted","Data":"627e3e6db0479ff6c8dc696094e4cc8bb5d62ab3ad28b5b6cde27656378d0405"}
Sep 09 20:42:04.341123 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:04.341059    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-6f4946db56-2njn8" podStartSLOduration=1.569027593 podStartE2EDuration="3.341042832s" totalImagesPullingTime="1.772015239s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:42:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:42:04.339116934 +0000 UTC m=+85.966744801" watchObservedRunningTime="2026-09-09 20:42:04.341042832 +0000 UTC m=+85.968670704"
Sep 09 20:42:06.555586 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:06.555281    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-684mg"]
Sep 09 20:42:06.559897 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:06.559874    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-jxl74"]
Sep 09 20:42:06.560035 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:06.560013    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-684mg"
Sep 09 20:42:06.562620 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:06.562597    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-dvqdx\""
Sep 09 20:42:06.562834 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:06.562819    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 09 20:42:06.563114 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:06.563088    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 09 20:42:06.563213 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:06.563137    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 09 20:42:06.563670 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:06.563655    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-jxl74"
Sep 09 20:42:06.565936 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:06.565916    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 09 20:42:06.566039 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:06.566023    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-b7wjx\""
Sep 09 20:42:06.566247 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:06.566233    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 09 20:42:06.585117 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:06.585085    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-jxl74"]
Sep 09 20:42:06.639848 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:06.639806    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4978df08-2309-45d0-85f2-614c151114ea-node-exporter-textfile\") pod \"node-exporter-684mg\" (UID: \"4978df08-2309-45d0-85f2-614c151114ea\") " pod="openshift-monitoring/node-exporter-684mg"
Sep 09 20:42:06.639848 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:06.639853    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4978df08-2309-45d0-85f2-614c151114ea-node-exporter-accelerators-collector-config\") pod \"node-exporter-684mg\" (UID: \"4978df08-2309-45d0-85f2-614c151114ea\") " pod="openshift-monitoring/node-exporter-684mg"
Sep 09 20:42:06.640162 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:06.639916    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4978df08-2309-45d0-85f2-614c151114ea-metrics-client-ca\") pod \"node-exporter-684mg\" (UID: \"4978df08-2309-45d0-85f2-614c151114ea\") " pod="openshift-monitoring/node-exporter-684mg"
Sep 09 20:42:06.640162 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:06.639957    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/59748361-30b6-44c6-abbb-3a488d57f060-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-jxl74\" (UID: \"59748361-30b6-44c6-abbb-3a488d57f060\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-jxl74"
Sep 09 20:42:06.640162 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:06.639993    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/59748361-30b6-44c6-abbb-3a488d57f060-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-jxl74\" (UID: \"59748361-30b6-44c6-abbb-3a488d57f060\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-jxl74"
Sep 09 20:42:06.640162 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:06.640019    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4978df08-2309-45d0-85f2-614c151114ea-sys\") pod \"node-exporter-684mg\" (UID: \"4978df08-2309-45d0-85f2-614c151114ea\") " pod="openshift-monitoring/node-exporter-684mg"
Sep 09 20:42:06.640162 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:06.640042    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4978df08-2309-45d0-85f2-614c151114ea-node-exporter-tls\") pod \"node-exporter-684mg\" (UID: \"4978df08-2309-45d0-85f2-614c151114ea\") " pod="openshift-monitoring/node-exporter-684mg"
Sep 09 20:42:06.640162 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:06.640058    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4978df08-2309-45d0-85f2-614c151114ea-node-exporter-wtmp\") pod \"node-exporter-684mg\" (UID: \"4978df08-2309-45d0-85f2-614c151114ea\") " pod="openshift-monitoring/node-exporter-684mg"
Sep 09 20:42:06.640162 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:06.640123    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/59748361-30b6-44c6-abbb-3a488d57f060-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-jxl74\" (UID: \"59748361-30b6-44c6-abbb-3a488d57f060\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-jxl74"
Sep 09 20:42:06.640162 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:06.640147    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96bjs\" (UniqueName: \"kubernetes.io/projected/59748361-30b6-44c6-abbb-3a488d57f060-kube-api-access-96bjs\") pod \"openshift-state-metrics-657fd6c54-jxl74\" (UID: \"59748361-30b6-44c6-abbb-3a488d57f060\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-jxl74"
Sep 09 20:42:06.640594 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:06.640173    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4978df08-2309-45d0-85f2-614c151114ea-root\") pod \"node-exporter-684mg\" (UID: \"4978df08-2309-45d0-85f2-614c151114ea\") " pod="openshift-monitoring/node-exporter-684mg"
Sep 09 20:42:06.640594 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:06.640196    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4978df08-2309-45d0-85f2-614c151114ea-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-684mg\" (UID: \"4978df08-2309-45d0-85f2-614c151114ea\") " pod="openshift-monitoring/node-exporter-684mg"
Sep 09 20:42:06.640594 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:06.640262    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpwlj\" (UniqueName: \"kubernetes.io/projected/4978df08-2309-45d0-85f2-614c151114ea-kube-api-access-fpwlj\") pod \"node-exporter-684mg\" (UID: \"4978df08-2309-45d0-85f2-614c151114ea\") " pod="openshift-monitoring/node-exporter-684mg"
Sep 09 20:42:06.741374 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:06.741344    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/59748361-30b6-44c6-abbb-3a488d57f060-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-jxl74\" (UID: \"59748361-30b6-44c6-abbb-3a488d57f060\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-jxl74"
Sep 09 20:42:06.741374 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:06.741377    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4978df08-2309-45d0-85f2-614c151114ea-sys\") pod \"node-exporter-684mg\" (UID: \"4978df08-2309-45d0-85f2-614c151114ea\") " pod="openshift-monitoring/node-exporter-684mg"
Sep 09 20:42:06.741532 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:06.741396    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4978df08-2309-45d0-85f2-614c151114ea-node-exporter-tls\") pod \"node-exporter-684mg\" (UID: \"4978df08-2309-45d0-85f2-614c151114ea\") " pod="openshift-monitoring/node-exporter-684mg"
Sep 09 20:42:06.741532 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:06.741465    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4978df08-2309-45d0-85f2-614c151114ea-sys\") pod \"node-exporter-684mg\" (UID: \"4978df08-2309-45d0-85f2-614c151114ea\") " pod="openshift-monitoring/node-exporter-684mg"
Sep 09 20:42:06.741532 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:42:06.741485    2581 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 09 20:42:06.741644 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:42:06.741541    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4978df08-2309-45d0-85f2-614c151114ea-node-exporter-tls podName:4978df08-2309-45d0-85f2-614c151114ea nodeName:}" failed. No retries permitted until 2026-09-09 20:42:07.241525324 +0000 UTC m=+88.869153178 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/4978df08-2309-45d0-85f2-614c151114ea-node-exporter-tls") pod "node-exporter-684mg" (UID: "4978df08-2309-45d0-85f2-614c151114ea") : secret "node-exporter-tls" not found
Sep 09 20:42:06.741644 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:06.741563    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4978df08-2309-45d0-85f2-614c151114ea-node-exporter-wtmp\") pod \"node-exporter-684mg\" (UID: \"4978df08-2309-45d0-85f2-614c151114ea\") " pod="openshift-monitoring/node-exporter-684mg"
Sep 09 20:42:06.741644 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:06.741615    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/59748361-30b6-44c6-abbb-3a488d57f060-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-jxl74\" (UID: \"59748361-30b6-44c6-abbb-3a488d57f060\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-jxl74"
Sep 09 20:42:06.741752 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:06.741642    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96bjs\" (UniqueName: \"kubernetes.io/projected/59748361-30b6-44c6-abbb-3a488d57f060-kube-api-access-96bjs\") pod \"openshift-state-metrics-657fd6c54-jxl74\" (UID: \"59748361-30b6-44c6-abbb-3a488d57f060\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-jxl74"
Sep 09 20:42:06.741752 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:06.741683    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4978df08-2309-45d0-85f2-614c151114ea-node-exporter-wtmp\") pod \"node-exporter-684mg\" (UID: \"4978df08-2309-45d0-85f2-614c151114ea\") " pod="openshift-monitoring/node-exporter-684mg"
Sep 09 20:42:06.741752 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:06.741725    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4978df08-2309-45d0-85f2-614c151114ea-root\") pod \"node-exporter-684mg\" (UID: \"4978df08-2309-45d0-85f2-614c151114ea\") " pod="openshift-monitoring/node-exporter-684mg"
Sep 09 20:42:06.741874 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:06.741753    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4978df08-2309-45d0-85f2-614c151114ea-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-684mg\" (UID: \"4978df08-2309-45d0-85f2-614c151114ea\") " pod="openshift-monitoring/node-exporter-684mg"
Sep 09 20:42:06.741874 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:06.741785    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpwlj\" (UniqueName: \"kubernetes.io/projected/4978df08-2309-45d0-85f2-614c151114ea-kube-api-access-fpwlj\") pod \"node-exporter-684mg\" (UID: \"4978df08-2309-45d0-85f2-614c151114ea\") " pod="openshift-monitoring/node-exporter-684mg"
Sep 09 20:42:06.741874 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:06.741813    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4978df08-2309-45d0-85f2-614c151114ea-node-exporter-textfile\") pod \"node-exporter-684mg\" (UID: \"4978df08-2309-45d0-85f2-614c151114ea\") " pod="openshift-monitoring/node-exporter-684mg"
Sep 09 20:42:06.741874 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:06.741826    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4978df08-2309-45d0-85f2-614c151114ea-root\") pod \"node-exporter-684mg\" (UID: \"4978df08-2309-45d0-85f2-614c151114ea\") " pod="openshift-monitoring/node-exporter-684mg"
Sep 09 20:42:06.741874 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:06.741846    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4978df08-2309-45d0-85f2-614c151114ea-node-exporter-accelerators-collector-config\") pod \"node-exporter-684mg\" (UID: \"4978df08-2309-45d0-85f2-614c151114ea\") " pod="openshift-monitoring/node-exporter-684mg"
Sep 09 20:42:06.742145 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:06.741897    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4978df08-2309-45d0-85f2-614c151114ea-metrics-client-ca\") pod \"node-exporter-684mg\" (UID: \"4978df08-2309-45d0-85f2-614c151114ea\") " pod="openshift-monitoring/node-exporter-684mg"
Sep 09 20:42:06.742145 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:06.741928    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/59748361-30b6-44c6-abbb-3a488d57f060-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-jxl74\" (UID: \"59748361-30b6-44c6-abbb-3a488d57f060\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-jxl74"
Sep 09 20:42:06.742145 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:42:06.742049    2581 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Sep 09 20:42:06.742145 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:42:06.742140    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59748361-30b6-44c6-abbb-3a488d57f060-openshift-state-metrics-tls podName:59748361-30b6-44c6-abbb-3a488d57f060 nodeName:}" failed. No retries permitted until 2026-09-09 20:42:07.242093652 +0000 UTC m=+88.869721516 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/59748361-30b6-44c6-abbb-3a488d57f060-openshift-state-metrics-tls") pod "openshift-state-metrics-657fd6c54-jxl74" (UID: "59748361-30b6-44c6-abbb-3a488d57f060") : secret "openshift-state-metrics-tls" not found
Sep 09 20:42:06.742324 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:06.742215    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4978df08-2309-45d0-85f2-614c151114ea-node-exporter-textfile\") pod \"node-exporter-684mg\" (UID: \"4978df08-2309-45d0-85f2-614c151114ea\") " pod="openshift-monitoring/node-exporter-684mg"
Sep 09 20:42:06.742434 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:06.742415    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/59748361-30b6-44c6-abbb-3a488d57f060-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-jxl74\" (UID: \"59748361-30b6-44c6-abbb-3a488d57f060\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-jxl74"
Sep 09 20:42:06.742488 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:06.742466    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4978df08-2309-45d0-85f2-614c151114ea-node-exporter-accelerators-collector-config\") pod \"node-exporter-684mg\" (UID: \"4978df08-2309-45d0-85f2-614c151114ea\") " pod="openshift-monitoring/node-exporter-684mg"
Sep 09 20:42:06.742644 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:06.742621    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4978df08-2309-45d0-85f2-614c151114ea-metrics-client-ca\") pod \"node-exporter-684mg\" (UID: \"4978df08-2309-45d0-85f2-614c151114ea\") " pod="openshift-monitoring/node-exporter-684mg"
Sep 09 20:42:06.744022 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:06.744000    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4978df08-2309-45d0-85f2-614c151114ea-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-684mg\" (UID: \"4978df08-2309-45d0-85f2-614c151114ea\") " pod="openshift-monitoring/node-exporter-684mg"
Sep 09 20:42:06.744075 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:06.744053    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/59748361-30b6-44c6-abbb-3a488d57f060-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-jxl74\" (UID: \"59748361-30b6-44c6-abbb-3a488d57f060\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-jxl74"
Sep 09 20:42:06.752377 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:06.752350    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpwlj\" (UniqueName: \"kubernetes.io/projected/4978df08-2309-45d0-85f2-614c151114ea-kube-api-access-fpwlj\") pod \"node-exporter-684mg\" (UID: \"4978df08-2309-45d0-85f2-614c151114ea\") " pod="openshift-monitoring/node-exporter-684mg"
Sep 09 20:42:06.752491 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:06.752464    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-96bjs\" (UniqueName: \"kubernetes.io/projected/59748361-30b6-44c6-abbb-3a488d57f060-kube-api-access-96bjs\") pod \"openshift-state-metrics-657fd6c54-jxl74\" (UID: \"59748361-30b6-44c6-abbb-3a488d57f060\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-jxl74"
Sep 09 20:42:07.246446 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:07.246412    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/59748361-30b6-44c6-abbb-3a488d57f060-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-jxl74\" (UID: \"59748361-30b6-44c6-abbb-3a488d57f060\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-jxl74"
Sep 09 20:42:07.246614 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:07.246466    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4978df08-2309-45d0-85f2-614c151114ea-node-exporter-tls\") pod \"node-exporter-684mg\" (UID: \"4978df08-2309-45d0-85f2-614c151114ea\") " pod="openshift-monitoring/node-exporter-684mg"
Sep 09 20:42:07.248928 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:07.248900    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/59748361-30b6-44c6-abbb-3a488d57f060-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-jxl74\" (UID: \"59748361-30b6-44c6-abbb-3a488d57f060\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-jxl74"
Sep 09 20:42:07.249077 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:07.249012    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4978df08-2309-45d0-85f2-614c151114ea-node-exporter-tls\") pod \"node-exporter-684mg\" (UID: \"4978df08-2309-45d0-85f2-614c151114ea\") " pod="openshift-monitoring/node-exporter-684mg"
Sep 09 20:42:07.470289 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:07.470254    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-684mg"
Sep 09 20:42:07.475342 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:07.475314    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-jxl74"
Sep 09 20:42:07.491248 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:42:07.489218    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4978df08_2309_45d0_85f2_614c151114ea.slice/crio-2a3eb4c8df127a7bddc25a4af569435dd62dc04137fb8f5d4d0e089f0b14066b WatchSource:0}: Error finding container 2a3eb4c8df127a7bddc25a4af569435dd62dc04137fb8f5d4d0e089f0b14066b: Status 404 returned error can't find the container with id 2a3eb4c8df127a7bddc25a4af569435dd62dc04137fb8f5d4d0e089f0b14066b
Sep 09 20:42:07.758664 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:07.758620    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-jxl74"]
Sep 09 20:42:07.762452 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:42:07.762429    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59748361_30b6_44c6_abbb_3a488d57f060.slice/crio-768026ddee48dc43745fe13b40d647fc7a0567c0340746af1a392ca624d85543 WatchSource:0}: Error finding container 768026ddee48dc43745fe13b40d647fc7a0567c0340746af1a392ca624d85543: Status 404 returned error can't find the container with id 768026ddee48dc43745fe13b40d647fc7a0567c0340746af1a392ca624d85543
Sep 09 20:42:08.344662 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:08.344627    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-684mg" event={"ID":"4978df08-2309-45d0-85f2-614c151114ea","Type":"ContainerStarted","Data":"2a3eb4c8df127a7bddc25a4af569435dd62dc04137fb8f5d4d0e089f0b14066b"}
Sep 09 20:42:08.346742 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:08.346713    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-jxl74" event={"ID":"59748361-30b6-44c6-abbb-3a488d57f060","Type":"ContainerStarted","Data":"483085fd8635a9ffada1c70e351796e10e23e9421d3e9ca1f91d8e9489324e21"}
Sep 09 20:42:08.346888 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:08.346764    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-jxl74" event={"ID":"59748361-30b6-44c6-abbb-3a488d57f060","Type":"ContainerStarted","Data":"64d6bcfed77d92ff54f0c3acc120cd4a3767edc0d35fa1c8c1212201ef6a96eb"}
Sep 09 20:42:08.346888 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:08.346783    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-jxl74" event={"ID":"59748361-30b6-44c6-abbb-3a488d57f060","Type":"ContainerStarted","Data":"768026ddee48dc43745fe13b40d647fc7a0567c0340746af1a392ca624d85543"}
Sep 09 20:42:09.352228 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:09.352133    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-jxl74" event={"ID":"59748361-30b6-44c6-abbb-3a488d57f060","Type":"ContainerStarted","Data":"b073935f86c48f9fdb9cdaef868e2cf050f594a06f8fa3607a188707f465349d"}
Sep 09 20:42:09.353864 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:09.353831    2581 generic.go:356] "Generic (PLEG): container finished" podID="4978df08-2309-45d0-85f2-614c151114ea" containerID="838cdf9b8fb90396133c65604df88e69905cfae03bb209925d0f143caaeb6735" exitCode=0
Sep 09 20:42:09.353983 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:09.353881    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-684mg" event={"ID":"4978df08-2309-45d0-85f2-614c151114ea","Type":"ContainerDied","Data":"838cdf9b8fb90396133c65604df88e69905cfae03bb209925d0f143caaeb6735"}
Sep 09 20:42:09.370850 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:09.370806    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-jxl74" podStartSLOduration=2.273105762 podStartE2EDuration="3.370793005s" totalImagesPullingTime="1.097687243s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:42:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:42:09.370252813 +0000 UTC m=+90.997880688" watchObservedRunningTime="2026-09-09 20:42:09.370793005 +0000 UTC m=+90.998420890"
Sep 09 20:42:09.523905 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:09.523872    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-75b99b9678-9pr69"]
Sep 09 20:42:09.527339 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:09.527317    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-75b99b9678-9pr69"
Sep 09 20:42:09.529843 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:09.529815    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 09 20:42:09.529969 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:09.529820    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 09 20:42:09.529969 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:09.529891    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 09 20:42:09.530318 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:09.530301    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-g8wpn\""
Sep 09 20:42:09.530418 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:09.530347    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 09 20:42:09.530418 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:09.530366    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-avra0gpedn1d4\""
Sep 09 20:42:09.530519 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:09.530439    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 09 20:42:09.540954 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:09.540936    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-75b99b9678-9pr69"]
Sep 09 20:42:09.668608 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:09.668514    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/e34537f9-daf0-43cb-a80b-6ad6f3ace41b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-75b99b9678-9pr69\" (UID: \"e34537f9-daf0-43cb-a80b-6ad6f3ace41b\") " pod="openshift-monitoring/thanos-querier-75b99b9678-9pr69"
Sep 09 20:42:09.668608 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:09.668559    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/e34537f9-daf0-43cb-a80b-6ad6f3ace41b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-75b99b9678-9pr69\" (UID: \"e34537f9-daf0-43cb-a80b-6ad6f3ace41b\") " pod="openshift-monitoring/thanos-querier-75b99b9678-9pr69"
Sep 09 20:42:09.668790 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:09.668640    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e34537f9-daf0-43cb-a80b-6ad6f3ace41b-secret-grpc-tls\") pod \"thanos-querier-75b99b9678-9pr69\" (UID: \"e34537f9-daf0-43cb-a80b-6ad6f3ace41b\") " pod="openshift-monitoring/thanos-querier-75b99b9678-9pr69"
Sep 09 20:42:09.668790 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:09.668689    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/e34537f9-daf0-43cb-a80b-6ad6f3ace41b-secret-thanos-querier-tls\") pod \"thanos-querier-75b99b9678-9pr69\" (UID: \"e34537f9-daf0-43cb-a80b-6ad6f3ace41b\") " pod="openshift-monitoring/thanos-querier-75b99b9678-9pr69"
Sep 09 20:42:09.668790 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:09.668711    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e34537f9-daf0-43cb-a80b-6ad6f3ace41b-metrics-client-ca\") pod \"thanos-querier-75b99b9678-9pr69\" (UID: \"e34537f9-daf0-43cb-a80b-6ad6f3ace41b\") " pod="openshift-monitoring/thanos-querier-75b99b9678-9pr69"
Sep 09 20:42:09.668790 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:09.668727    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rk8j\" (UniqueName: \"kubernetes.io/projected/e34537f9-daf0-43cb-a80b-6ad6f3ace41b-kube-api-access-8rk8j\") pod \"thanos-querier-75b99b9678-9pr69\" (UID: \"e34537f9-daf0-43cb-a80b-6ad6f3ace41b\") " pod="openshift-monitoring/thanos-querier-75b99b9678-9pr69"
Sep 09 20:42:09.668790 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:09.668759    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e34537f9-daf0-43cb-a80b-6ad6f3ace41b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-75b99b9678-9pr69\" (UID: \"e34537f9-daf0-43cb-a80b-6ad6f3ace41b\") " pod="openshift-monitoring/thanos-querier-75b99b9678-9pr69"
Sep 09 20:42:09.668790 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:09.668778    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e34537f9-daf0-43cb-a80b-6ad6f3ace41b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-75b99b9678-9pr69\" (UID: \"e34537f9-daf0-43cb-a80b-6ad6f3ace41b\") " pod="openshift-monitoring/thanos-querier-75b99b9678-9pr69"
Sep 09 20:42:09.770010 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:09.769969    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e34537f9-daf0-43cb-a80b-6ad6f3ace41b-secret-grpc-tls\") pod \"thanos-querier-75b99b9678-9pr69\" (UID: \"e34537f9-daf0-43cb-a80b-6ad6f3ace41b\") " pod="openshift-monitoring/thanos-querier-75b99b9678-9pr69"
Sep 09 20:42:09.770231 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:09.770030    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/e34537f9-daf0-43cb-a80b-6ad6f3ace41b-secret-thanos-querier-tls\") pod \"thanos-querier-75b99b9678-9pr69\" (UID: \"e34537f9-daf0-43cb-a80b-6ad6f3ace41b\") " pod="openshift-monitoring/thanos-querier-75b99b9678-9pr69"
Sep 09 20:42:09.770231 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:09.770055    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e34537f9-daf0-43cb-a80b-6ad6f3ace41b-metrics-client-ca\") pod \"thanos-querier-75b99b9678-9pr69\" (UID: \"e34537f9-daf0-43cb-a80b-6ad6f3ace41b\") " pod="openshift-monitoring/thanos-querier-75b99b9678-9pr69"
Sep 09 20:42:09.770231 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:09.770082    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rk8j\" (UniqueName: \"kubernetes.io/projected/e34537f9-daf0-43cb-a80b-6ad6f3ace41b-kube-api-access-8rk8j\") pod \"thanos-querier-75b99b9678-9pr69\" (UID: \"e34537f9-daf0-43cb-a80b-6ad6f3ace41b\") " pod="openshift-monitoring/thanos-querier-75b99b9678-9pr69"
Sep 09 20:42:09.770231 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:09.770215    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e34537f9-daf0-43cb-a80b-6ad6f3ace41b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-75b99b9678-9pr69\" (UID: \"e34537f9-daf0-43cb-a80b-6ad6f3ace41b\") " pod="openshift-monitoring/thanos-querier-75b99b9678-9pr69"
Sep 09 20:42:09.770481 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:09.770456    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e34537f9-daf0-43cb-a80b-6ad6f3ace41b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-75b99b9678-9pr69\" (UID: \"e34537f9-daf0-43cb-a80b-6ad6f3ace41b\") " pod="openshift-monitoring/thanos-querier-75b99b9678-9pr69"
Sep 09 20:42:09.770591 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:09.770572    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/e34537f9-daf0-43cb-a80b-6ad6f3ace41b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-75b99b9678-9pr69\" (UID: \"e34537f9-daf0-43cb-a80b-6ad6f3ace41b\") " pod="openshift-monitoring/thanos-querier-75b99b9678-9pr69"
Sep 09 20:42:09.770662 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:09.770613    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/e34537f9-daf0-43cb-a80b-6ad6f3ace41b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-75b99b9678-9pr69\" (UID: \"e34537f9-daf0-43cb-a80b-6ad6f3ace41b\") " pod="openshift-monitoring/thanos-querier-75b99b9678-9pr69"
Sep 09 20:42:09.770916 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:09.770891    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e34537f9-daf0-43cb-a80b-6ad6f3ace41b-metrics-client-ca\") pod \"thanos-querier-75b99b9678-9pr69\" (UID: \"e34537f9-daf0-43cb-a80b-6ad6f3ace41b\") " pod="openshift-monitoring/thanos-querier-75b99b9678-9pr69"
Sep 09 20:42:09.772995 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:09.772954    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e34537f9-daf0-43cb-a80b-6ad6f3ace41b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-75b99b9678-9pr69\" (UID: \"e34537f9-daf0-43cb-a80b-6ad6f3ace41b\") " pod="openshift-monitoring/thanos-querier-75b99b9678-9pr69"
Sep 09 20:42:09.773225 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:09.773184    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e34537f9-daf0-43cb-a80b-6ad6f3ace41b-secret-grpc-tls\") pod \"thanos-querier-75b99b9678-9pr69\" (UID: \"e34537f9-daf0-43cb-a80b-6ad6f3ace41b\") " pod="openshift-monitoring/thanos-querier-75b99b9678-9pr69"
Sep 09 20:42:09.773336 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:09.773315    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/e34537f9-daf0-43cb-a80b-6ad6f3ace41b-secret-thanos-querier-tls\") pod \"thanos-querier-75b99b9678-9pr69\" (UID: \"e34537f9-daf0-43cb-a80b-6ad6f3ace41b\") " pod="openshift-monitoring/thanos-querier-75b99b9678-9pr69"
Sep 09 20:42:09.773443 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:09.773361    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e34537f9-daf0-43cb-a80b-6ad6f3ace41b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-75b99b9678-9pr69\" (UID: \"e34537f9-daf0-43cb-a80b-6ad6f3ace41b\") " pod="openshift-monitoring/thanos-querier-75b99b9678-9pr69"
Sep 09 20:42:09.773505 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:09.773452    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/e34537f9-daf0-43cb-a80b-6ad6f3ace41b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-75b99b9678-9pr69\" (UID: \"e34537f9-daf0-43cb-a80b-6ad6f3ace41b\") " pod="openshift-monitoring/thanos-querier-75b99b9678-9pr69"
Sep 09 20:42:09.773575 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:09.773536    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/e34537f9-daf0-43cb-a80b-6ad6f3ace41b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-75b99b9678-9pr69\" (UID: \"e34537f9-daf0-43cb-a80b-6ad6f3ace41b\") " pod="openshift-monitoring/thanos-querier-75b99b9678-9pr69"
Sep 09 20:42:09.780596 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:09.780572    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rk8j\" (UniqueName: \"kubernetes.io/projected/e34537f9-daf0-43cb-a80b-6ad6f3ace41b-kube-api-access-8rk8j\") pod \"thanos-querier-75b99b9678-9pr69\" (UID: \"e34537f9-daf0-43cb-a80b-6ad6f3ace41b\") " pod="openshift-monitoring/thanos-querier-75b99b9678-9pr69"
Sep 09 20:42:09.836332 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:09.836298    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-75b99b9678-9pr69"
Sep 09 20:42:10.114564 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:10.114538    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-75b99b9678-9pr69"]
Sep 09 20:42:10.116335 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:42:10.116310    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode34537f9_daf0_43cb_a80b_6ad6f3ace41b.slice/crio-80eb11ea0ab50a03e558a9261c401b874418bdc67a75cd86682b3656ed307e54 WatchSource:0}: Error finding container 80eb11ea0ab50a03e558a9261c401b874418bdc67a75cd86682b3656ed307e54: Status 404 returned error can't find the container with id 80eb11ea0ab50a03e558a9261c401b874418bdc67a75cd86682b3656ed307e54
Sep 09 20:42:10.360142 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:10.360091    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-684mg" event={"ID":"4978df08-2309-45d0-85f2-614c151114ea","Type":"ContainerStarted","Data":"e26c8108fc86595bff59eefc2e82e585532d73aeae0f77f8c97324f3d6e5ae0b"}
Sep 09 20:42:10.360560 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:10.360201    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-684mg" event={"ID":"4978df08-2309-45d0-85f2-614c151114ea","Type":"ContainerStarted","Data":"bf4912c2c3a9ba682b02e23ea14ffeade1650ee238f11ed29846b1a51516cc0c"}
Sep 09 20:42:10.361382 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:10.361349    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-75b99b9678-9pr69" event={"ID":"e34537f9-daf0-43cb-a80b-6ad6f3ace41b","Type":"ContainerStarted","Data":"80eb11ea0ab50a03e558a9261c401b874418bdc67a75cd86682b3656ed307e54"}
Sep 09 20:42:10.378584 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:10.378511    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-684mg" podStartSLOduration=3.231158903 podStartE2EDuration="4.378496964s" totalImagesPullingTime="1.077862365s" totalInitContainerRuntime="69.475696ms" isStatefulPod=true podCreationTimestamp="2026-09-09 20:42:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:42:10.376966766 +0000 UTC m=+92.004594640" watchObservedRunningTime="2026-09-09 20:42:10.378496964 +0000 UTC m=+92.006124837"
Sep 09 20:42:12.764798 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:12.764755    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 09 20:42:12.769535 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:12.769512    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:42:12.772466 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:12.772259    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 09 20:42:12.772466 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:12.772282    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 09 20:42:12.772466 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:12.772295    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 09 20:42:12.772466 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:12.772346    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 09 20:42:12.772466 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:12.772354    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 09 20:42:12.772466 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:12.772431    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-cnxcl\""
Sep 09 20:42:12.772851 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:12.772705    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 09 20:42:12.773712 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:12.773694    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 09 20:42:12.774170 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:12.774148    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-98no247imvcoc\""
Sep 09 20:42:12.774306 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:12.774028    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 09 20:42:12.774394 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:12.773939    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 09 20:42:12.774565 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:12.774078    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 09 20:42:12.774777 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:12.774068    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 09 20:42:12.775370 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:12.775168    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 09 20:42:12.775460 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:12.775176    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 09 20:42:12.776376 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:12.776357    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 09 20:42:12.779288 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:12.779272    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 09 20:42:12.785343 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:12.785318    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 09 20:42:12.904946 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:12.904901    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c8f2b641-7983-4cd8-9efd-29b1b462ff7b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c8f2b641-7983-4cd8-9efd-29b1b462ff7b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:42:12.904946 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:12.904951    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c8f2b641-7983-4cd8-9efd-29b1b462ff7b-web-config\") pod \"prometheus-k8s-0\" (UID: \"c8f2b641-7983-4cd8-9efd-29b1b462ff7b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:42:12.905203 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:12.904971    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c8f2b641-7983-4cd8-9efd-29b1b462ff7b-config-out\") pod \"prometheus-k8s-0\" (UID: \"c8f2b641-7983-4cd8-9efd-29b1b462ff7b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:42:12.905203 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:12.904989    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c8f2b641-7983-4cd8-9efd-29b1b462ff7b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c8f2b641-7983-4cd8-9efd-29b1b462ff7b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:42:12.905203 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:12.905011    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c8f2b641-7983-4cd8-9efd-29b1b462ff7b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c8f2b641-7983-4cd8-9efd-29b1b462ff7b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:42:12.905203 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:12.905028    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c8f2b641-7983-4cd8-9efd-29b1b462ff7b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c8f2b641-7983-4cd8-9efd-29b1b462ff7b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:42:12.905203 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:12.905045    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c8f2b641-7983-4cd8-9efd-29b1b462ff7b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c8f2b641-7983-4cd8-9efd-29b1b462ff7b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:42:12.905203 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:12.905163    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8f2b641-7983-4cd8-9efd-29b1b462ff7b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c8f2b641-7983-4cd8-9efd-29b1b462ff7b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:42:12.905203 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:12.905205    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmht7\" (UniqueName: \"kubernetes.io/projected/c8f2b641-7983-4cd8-9efd-29b1b462ff7b-kube-api-access-rmht7\") pod \"prometheus-k8s-0\" (UID: \"c8f2b641-7983-4cd8-9efd-29b1b462ff7b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:42:12.905511 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:12.905234    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c8f2b641-7983-4cd8-9efd-29b1b462ff7b-config\") pod \"prometheus-k8s-0\" (UID: \"c8f2b641-7983-4cd8-9efd-29b1b462ff7b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:42:12.905511 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:12.905287    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8f2b641-7983-4cd8-9efd-29b1b462ff7b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c8f2b641-7983-4cd8-9efd-29b1b462ff7b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:42:12.905511 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:12.905373    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8f2b641-7983-4cd8-9efd-29b1b462ff7b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c8f2b641-7983-4cd8-9efd-29b1b462ff7b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:42:12.905511 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:12.905412    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c8f2b641-7983-4cd8-9efd-29b1b462ff7b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c8f2b641-7983-4cd8-9efd-29b1b462ff7b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:42:12.905511 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:12.905430    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c8f2b641-7983-4cd8-9efd-29b1b462ff7b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c8f2b641-7983-4cd8-9efd-29b1b462ff7b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:42:12.905511 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:12.905450    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c8f2b641-7983-4cd8-9efd-29b1b462ff7b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c8f2b641-7983-4cd8-9efd-29b1b462ff7b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:42:12.905511 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:12.905473    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c8f2b641-7983-4cd8-9efd-29b1b462ff7b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c8f2b641-7983-4cd8-9efd-29b1b462ff7b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:42:12.905769 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:12.905530    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c8f2b641-7983-4cd8-9efd-29b1b462ff7b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c8f2b641-7983-4cd8-9efd-29b1b462ff7b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:42:12.905769 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:12.905592    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c8f2b641-7983-4cd8-9efd-29b1b462ff7b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c8f2b641-7983-4cd8-9efd-29b1b462ff7b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:42:12.905769 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:12.905612    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c8f2b641-7983-4cd8-9efd-29b1b462ff7b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c8f2b641-7983-4cd8-9efd-29b1b462ff7b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:42:12.905769 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:12.905630    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c8f2b641-7983-4cd8-9efd-29b1b462ff7b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c8f2b641-7983-4cd8-9efd-29b1b462ff7b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:42:13.006872 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:13.006839    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8f2b641-7983-4cd8-9efd-29b1b462ff7b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c8f2b641-7983-4cd8-9efd-29b1b462ff7b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:42:13.007086 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:13.006883    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8f2b641-7983-4cd8-9efd-29b1b462ff7b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c8f2b641-7983-4cd8-9efd-29b1b462ff7b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:42:13.007086 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:13.006901    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c8f2b641-7983-4cd8-9efd-29b1b462ff7b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c8f2b641-7983-4cd8-9efd-29b1b462ff7b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:42:13.007086 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:13.006918    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c8f2b641-7983-4cd8-9efd-29b1b462ff7b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c8f2b641-7983-4cd8-9efd-29b1b462ff7b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:42:13.007086 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:13.006942    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c8f2b641-7983-4cd8-9efd-29b1b462ff7b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c8f2b641-7983-4cd8-9efd-29b1b462ff7b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:42:13.007086 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:13.006959    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c8f2b641-7983-4cd8-9efd-29b1b462ff7b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c8f2b641-7983-4cd8-9efd-29b1b462ff7b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:42:13.007086 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:13.006984    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c8f2b641-7983-4cd8-9efd-29b1b462ff7b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c8f2b641-7983-4cd8-9efd-29b1b462ff7b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:42:13.007506 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:13.007477    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c8f2b641-7983-4cd8-9efd-29b1b462ff7b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c8f2b641-7983-4cd8-9efd-29b1b462ff7b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:42:13.008240 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:13.007635    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c8f2b641-7983-4cd8-9efd-29b1b462ff7b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c8f2b641-7983-4cd8-9efd-29b1b462ff7b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:42:13.008240 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:13.007686    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c8f2b641-7983-4cd8-9efd-29b1b462ff7b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c8f2b641-7983-4cd8-9efd-29b1b462ff7b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:42:13.008240 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:13.007720    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c8f2b641-7983-4cd8-9efd-29b1b462ff7b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c8f2b641-7983-4cd8-9efd-29b1b462ff7b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:42:13.008240 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:13.007790    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c8f2b641-7983-4cd8-9efd-29b1b462ff7b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c8f2b641-7983-4cd8-9efd-29b1b462ff7b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:42:13.008240 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:13.007817    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c8f2b641-7983-4cd8-9efd-29b1b462ff7b-web-config\") pod \"prometheus-k8s-0\" (UID: \"c8f2b641-7983-4cd8-9efd-29b1b462ff7b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:42:13.008240 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:13.007845    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c8f2b641-7983-4cd8-9efd-29b1b462ff7b-config-out\") pod \"prometheus-k8s-0\" (UID: \"c8f2b641-7983-4cd8-9efd-29b1b462ff7b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:42:13.008240 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:13.007865    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c8f2b641-7983-4cd8-9efd-29b1b462ff7b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c8f2b641-7983-4cd8-9efd-29b1b462ff7b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:42:13.008240 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:13.007891    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c8f2b641-7983-4cd8-9efd-29b1b462ff7b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c8f2b641-7983-4cd8-9efd-29b1b462ff7b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:42:13.008240 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:13.007921    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c8f2b641-7983-4cd8-9efd-29b1b462ff7b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c8f2b641-7983-4cd8-9efd-29b1b462ff7b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:42:13.008240 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:13.007953    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c8f2b641-7983-4cd8-9efd-29b1b462ff7b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c8f2b641-7983-4cd8-9efd-29b1b462ff7b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:42:13.008240 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:13.008000    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8f2b641-7983-4cd8-9efd-29b1b462ff7b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c8f2b641-7983-4cd8-9efd-29b1b462ff7b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:42:13.008240 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:13.008046    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmht7\" (UniqueName: \"kubernetes.io/projected/c8f2b641-7983-4cd8-9efd-29b1b462ff7b-kube-api-access-rmht7\") pod \"prometheus-k8s-0\" (UID: \"c8f2b641-7983-4cd8-9efd-29b1b462ff7b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:42:13.008240 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:13.008075    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c8f2b641-7983-4cd8-9efd-29b1b462ff7b-config\") pod \"prometheus-k8s-0\" (UID: \"c8f2b641-7983-4cd8-9efd-29b1b462ff7b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:42:13.008240 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:13.008116    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8f2b641-7983-4cd8-9efd-29b1b462ff7b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c8f2b641-7983-4cd8-9efd-29b1b462ff7b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:42:13.011408 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:13.009568    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c8f2b641-7983-4cd8-9efd-29b1b462ff7b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c8f2b641-7983-4cd8-9efd-29b1b462ff7b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:42:13.011408 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:13.010193    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8f2b641-7983-4cd8-9efd-29b1b462ff7b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c8f2b641-7983-4cd8-9efd-29b1b462ff7b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:42:13.011408 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:13.010291    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c8f2b641-7983-4cd8-9efd-29b1b462ff7b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c8f2b641-7983-4cd8-9efd-29b1b462ff7b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:42:13.011408 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:13.010347    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c8f2b641-7983-4cd8-9efd-29b1b462ff7b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c8f2b641-7983-4cd8-9efd-29b1b462ff7b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:42:13.011408 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:13.010794    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c8f2b641-7983-4cd8-9efd-29b1b462ff7b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c8f2b641-7983-4cd8-9efd-29b1b462ff7b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:42:13.012424 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:13.011936    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c8f2b641-7983-4cd8-9efd-29b1b462ff7b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c8f2b641-7983-4cd8-9efd-29b1b462ff7b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:42:13.012424 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:13.012032    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8f2b641-7983-4cd8-9efd-29b1b462ff7b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c8f2b641-7983-4cd8-9efd-29b1b462ff7b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:42:13.012424 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:13.012386    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c8f2b641-7983-4cd8-9efd-29b1b462ff7b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c8f2b641-7983-4cd8-9efd-29b1b462ff7b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:42:13.012627 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:13.012506    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c8f2b641-7983-4cd8-9efd-29b1b462ff7b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c8f2b641-7983-4cd8-9efd-29b1b462ff7b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:42:13.012627 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:13.012528    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c8f2b641-7983-4cd8-9efd-29b1b462ff7b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c8f2b641-7983-4cd8-9efd-29b1b462ff7b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:42:13.013731 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:13.013553    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c8f2b641-7983-4cd8-9efd-29b1b462ff7b-config-out\") pod \"prometheus-k8s-0\" (UID: \"c8f2b641-7983-4cd8-9efd-29b1b462ff7b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:42:13.013731 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:13.013689    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c8f2b641-7983-4cd8-9efd-29b1b462ff7b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c8f2b641-7983-4cd8-9efd-29b1b462ff7b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:42:13.013731 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:13.013694    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c8f2b641-7983-4cd8-9efd-29b1b462ff7b-web-config\") pod \"prometheus-k8s-0\" (UID: \"c8f2b641-7983-4cd8-9efd-29b1b462ff7b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:42:13.014318 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:13.014291    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c8f2b641-7983-4cd8-9efd-29b1b462ff7b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c8f2b641-7983-4cd8-9efd-29b1b462ff7b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:42:13.014415 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:13.014362    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/c8f2b641-7983-4cd8-9efd-29b1b462ff7b-config\") pod \"prometheus-k8s-0\" (UID: \"c8f2b641-7983-4cd8-9efd-29b1b462ff7b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:42:13.015315 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:13.014924    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c8f2b641-7983-4cd8-9efd-29b1b462ff7b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c8f2b641-7983-4cd8-9efd-29b1b462ff7b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:42:13.015315 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:13.015235    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c8f2b641-7983-4cd8-9efd-29b1b462ff7b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c8f2b641-7983-4cd8-9efd-29b1b462ff7b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:42:13.017853 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:13.017829    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmht7\" (UniqueName: \"kubernetes.io/projected/c8f2b641-7983-4cd8-9efd-29b1b462ff7b-kube-api-access-rmht7\") pod \"prometheus-k8s-0\" (UID: \"c8f2b641-7983-4cd8-9efd-29b1b462ff7b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:42:13.081842 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:13.081811    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:42:13.278932 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:13.278864    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-69887ff495-l9r6j"
Sep 09 20:42:13.374787 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:13.374680    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 09 20:42:13.377486 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:42:13.377452    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8f2b641_7983_4cd8_9efd_29b1b462ff7b.slice/crio-17926064549e2ae695c724b554118313c0e3e534cacf76d998ec3672d0be960d WatchSource:0}: Error finding container 17926064549e2ae695c724b554118313c0e3e534cacf76d998ec3672d0be960d: Status 404 returned error can't find the container with id 17926064549e2ae695c724b554118313c0e3e534cacf76d998ec3672d0be960d
Sep 09 20:42:13.380146 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:13.380120    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-75b99b9678-9pr69" event={"ID":"e34537f9-daf0-43cb-a80b-6ad6f3ace41b","Type":"ContainerStarted","Data":"d912382a0afd440452357e774e04d978ce8584206d3a2188492a2e4c5f7f233e"}
Sep 09 20:42:13.380240 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:13.380190    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-75b99b9678-9pr69" event={"ID":"e34537f9-daf0-43cb-a80b-6ad6f3ace41b","Type":"ContainerStarted","Data":"9a4e746787fb2e967576e42b481c270c4584f8e76428b45d8cbfd11194b230ce"}
Sep 09 20:42:13.380240 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:13.380231    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-75b99b9678-9pr69" event={"ID":"e34537f9-daf0-43cb-a80b-6ad6f3ace41b","Type":"ContainerStarted","Data":"562ed68f2ee80d78d4a1166287eb190eb17501e124ebff58aea2d9d19d90462e"}
Sep 09 20:42:14.387531 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:14.387495    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-75b99b9678-9pr69" event={"ID":"e34537f9-daf0-43cb-a80b-6ad6f3ace41b","Type":"ContainerStarted","Data":"abcd2400b5c313c55a513f12c41c8f1d8e43bd062af992c66cdd759b3f3491f6"}
Sep 09 20:42:14.387974 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:14.387568    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-75b99b9678-9pr69" event={"ID":"e34537f9-daf0-43cb-a80b-6ad6f3ace41b","Type":"ContainerStarted","Data":"885add9edb024b66b4d772734f73da6f5bf93ff6c5853703d1f3b8dec008a943"}
Sep 09 20:42:14.387974 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:14.387607    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-75b99b9678-9pr69"
Sep 09 20:42:14.387974 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:14.387645    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-75b99b9678-9pr69" event={"ID":"e34537f9-daf0-43cb-a80b-6ad6f3ace41b","Type":"ContainerStarted","Data":"9f2c18ca6e009784b22e237e62b011fa7e1b8580271f57c8b2c2c8f349aba57e"}
Sep 09 20:42:14.388811 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:14.388790    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c8f2b641-7983-4cd8-9efd-29b1b462ff7b","Type":"ContainerStarted","Data":"17926064549e2ae695c724b554118313c0e3e534cacf76d998ec3672d0be960d"}
Sep 09 20:42:14.412432 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:14.412386    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-75b99b9678-9pr69" podStartSLOduration=2.136468074 podStartE2EDuration="5.412369899s" totalImagesPullingTime="3.275901825s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:42:09 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:42:14.409988061 +0000 UTC m=+96.037615936" watchObservedRunningTime="2026-09-09 20:42:14.412369899 +0000 UTC m=+96.039997789"
Sep 09 20:42:15.394886 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:15.394847    2581 generic.go:356] "Generic (PLEG): container finished" podID="c8f2b641-7983-4cd8-9efd-29b1b462ff7b" containerID="f3b7d430c09c4a4e9f74dc8ff729d7c7bfb034ca8258159d3450c6fa65d8407b" exitCode=0
Sep 09 20:42:15.395298 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:15.394934    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c8f2b641-7983-4cd8-9efd-29b1b462ff7b","Type":"ContainerDied","Data":"f3b7d430c09c4a4e9f74dc8ff729d7c7bfb034ca8258159d3450c6fa65d8407b"}
Sep 09 20:42:19.415369 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:19.415331    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c8f2b641-7983-4cd8-9efd-29b1b462ff7b","Type":"ContainerStarted","Data":"c936888096d51376bf471180ccbc01ae595fd865be3db708af7cab73dda83af6"}
Sep 09 20:42:19.415876 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:19.415410    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c8f2b641-7983-4cd8-9efd-29b1b462ff7b","Type":"ContainerStarted","Data":"95cd0afb9fc56aebcafc63f25f8c5e964708ea4140c22c80bb1bf2001789103e"}
Sep 09 20:42:19.415876 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:19.415453    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c8f2b641-7983-4cd8-9efd-29b1b462ff7b","Type":"ContainerStarted","Data":"35552be5019a8c1a077bdab24afdc750302f576b5abdb3819dd21740eb21a791"}
Sep 09 20:42:19.415876 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:19.415501    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c8f2b641-7983-4cd8-9efd-29b1b462ff7b","Type":"ContainerStarted","Data":"59aac44d2f462c86bfc9d7f1c09a2b2c9d5660b57e298779a5727c06e9f48b79"}
Sep 09 20:42:19.415876 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:19.415544    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c8f2b641-7983-4cd8-9efd-29b1b462ff7b","Type":"ContainerStarted","Data":"b7a5c4a0f59c17f3fdcd0b95ebe825b65d5b8e85a43b126993c73e6862c6d74b"}
Sep 09 20:42:19.415876 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:19.415625    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c8f2b641-7983-4cd8-9efd-29b1b462ff7b","Type":"ContainerStarted","Data":"a4a2164f3412b20089cca35992b4e456aefbe554e2f7f0c89f84c2e605c89eb0"}
Sep 09 20:42:19.444996 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:19.444939    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.087570045 podStartE2EDuration="7.444921667s" totalImagesPullingTime="4.33680979s" totalInitContainerRuntime="20.541832ms" isStatefulPod=false podCreationTimestamp="2026-09-09 20:42:12 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:42:19.442769153 +0000 UTC m=+101.070397050" watchObservedRunningTime="2026-09-09 20:42:19.444921667 +0000 UTC m=+101.072549614"
Sep 09 20:42:20.402400 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:20.402369    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-75b99b9678-9pr69"
Sep 09 20:42:21.267773 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:21.267743    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-2c5hn"
Sep 09 20:42:23.082407 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:23.082371    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:42:56.564589 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:56.564550    2581 generic.go:356] "Generic (PLEG): container finished" podID="4eb46131-2f57-408b-9d0f-a9ee9b55c8ef" containerID="e1402a01c29e9c1f6d000bf974ccaa2182d83bec3a8e9c4f33f1c337b944ed3c" exitCode=0
Sep 09 20:42:56.565051 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:56.564614    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-r5kxh" event={"ID":"4eb46131-2f57-408b-9d0f-a9ee9b55c8ef","Type":"ContainerDied","Data":"e1402a01c29e9c1f6d000bf974ccaa2182d83bec3a8e9c4f33f1c337b944ed3c"}
Sep 09 20:42:56.565051 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:56.564969    2581 scope.go:117] "RemoveContainer" containerID="e1402a01c29e9c1f6d000bf974ccaa2182d83bec3a8e9c4f33f1c337b944ed3c"
Sep 09 20:42:57.569939 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:57.569888    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7f6c8f5bc8-h8tgh_d5d449a3-ab1b-4899-842d-fcc9accb0838/router/0.log"
Sep 09 20:42:57.571481 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:57.571449    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-r5kxh" event={"ID":"4eb46131-2f57-408b-9d0f-a9ee9b55c8ef","Type":"ContainerStarted","Data":"4560dfdcd3f3afe3989ee382aacc544ef74afa129a6f4ead632ad7a30c7d3b38"}
Sep 09 20:42:57.576347 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:42:57.576305    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4zgnb_933d2649-0bcc-4c52-a0ac-65f33ba49741/serve-healthcheck-canary/0.log"
Sep 09 20:43:13.082949 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:43:13.082902    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:43:13.099978 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:43:13.099938    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:43:13.657779 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:43:13.657747    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 09 20:45:38.738453 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:45:38.738411    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-wsv4v_2b1f9a6e-7a79-4e89-ac3f-4658d6d91550/console-operator/1.log"
Sep 09 20:45:38.741029 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:45:38.741000    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-wsv4v_2b1f9a6e-7a79-4e89-ac3f-4658d6d91550/console-operator/1.log"
Sep 09 20:45:38.746488 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:45:38.746426    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-75wj4_55b4fbae-29dc-41bf-a907-b1412b852cef/ovn-acl-logging/0.log"
Sep 09 20:45:38.755204 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:45:38.755171    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-75wj4_55b4fbae-29dc-41bf-a907-b1412b852cef/ovn-acl-logging/0.log"
Sep 09 20:45:38.755721 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:45:38.755700    2581 kubelet.go:1648] "Image garbage collection succeeded"
Sep 09 20:46:18.820503 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:18.820472    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6d46659dfc-vjcbm"]
Sep 09 20:46:18.823966 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:18.823945    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6d46659dfc-vjcbm"
Sep 09 20:46:18.828901 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:18.828864    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 09 20:46:18.829125 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:18.828990    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 09 20:46:18.829323 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:18.829302    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 09 20:46:18.829684 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:18.829538    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 09 20:46:18.829684 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:18.829606    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-4jzzz\""
Sep 09 20:46:18.846798 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:18.846766    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6d46659dfc-vjcbm"]
Sep 09 20:46:18.867999 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:18.867937    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ba2f8e6c-79c7-4f53-8fdf-bc0b952a7082-manifests\") pod \"opendatahub-operator-controller-manager-6d46659dfc-vjcbm\" (UID: \"ba2f8e6c-79c7-4f53-8fdf-bc0b952a7082\") " pod="opendatahub/opendatahub-operator-controller-manager-6d46659dfc-vjcbm"
Sep 09 20:46:18.868227 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:18.868079    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ba2f8e6c-79c7-4f53-8fdf-bc0b952a7082-webhook-cert\") pod \"opendatahub-operator-controller-manager-6d46659dfc-vjcbm\" (UID: \"ba2f8e6c-79c7-4f53-8fdf-bc0b952a7082\") " pod="opendatahub/opendatahub-operator-controller-manager-6d46659dfc-vjcbm"
Sep 09 20:46:18.868227 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:18.868155    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c696v\" (UniqueName: \"kubernetes.io/projected/ba2f8e6c-79c7-4f53-8fdf-bc0b952a7082-kube-api-access-c696v\") pod \"opendatahub-operator-controller-manager-6d46659dfc-vjcbm\" (UID: \"ba2f8e6c-79c7-4f53-8fdf-bc0b952a7082\") " pod="opendatahub/opendatahub-operator-controller-manager-6d46659dfc-vjcbm"
Sep 09 20:46:18.868350 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:18.868234    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ba2f8e6c-79c7-4f53-8fdf-bc0b952a7082-tmp\") pod \"opendatahub-operator-controller-manager-6d46659dfc-vjcbm\" (UID: \"ba2f8e6c-79c7-4f53-8fdf-bc0b952a7082\") " pod="opendatahub/opendatahub-operator-controller-manager-6d46659dfc-vjcbm"
Sep 09 20:46:18.868469 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:18.868442    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ba2f8e6c-79c7-4f53-8fdf-bc0b952a7082-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6d46659dfc-vjcbm\" (UID: \"ba2f8e6c-79c7-4f53-8fdf-bc0b952a7082\") " pod="opendatahub/opendatahub-operator-controller-manager-6d46659dfc-vjcbm"
Sep 09 20:46:18.969344 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:18.969297    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ba2f8e6c-79c7-4f53-8fdf-bc0b952a7082-tmp\") pod \"opendatahub-operator-controller-manager-6d46659dfc-vjcbm\" (UID: \"ba2f8e6c-79c7-4f53-8fdf-bc0b952a7082\") " pod="opendatahub/opendatahub-operator-controller-manager-6d46659dfc-vjcbm"
Sep 09 20:46:18.969531 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:18.969463    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ba2f8e6c-79c7-4f53-8fdf-bc0b952a7082-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6d46659dfc-vjcbm\" (UID: \"ba2f8e6c-79c7-4f53-8fdf-bc0b952a7082\") " pod="opendatahub/opendatahub-operator-controller-manager-6d46659dfc-vjcbm"
Sep 09 20:46:18.969581 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:18.969547    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ba2f8e6c-79c7-4f53-8fdf-bc0b952a7082-manifests\") pod \"opendatahub-operator-controller-manager-6d46659dfc-vjcbm\" (UID: \"ba2f8e6c-79c7-4f53-8fdf-bc0b952a7082\") " pod="opendatahub/opendatahub-operator-controller-manager-6d46659dfc-vjcbm"
Sep 09 20:46:18.969581 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:18.969566    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ba2f8e6c-79c7-4f53-8fdf-bc0b952a7082-webhook-cert\") pod \"opendatahub-operator-controller-manager-6d46659dfc-vjcbm\" (UID: \"ba2f8e6c-79c7-4f53-8fdf-bc0b952a7082\") " pod="opendatahub/opendatahub-operator-controller-manager-6d46659dfc-vjcbm"
Sep 09 20:46:18.969655 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:18.969586    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c696v\" (UniqueName: \"kubernetes.io/projected/ba2f8e6c-79c7-4f53-8fdf-bc0b952a7082-kube-api-access-c696v\") pod \"opendatahub-operator-controller-manager-6d46659dfc-vjcbm\" (UID: \"ba2f8e6c-79c7-4f53-8fdf-bc0b952a7082\") " pod="opendatahub/opendatahub-operator-controller-manager-6d46659dfc-vjcbm"
Sep 09 20:46:18.969931 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:18.969910    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ba2f8e6c-79c7-4f53-8fdf-bc0b952a7082-manifests\") pod \"opendatahub-operator-controller-manager-6d46659dfc-vjcbm\" (UID: \"ba2f8e6c-79c7-4f53-8fdf-bc0b952a7082\") " pod="opendatahub/opendatahub-operator-controller-manager-6d46659dfc-vjcbm"
Sep 09 20:46:18.971730 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:18.971705    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ba2f8e6c-79c7-4f53-8fdf-bc0b952a7082-tmp\") pod \"opendatahub-operator-controller-manager-6d46659dfc-vjcbm\" (UID: \"ba2f8e6c-79c7-4f53-8fdf-bc0b952a7082\") " pod="opendatahub/opendatahub-operator-controller-manager-6d46659dfc-vjcbm"
Sep 09 20:46:18.972000 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:18.971981    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ba2f8e6c-79c7-4f53-8fdf-bc0b952a7082-webhook-cert\") pod \"opendatahub-operator-controller-manager-6d46659dfc-vjcbm\" (UID: \"ba2f8e6c-79c7-4f53-8fdf-bc0b952a7082\") " pod="opendatahub/opendatahub-operator-controller-manager-6d46659dfc-vjcbm"
Sep 09 20:46:18.972096 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:18.972065    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ba2f8e6c-79c7-4f53-8fdf-bc0b952a7082-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6d46659dfc-vjcbm\" (UID: \"ba2f8e6c-79c7-4f53-8fdf-bc0b952a7082\") " pod="opendatahub/opendatahub-operator-controller-manager-6d46659dfc-vjcbm"
Sep 09 20:46:18.978402 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:18.978378    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c696v\" (UniqueName: \"kubernetes.io/projected/ba2f8e6c-79c7-4f53-8fdf-bc0b952a7082-kube-api-access-c696v\") pod \"opendatahub-operator-controller-manager-6d46659dfc-vjcbm\" (UID: \"ba2f8e6c-79c7-4f53-8fdf-bc0b952a7082\") " pod="opendatahub/opendatahub-operator-controller-manager-6d46659dfc-vjcbm"
Sep 09 20:46:19.134980 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:19.134889    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6d46659dfc-vjcbm"
Sep 09 20:46:19.536428 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:19.536399    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6d46659dfc-vjcbm"]
Sep 09 20:46:19.539744 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:46:19.539710    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba2f8e6c_79c7_4f53_8fdf_bc0b952a7082.slice/crio-f223d8178929c66667b3362db5bdf51f3e418aac874573d19b83ad23e9155e72 WatchSource:0}: Error finding container f223d8178929c66667b3362db5bdf51f3e418aac874573d19b83ad23e9155e72: Status 404 returned error can't find the container with id f223d8178929c66667b3362db5bdf51f3e418aac874573d19b83ad23e9155e72
Sep 09 20:46:19.541475 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:19.541455    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 20:46:20.392809 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:20.392768    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6d46659dfc-vjcbm" event={"ID":"ba2f8e6c-79c7-4f53-8fdf-bc0b952a7082","Type":"ContainerStarted","Data":"f223d8178929c66667b3362db5bdf51f3e418aac874573d19b83ad23e9155e72"}
Sep 09 20:46:23.408886 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:23.408849    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6d46659dfc-vjcbm" event={"ID":"ba2f8e6c-79c7-4f53-8fdf-bc0b952a7082","Type":"ContainerStarted","Data":"28822a4b97be23ded6c49f1c1f02957137acec8f6c21dcfa0717a2f3003ffdbf"}
Sep 09 20:46:24.415412 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:24.415370    2581 generic.go:356] "Generic (PLEG): container finished" podID="ba2f8e6c-79c7-4f53-8fdf-bc0b952a7082" containerID="28822a4b97be23ded6c49f1c1f02957137acec8f6c21dcfa0717a2f3003ffdbf" exitCode=0
Sep 09 20:46:24.415826 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:24.415437    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6d46659dfc-vjcbm" event={"ID":"ba2f8e6c-79c7-4f53-8fdf-bc0b952a7082","Type":"ContainerDied","Data":"28822a4b97be23ded6c49f1c1f02957137acec8f6c21dcfa0717a2f3003ffdbf"}
Sep 09 20:46:25.427081 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:25.427032    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6d46659dfc-vjcbm" event={"ID":"ba2f8e6c-79c7-4f53-8fdf-bc0b952a7082","Type":"ContainerStarted","Data":"d95be34a6dcf0f96154a3305f41bcf96ce4fd315f1db6f0b26ddc810cae01edf"}
Sep 09 20:46:25.427748 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:25.427724    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6d46659dfc-vjcbm"
Sep 09 20:46:36.434486 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:36.434398    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6d46659dfc-vjcbm"
Sep 09 20:46:36.474999 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:36.474938    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6d46659dfc-vjcbm" podStartSLOduration=13.82717147 podStartE2EDuration="18.474919913s" totalImagesPullingTime="3.527613507s" totalInitContainerRuntime="1.120134936s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:46:18 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:46:25.468957509 +0000 UTC m=+347.096585382" watchObservedRunningTime="2026-09-09 20:46:36.474919913 +0000 UTC m=+358.102547785"
Sep 09 20:46:37.002937 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:37.002898    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-58fbd4c7f6-mvldh"]
Sep 09 20:46:37.009069 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:37.009047    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-58fbd4c7f6-mvldh"
Sep 09 20:46:37.022119 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:37.019495    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 09 20:46:37.022119 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:37.019601    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 09 20:46:37.022119 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:37.019849    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-mj7jb\""
Sep 09 20:46:37.022119 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:37.019854    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 09 20:46:37.022119 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:37.020143    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 09 20:46:37.022119 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:37.020398    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 09 20:46:37.024746 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:37.024714    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-58fbd4c7f6-mvldh"]
Sep 09 20:46:37.130024 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:37.129966    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d970c548-c924-495f-8362-85fe35786ef4-manager-config\") pod \"lws-controller-manager-58fbd4c7f6-mvldh\" (UID: \"d970c548-c924-495f-8362-85fe35786ef4\") " pod="openshift-lws-operator/lws-controller-manager-58fbd4c7f6-mvldh"
Sep 09 20:46:37.130024 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:37.130026    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d970c548-c924-495f-8362-85fe35786ef4-metrics-cert\") pod \"lws-controller-manager-58fbd4c7f6-mvldh\" (UID: \"d970c548-c924-495f-8362-85fe35786ef4\") " pod="openshift-lws-operator/lws-controller-manager-58fbd4c7f6-mvldh"
Sep 09 20:46:37.130267 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:37.130138    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d970c548-c924-495f-8362-85fe35786ef4-cert\") pod \"lws-controller-manager-58fbd4c7f6-mvldh\" (UID: \"d970c548-c924-495f-8362-85fe35786ef4\") " pod="openshift-lws-operator/lws-controller-manager-58fbd4c7f6-mvldh"
Sep 09 20:46:37.130267 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:37.130175    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngrw2\" (UniqueName: \"kubernetes.io/projected/d970c548-c924-495f-8362-85fe35786ef4-kube-api-access-ngrw2\") pod \"lws-controller-manager-58fbd4c7f6-mvldh\" (UID: \"d970c548-c924-495f-8362-85fe35786ef4\") " pod="openshift-lws-operator/lws-controller-manager-58fbd4c7f6-mvldh"
Sep 09 20:46:37.231062 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:37.231023    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d970c548-c924-495f-8362-85fe35786ef4-manager-config\") pod \"lws-controller-manager-58fbd4c7f6-mvldh\" (UID: \"d970c548-c924-495f-8362-85fe35786ef4\") " pod="openshift-lws-operator/lws-controller-manager-58fbd4c7f6-mvldh"
Sep 09 20:46:37.231062 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:37.231061    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d970c548-c924-495f-8362-85fe35786ef4-metrics-cert\") pod \"lws-controller-manager-58fbd4c7f6-mvldh\" (UID: \"d970c548-c924-495f-8362-85fe35786ef4\") " pod="openshift-lws-operator/lws-controller-manager-58fbd4c7f6-mvldh"
Sep 09 20:46:37.231277 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:37.231151    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d970c548-c924-495f-8362-85fe35786ef4-cert\") pod \"lws-controller-manager-58fbd4c7f6-mvldh\" (UID: \"d970c548-c924-495f-8362-85fe35786ef4\") " pod="openshift-lws-operator/lws-controller-manager-58fbd4c7f6-mvldh"
Sep 09 20:46:37.231277 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:37.231177    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngrw2\" (UniqueName: \"kubernetes.io/projected/d970c548-c924-495f-8362-85fe35786ef4-kube-api-access-ngrw2\") pod \"lws-controller-manager-58fbd4c7f6-mvldh\" (UID: \"d970c548-c924-495f-8362-85fe35786ef4\") " pod="openshift-lws-operator/lws-controller-manager-58fbd4c7f6-mvldh"
Sep 09 20:46:37.231802 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:37.231775    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d970c548-c924-495f-8362-85fe35786ef4-manager-config\") pod \"lws-controller-manager-58fbd4c7f6-mvldh\" (UID: \"d970c548-c924-495f-8362-85fe35786ef4\") " pod="openshift-lws-operator/lws-controller-manager-58fbd4c7f6-mvldh"
Sep 09 20:46:37.233683 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:37.233662    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d970c548-c924-495f-8362-85fe35786ef4-metrics-cert\") pod \"lws-controller-manager-58fbd4c7f6-mvldh\" (UID: \"d970c548-c924-495f-8362-85fe35786ef4\") " pod="openshift-lws-operator/lws-controller-manager-58fbd4c7f6-mvldh"
Sep 09 20:46:37.233800 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:37.233717    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d970c548-c924-495f-8362-85fe35786ef4-cert\") pod \"lws-controller-manager-58fbd4c7f6-mvldh\" (UID: \"d970c548-c924-495f-8362-85fe35786ef4\") " pod="openshift-lws-operator/lws-controller-manager-58fbd4c7f6-mvldh"
Sep 09 20:46:37.240961 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:37.240931    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngrw2\" (UniqueName: \"kubernetes.io/projected/d970c548-c924-495f-8362-85fe35786ef4-kube-api-access-ngrw2\") pod \"lws-controller-manager-58fbd4c7f6-mvldh\" (UID: \"d970c548-c924-495f-8362-85fe35786ef4\") " pod="openshift-lws-operator/lws-controller-manager-58fbd4c7f6-mvldh"
Sep 09 20:46:37.320368 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:37.320256    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-58fbd4c7f6-mvldh"
Sep 09 20:46:37.602562 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:37.602533    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-58fbd4c7f6-mvldh"]
Sep 09 20:46:37.605258 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:46:37.605225    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd970c548_c924_495f_8362_85fe35786ef4.slice/crio-57dabd85166f034b5252a3d5c8c136db99bc3f5bb0f9c3829e4a22c40b397408 WatchSource:0}: Error finding container 57dabd85166f034b5252a3d5c8c136db99bc3f5bb0f9c3829e4a22c40b397408: Status 404 returned error can't find the container with id 57dabd85166f034b5252a3d5c8c136db99bc3f5bb0f9c3829e4a22c40b397408
Sep 09 20:46:38.489206 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:38.489066    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-58fbd4c7f6-mvldh" event={"ID":"d970c548-c924-495f-8362-85fe35786ef4","Type":"ContainerStarted","Data":"57dabd85166f034b5252a3d5c8c136db99bc3f5bb0f9c3829e4a22c40b397408"}
Sep 09 20:46:40.500574 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:40.500535    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-58fbd4c7f6-mvldh" event={"ID":"d970c548-c924-495f-8362-85fe35786ef4","Type":"ContainerStarted","Data":"75b9a0a574509115b0c40b8039b40b4b42ece6c0d3bb0c227508891872258900"}
Sep 09 20:46:40.500969 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:40.500644    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-58fbd4c7f6-mvldh"
Sep 09 20:46:40.554654 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:40.554595    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-58fbd4c7f6-mvldh" podStartSLOduration=2.001222248 podStartE2EDuration="4.554575349s" totalImagesPullingTime="2.553353101s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:46:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:46:40.552615062 +0000 UTC m=+362.180242936" watchObservedRunningTime="2026-09-09 20:46:40.554575349 +0000 UTC m=+362.182203222"
Sep 09 20:46:50.283919 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:50.283874    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-6k5n7"]
Sep 09 20:46:50.287651 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:50.287630    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-6k5n7"
Sep 09 20:46:50.297240 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:50.297214    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-qdqfh\""
Sep 09 20:46:50.297693 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:50.297675    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Sep 09 20:46:50.316894 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:50.316855    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-6k5n7"]
Sep 09 20:46:50.344440 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:50.344404    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fd53ab69-cbaa-4965-9333-96549bbd2c70-cert\") pod \"odh-model-controller-7998c67dc4-6k5n7\" (UID: \"fd53ab69-cbaa-4965-9333-96549bbd2c70\") " pod="opendatahub/odh-model-controller-7998c67dc4-6k5n7"
Sep 09 20:46:50.344610 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:50.344454    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86lpg\" (UniqueName: \"kubernetes.io/projected/fd53ab69-cbaa-4965-9333-96549bbd2c70-kube-api-access-86lpg\") pod \"odh-model-controller-7998c67dc4-6k5n7\" (UID: \"fd53ab69-cbaa-4965-9333-96549bbd2c70\") " pod="opendatahub/odh-model-controller-7998c67dc4-6k5n7"
Sep 09 20:46:50.445239 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:50.445200    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fd53ab69-cbaa-4965-9333-96549bbd2c70-cert\") pod \"odh-model-controller-7998c67dc4-6k5n7\" (UID: \"fd53ab69-cbaa-4965-9333-96549bbd2c70\") " pod="opendatahub/odh-model-controller-7998c67dc4-6k5n7"
Sep 09 20:46:50.445425 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:50.445277    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86lpg\" (UniqueName: \"kubernetes.io/projected/fd53ab69-cbaa-4965-9333-96549bbd2c70-kube-api-access-86lpg\") pod \"odh-model-controller-7998c67dc4-6k5n7\" (UID: \"fd53ab69-cbaa-4965-9333-96549bbd2c70\") " pod="opendatahub/odh-model-controller-7998c67dc4-6k5n7"
Sep 09 20:46:50.445425 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:46:50.445358    2581 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Sep 09 20:46:50.445505 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:46:50.445431    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd53ab69-cbaa-4965-9333-96549bbd2c70-cert podName:fd53ab69-cbaa-4965-9333-96549bbd2c70 nodeName:}" failed. No retries permitted until 2026-09-09 20:46:50.945409772 +0000 UTC m=+372.573037641 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/fd53ab69-cbaa-4965-9333-96549bbd2c70-cert") pod "odh-model-controller-7998c67dc4-6k5n7" (UID: "fd53ab69-cbaa-4965-9333-96549bbd2c70") : secret "odh-model-controller-webhook-cert" not found
Sep 09 20:46:50.480259 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:50.480223    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-86lpg\" (UniqueName: \"kubernetes.io/projected/fd53ab69-cbaa-4965-9333-96549bbd2c70-kube-api-access-86lpg\") pod \"odh-model-controller-7998c67dc4-6k5n7\" (UID: \"fd53ab69-cbaa-4965-9333-96549bbd2c70\") " pod="opendatahub/odh-model-controller-7998c67dc4-6k5n7"
Sep 09 20:46:50.950776 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:50.950728    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fd53ab69-cbaa-4965-9333-96549bbd2c70-cert\") pod \"odh-model-controller-7998c67dc4-6k5n7\" (UID: \"fd53ab69-cbaa-4965-9333-96549bbd2c70\") " pod="opendatahub/odh-model-controller-7998c67dc4-6k5n7"
Sep 09 20:46:50.953317 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:50.953294    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fd53ab69-cbaa-4965-9333-96549bbd2c70-cert\") pod \"odh-model-controller-7998c67dc4-6k5n7\" (UID: \"fd53ab69-cbaa-4965-9333-96549bbd2c70\") " pod="opendatahub/odh-model-controller-7998c67dc4-6k5n7"
Sep 09 20:46:51.197947 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:51.197905    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-6k5n7"
Sep 09 20:46:51.506665 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:51.506631    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-58fbd4c7f6-mvldh"
Sep 09 20:46:51.581415 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:46:51.581382    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd53ab69_cbaa_4965_9333_96549bbd2c70.slice/crio-5d2c89f881bfcf1780e04c769e982296fd5b44c5f1a08f6cca28b88cccb50d6b WatchSource:0}: Error finding container 5d2c89f881bfcf1780e04c769e982296fd5b44c5f1a08f6cca28b88cccb50d6b: Status 404 returned error can't find the container with id 5d2c89f881bfcf1780e04c769e982296fd5b44c5f1a08f6cca28b88cccb50d6b
Sep 09 20:46:51.585272 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:51.585245    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-6k5n7"]
Sep 09 20:46:51.755067 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:51.755031    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-8l6p8"]
Sep 09 20:46:51.759664 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:51.759602    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-8l6p8"
Sep 09 20:46:51.771887 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:51.771866    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Sep 09 20:46:51.772208 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:51.772194    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-r784g\""
Sep 09 20:46:51.834822 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:51.834782    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-8l6p8"]
Sep 09 20:46:51.857254 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:51.857214    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87cp8\" (UniqueName: \"kubernetes.io/projected/8dcbd9c0-e30e-44ac-997d-27e33b31371d-kube-api-access-87cp8\") pod \"model-serving-api-54d89bd5fc-8l6p8\" (UID: \"8dcbd9c0-e30e-44ac-997d-27e33b31371d\") " pod="opendatahub/model-serving-api-54d89bd5fc-8l6p8"
Sep 09 20:46:51.857429 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:51.857276    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8dcbd9c0-e30e-44ac-997d-27e33b31371d-tls-certs\") pod \"model-serving-api-54d89bd5fc-8l6p8\" (UID: \"8dcbd9c0-e30e-44ac-997d-27e33b31371d\") " pod="opendatahub/model-serving-api-54d89bd5fc-8l6p8"
Sep 09 20:46:51.958028 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:51.957992    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8dcbd9c0-e30e-44ac-997d-27e33b31371d-tls-certs\") pod \"model-serving-api-54d89bd5fc-8l6p8\" (UID: \"8dcbd9c0-e30e-44ac-997d-27e33b31371d\") " pod="opendatahub/model-serving-api-54d89bd5fc-8l6p8"
Sep 09 20:46:51.958218 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:51.958089    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87cp8\" (UniqueName: \"kubernetes.io/projected/8dcbd9c0-e30e-44ac-997d-27e33b31371d-kube-api-access-87cp8\") pod \"model-serving-api-54d89bd5fc-8l6p8\" (UID: \"8dcbd9c0-e30e-44ac-997d-27e33b31371d\") " pod="opendatahub/model-serving-api-54d89bd5fc-8l6p8"
Sep 09 20:46:51.960756 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:51.960721    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8dcbd9c0-e30e-44ac-997d-27e33b31371d-tls-certs\") pod \"model-serving-api-54d89bd5fc-8l6p8\" (UID: \"8dcbd9c0-e30e-44ac-997d-27e33b31371d\") " pod="opendatahub/model-serving-api-54d89bd5fc-8l6p8"
Sep 09 20:46:51.990289 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:51.990249    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-87cp8\" (UniqueName: \"kubernetes.io/projected/8dcbd9c0-e30e-44ac-997d-27e33b31371d-kube-api-access-87cp8\") pod \"model-serving-api-54d89bd5fc-8l6p8\" (UID: \"8dcbd9c0-e30e-44ac-997d-27e33b31371d\") " pod="opendatahub/model-serving-api-54d89bd5fc-8l6p8"
Sep 09 20:46:52.070777 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:52.070675    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-8l6p8"
Sep 09 20:46:52.368658 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:52.368273    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-8l6p8"]
Sep 09 20:46:52.371318 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:46:52.371287    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8dcbd9c0_e30e_44ac_997d_27e33b31371d.slice/crio-1ed28ba31e1b04038dd57d36b4375b5a812e7a905b24ce24f0d9c2f6ae12f866 WatchSource:0}: Error finding container 1ed28ba31e1b04038dd57d36b4375b5a812e7a905b24ce24f0d9c2f6ae12f866: Status 404 returned error can't find the container with id 1ed28ba31e1b04038dd57d36b4375b5a812e7a905b24ce24f0d9c2f6ae12f866
Sep 09 20:46:52.556881 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:52.556836    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-6k5n7" event={"ID":"fd53ab69-cbaa-4965-9333-96549bbd2c70","Type":"ContainerStarted","Data":"5d2c89f881bfcf1780e04c769e982296fd5b44c5f1a08f6cca28b88cccb50d6b"}
Sep 09 20:46:52.558700 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:52.558638    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-8l6p8" event={"ID":"8dcbd9c0-e30e-44ac-997d-27e33b31371d","Type":"ContainerStarted","Data":"1ed28ba31e1b04038dd57d36b4375b5a812e7a905b24ce24f0d9c2f6ae12f866"}
Sep 09 20:46:54.297351 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:54.297313    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7"]
Sep 09 20:46:54.301775 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:54.301731    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7"
Sep 09 20:46:54.322836 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:54.322802    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 09 20:46:54.323056 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:54.323023    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-qml9s\""
Sep 09 20:46:54.382578 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:54.382542    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a96bea0e-3a19-4fbf-84b4-e06fb29de75e-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7\" (UID: \"a96bea0e-3a19-4fbf-84b4-e06fb29de75e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7"
Sep 09 20:46:54.382751 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:54.382586    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-928pp\" (UniqueName: \"kubernetes.io/projected/a96bea0e-3a19-4fbf-84b4-e06fb29de75e-kube-api-access-928pp\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7\" (UID: \"a96bea0e-3a19-4fbf-84b4-e06fb29de75e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7"
Sep 09 20:46:54.382751 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:54.382665    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a96bea0e-3a19-4fbf-84b4-e06fb29de75e-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7\" (UID: \"a96bea0e-3a19-4fbf-84b4-e06fb29de75e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7"
Sep 09 20:46:54.382751 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:54.382702    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a96bea0e-3a19-4fbf-84b4-e06fb29de75e-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7\" (UID: \"a96bea0e-3a19-4fbf-84b4-e06fb29de75e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7"
Sep 09 20:46:54.382751 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:54.382724    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a96bea0e-3a19-4fbf-84b4-e06fb29de75e-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7\" (UID: \"a96bea0e-3a19-4fbf-84b4-e06fb29de75e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7"
Sep 09 20:46:54.382953 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:54.382854    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a96bea0e-3a19-4fbf-84b4-e06fb29de75e-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7\" (UID: \"a96bea0e-3a19-4fbf-84b4-e06fb29de75e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7"
Sep 09 20:46:54.382953 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:54.382902    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a96bea0e-3a19-4fbf-84b4-e06fb29de75e-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7\" (UID: \"a96bea0e-3a19-4fbf-84b4-e06fb29de75e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7"
Sep 09 20:46:54.383029 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:54.382972    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a96bea0e-3a19-4fbf-84b4-e06fb29de75e-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7\" (UID: \"a96bea0e-3a19-4fbf-84b4-e06fb29de75e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7"
Sep 09 20:46:54.383029 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:54.383009    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a96bea0e-3a19-4fbf-84b4-e06fb29de75e-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7\" (UID: \"a96bea0e-3a19-4fbf-84b4-e06fb29de75e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7"
Sep 09 20:46:54.394235 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:54.394199    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7"]
Sep 09 20:46:54.483965 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:54.483898    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a96bea0e-3a19-4fbf-84b4-e06fb29de75e-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7\" (UID: \"a96bea0e-3a19-4fbf-84b4-e06fb29de75e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7"
Sep 09 20:46:54.484207 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:54.484041    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-928pp\" (UniqueName: \"kubernetes.io/projected/a96bea0e-3a19-4fbf-84b4-e06fb29de75e-kube-api-access-928pp\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7\" (UID: \"a96bea0e-3a19-4fbf-84b4-e06fb29de75e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7"
Sep 09 20:46:54.484207 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:54.484094    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a96bea0e-3a19-4fbf-84b4-e06fb29de75e-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7\" (UID: \"a96bea0e-3a19-4fbf-84b4-e06fb29de75e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7"
Sep 09 20:46:54.484318 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:54.484231    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a96bea0e-3a19-4fbf-84b4-e06fb29de75e-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7\" (UID: \"a96bea0e-3a19-4fbf-84b4-e06fb29de75e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7"
Sep 09 20:46:54.484318 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:54.484284    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a96bea0e-3a19-4fbf-84b4-e06fb29de75e-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7\" (UID: \"a96bea0e-3a19-4fbf-84b4-e06fb29de75e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7"
Sep 09 20:46:54.484383 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:54.484349    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a96bea0e-3a19-4fbf-84b4-e06fb29de75e-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7\" (UID: \"a96bea0e-3a19-4fbf-84b4-e06fb29de75e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7"
Sep 09 20:46:54.484383 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:54.484355    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a96bea0e-3a19-4fbf-84b4-e06fb29de75e-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7\" (UID: \"a96bea0e-3a19-4fbf-84b4-e06fb29de75e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7"
Sep 09 20:46:54.484450 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:54.484393    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a96bea0e-3a19-4fbf-84b4-e06fb29de75e-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7\" (UID: \"a96bea0e-3a19-4fbf-84b4-e06fb29de75e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7"
Sep 09 20:46:54.484450 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:54.484443    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a96bea0e-3a19-4fbf-84b4-e06fb29de75e-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7\" (UID: \"a96bea0e-3a19-4fbf-84b4-e06fb29de75e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7"
Sep 09 20:46:54.484511 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:54.484466    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a96bea0e-3a19-4fbf-84b4-e06fb29de75e-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7\" (UID: \"a96bea0e-3a19-4fbf-84b4-e06fb29de75e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7"
Sep 09 20:46:54.484511 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:54.484483    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a96bea0e-3a19-4fbf-84b4-e06fb29de75e-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7\" (UID: \"a96bea0e-3a19-4fbf-84b4-e06fb29de75e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7"
Sep 09 20:46:54.485013 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:54.484962    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a96bea0e-3a19-4fbf-84b4-e06fb29de75e-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7\" (UID: \"a96bea0e-3a19-4fbf-84b4-e06fb29de75e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7"
Sep 09 20:46:54.485013 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:54.484985    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a96bea0e-3a19-4fbf-84b4-e06fb29de75e-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7\" (UID: \"a96bea0e-3a19-4fbf-84b4-e06fb29de75e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7"
Sep 09 20:46:54.485220 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:54.485188    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a96bea0e-3a19-4fbf-84b4-e06fb29de75e-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7\" (UID: \"a96bea0e-3a19-4fbf-84b4-e06fb29de75e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7"
Sep 09 20:46:54.487335 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:54.487310    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a96bea0e-3a19-4fbf-84b4-e06fb29de75e-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7\" (UID: \"a96bea0e-3a19-4fbf-84b4-e06fb29de75e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7"
Sep 09 20:46:54.487538 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:54.487520    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a96bea0e-3a19-4fbf-84b4-e06fb29de75e-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7\" (UID: \"a96bea0e-3a19-4fbf-84b4-e06fb29de75e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7"
Sep 09 20:46:54.505468 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:54.505432    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a96bea0e-3a19-4fbf-84b4-e06fb29de75e-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7\" (UID: \"a96bea0e-3a19-4fbf-84b4-e06fb29de75e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7"
Sep 09 20:46:54.505604 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:54.505557    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-928pp\" (UniqueName: \"kubernetes.io/projected/a96bea0e-3a19-4fbf-84b4-e06fb29de75e-kube-api-access-928pp\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7\" (UID: \"a96bea0e-3a19-4fbf-84b4-e06fb29de75e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7"
Sep 09 20:46:54.616757 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:54.616668    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7"
Sep 09 20:46:55.754746 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:55.754713    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7"]
Sep 09 20:46:55.759168 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:46:55.759132    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda96bea0e_3a19_4fbf_84b4_e06fb29de75e.slice/crio-ba86d3422ac644c36c280a513668170ff2d0aa4cb595f4ef77b3b50e8fc861af WatchSource:0}: Error finding container ba86d3422ac644c36c280a513668170ff2d0aa4cb595f4ef77b3b50e8fc861af: Status 404 returned error can't find the container with id ba86d3422ac644c36c280a513668170ff2d0aa4cb595f4ef77b3b50e8fc861af
Sep 09 20:46:56.582259 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:56.582220    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7" event={"ID":"a96bea0e-3a19-4fbf-84b4-e06fb29de75e","Type":"ContainerStarted","Data":"ba86d3422ac644c36c280a513668170ff2d0aa4cb595f4ef77b3b50e8fc861af"}
Sep 09 20:46:56.584350 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:56.584296    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-6k5n7" event={"ID":"fd53ab69-cbaa-4965-9333-96549bbd2c70","Type":"ContainerStarted","Data":"a2637ec6cfedee60a952df796930a3d081610e2843fa919fde329ffb4cdb372a"}
Sep 09 20:46:56.584520 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:56.584363    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-6k5n7"
Sep 09 20:46:56.586298 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:56.586267    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-8l6p8" event={"ID":"8dcbd9c0-e30e-44ac-997d-27e33b31371d","Type":"ContainerStarted","Data":"7b97d3ac91cc5fae6a078a9a771b9fcf2e2c429bb325ad42b962113d085de2d9"}
Sep 09 20:46:56.586433 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:56.586417    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-8l6p8"
Sep 09 20:46:56.661569 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:56.661507    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-6k5n7" podStartSLOduration=2.779158371 podStartE2EDuration="6.661488683s" totalImagesPullingTime="3.882330312s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:46:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:46:56.660583241 +0000 UTC m=+378.288211114" watchObservedRunningTime="2026-09-09 20:46:56.661488683 +0000 UTC m=+378.289116556"
Sep 09 20:46:56.700651 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:56.700589    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-8l6p8" podStartSLOduration=2.566287535 podStartE2EDuration="5.700564534s" totalImagesPullingTime="3.134276999s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:46:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:46:56.699748883 +0000 UTC m=+378.327376760" watchObservedRunningTime="2026-09-09 20:46:56.700564534 +0000 UTC m=+378.328192423"
Sep 09 20:46:58.237920 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:58.237871    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:46:58.238237 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:58.237961    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:46:58.238237 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:58.238005    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:46:58.598013 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:58.597913    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7" event={"ID":"a96bea0e-3a19-4fbf-84b4-e06fb29de75e","Type":"ContainerStarted","Data":"2e5e26d3c461b389d7e09a3bd11470846a53932fb5afff073f911cecf09b46d3"}
Sep 09 20:46:58.617798 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:58.617755    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7"
Sep 09 20:46:58.652201 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:58.652088    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7" podStartSLOduration=2.175897192 podStartE2EDuration="4.652067973s" totalImagesPullingTime="2.476170781s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:46:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:46:58.651233496 +0000 UTC m=+380.278861369" watchObservedRunningTime="2026-09-09 20:46:58.652067973 +0000 UTC m=+380.279695846"
Sep 09 20:46:59.658157 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:46:59.658115    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7"
Sep 09 20:47:00.607174 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:47:00.607138    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7"
Sep 09 20:47:00.608261 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:47:00.608238    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7"
Sep 09 20:47:02.435936 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:47:02.435899    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-wpxt6"]
Sep 09 20:47:02.439378 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:47:02.439350    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wpxt6"
Sep 09 20:47:02.448485 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:47:02.448454    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 09 20:47:02.448719 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:47:02.448705    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-s9d4v\""
Sep 09 20:47:02.448771 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:47:02.448755    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 09 20:47:02.472425 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:47:02.472396    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 09 20:47:02.478680 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:47:02.478647    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-wpxt6"]
Sep 09 20:47:02.562041 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:47:02.561991    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/cc763fcc-7c3b-4187-a9fa-3763f70cb349-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-wpxt6\" (UID: \"cc763fcc-7c3b-4187-a9fa-3763f70cb349\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wpxt6"
Sep 09 20:47:02.562041 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:47:02.562045    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vgxh\" (UniqueName: \"kubernetes.io/projected/cc763fcc-7c3b-4187-a9fa-3763f70cb349-kube-api-access-2vgxh\") pod \"istiod-openshift-gateway-5677ccb49b-wpxt6\" (UID: \"cc763fcc-7c3b-4187-a9fa-3763f70cb349\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wpxt6"
Sep 09 20:47:02.562351 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:47:02.562179    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/cc763fcc-7c3b-4187-a9fa-3763f70cb349-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-wpxt6\" (UID: \"cc763fcc-7c3b-4187-a9fa-3763f70cb349\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wpxt6"
Sep 09 20:47:02.562351 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:47:02.562294    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc763fcc-7c3b-4187-a9fa-3763f70cb349-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-wpxt6\" (UID: \"cc763fcc-7c3b-4187-a9fa-3763f70cb349\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wpxt6"
Sep 09 20:47:02.562351 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:47:02.562321    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc763fcc-7c3b-4187-a9fa-3763f70cb349-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-wpxt6\" (UID: \"cc763fcc-7c3b-4187-a9fa-3763f70cb349\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wpxt6"
Sep 09 20:47:02.562495 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:47:02.562362    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/cc763fcc-7c3b-4187-a9fa-3763f70cb349-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-wpxt6\" (UID: \"cc763fcc-7c3b-4187-a9fa-3763f70cb349\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wpxt6"
Sep 09 20:47:02.562495 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:47:02.562394    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/cc763fcc-7c3b-4187-a9fa-3763f70cb349-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-wpxt6\" (UID: \"cc763fcc-7c3b-4187-a9fa-3763f70cb349\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wpxt6"
Sep 09 20:47:02.663452 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:47:02.663413    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/cc763fcc-7c3b-4187-a9fa-3763f70cb349-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-wpxt6\" (UID: \"cc763fcc-7c3b-4187-a9fa-3763f70cb349\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wpxt6"
Sep 09 20:47:02.663644 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:47:02.663461    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vgxh\" (UniqueName: \"kubernetes.io/projected/cc763fcc-7c3b-4187-a9fa-3763f70cb349-kube-api-access-2vgxh\") pod \"istiod-openshift-gateway-5677ccb49b-wpxt6\" (UID: \"cc763fcc-7c3b-4187-a9fa-3763f70cb349\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wpxt6"
Sep 09 20:47:02.663644 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:47:02.663506    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/cc763fcc-7c3b-4187-a9fa-3763f70cb349-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-wpxt6\" (UID: \"cc763fcc-7c3b-4187-a9fa-3763f70cb349\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wpxt6"
Sep 09 20:47:02.663644 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:47:02.663564    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc763fcc-7c3b-4187-a9fa-3763f70cb349-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-wpxt6\" (UID: \"cc763fcc-7c3b-4187-a9fa-3763f70cb349\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wpxt6"
Sep 09 20:47:02.663644 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:47:02.663586    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc763fcc-7c3b-4187-a9fa-3763f70cb349-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-wpxt6\" (UID: \"cc763fcc-7c3b-4187-a9fa-3763f70cb349\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wpxt6"
Sep 09 20:47:02.663644 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:47:02.663624    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/cc763fcc-7c3b-4187-a9fa-3763f70cb349-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-wpxt6\" (UID: \"cc763fcc-7c3b-4187-a9fa-3763f70cb349\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wpxt6"
Sep 09 20:47:02.663883 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:47:02.663653    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/cc763fcc-7c3b-4187-a9fa-3763f70cb349-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-wpxt6\" (UID: \"cc763fcc-7c3b-4187-a9fa-3763f70cb349\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wpxt6"
Sep 09 20:47:02.664547 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:47:02.664506    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/cc763fcc-7c3b-4187-a9fa-3763f70cb349-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-wpxt6\" (UID: \"cc763fcc-7c3b-4187-a9fa-3763f70cb349\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wpxt6"
Sep 09 20:47:02.666200 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:47:02.666172    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/cc763fcc-7c3b-4187-a9fa-3763f70cb349-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-wpxt6\" (UID: \"cc763fcc-7c3b-4187-a9fa-3763f70cb349\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wpxt6"
Sep 09 20:47:02.666200 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:47:02.666172    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/cc763fcc-7c3b-4187-a9fa-3763f70cb349-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-wpxt6\" (UID: \"cc763fcc-7c3b-4187-a9fa-3763f70cb349\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wpxt6"
Sep 09 20:47:02.666383 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:47:02.666311    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc763fcc-7c3b-4187-a9fa-3763f70cb349-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-wpxt6\" (UID: \"cc763fcc-7c3b-4187-a9fa-3763f70cb349\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wpxt6"
Sep 09 20:47:02.666383 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:47:02.666343    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/cc763fcc-7c3b-4187-a9fa-3763f70cb349-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-wpxt6\" (UID: \"cc763fcc-7c3b-4187-a9fa-3763f70cb349\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wpxt6"
Sep 09 20:47:02.677813 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:47:02.677781    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vgxh\" (UniqueName: \"kubernetes.io/projected/cc763fcc-7c3b-4187-a9fa-3763f70cb349-kube-api-access-2vgxh\") pod \"istiod-openshift-gateway-5677ccb49b-wpxt6\" (UID: \"cc763fcc-7c3b-4187-a9fa-3763f70cb349\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wpxt6"
Sep 09 20:47:02.680321 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:47:02.680295    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc763fcc-7c3b-4187-a9fa-3763f70cb349-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-wpxt6\" (UID: \"cc763fcc-7c3b-4187-a9fa-3763f70cb349\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wpxt6"
Sep 09 20:47:02.748872 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:47:02.748836    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wpxt6"
Sep 09 20:47:03.071858 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:47:03.071773    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-wpxt6"]
Sep 09 20:47:03.075250 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:47:03.075221    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc763fcc_7c3b_4187_a9fa_3763f70cb349.slice/crio-0b4ea8ff9e1cc9f9c52f009564830200beff498bb0e1e0ea00abf802e5b06fd2 WatchSource:0}: Error finding container 0b4ea8ff9e1cc9f9c52f009564830200beff498bb0e1e0ea00abf802e5b06fd2: Status 404 returned error can't find the container with id 0b4ea8ff9e1cc9f9c52f009564830200beff498bb0e1e0ea00abf802e5b06fd2
Sep 09 20:47:03.621957 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:47:03.621917    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wpxt6" event={"ID":"cc763fcc-7c3b-4187-a9fa-3763f70cb349","Type":"ContainerStarted","Data":"0b4ea8ff9e1cc9f9c52f009564830200beff498bb0e1e0ea00abf802e5b06fd2"}
Sep 09 20:47:05.400404 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:47:05.400361    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:47:05.400715 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:47:05.400431    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:47:05.633683 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:47:05.633641    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wpxt6" event={"ID":"cc763fcc-7c3b-4187-a9fa-3763f70cb349","Type":"ContainerStarted","Data":"1d0c89a6b92f0480a19551ef7c7c890cbc495b6da46fc4d68ae0c79a3f4ea960"}
Sep 09 20:47:05.633890 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:47:05.633778    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wpxt6"
Sep 09 20:47:06.640766 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:47:06.640729    2581 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-wpxt6 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 09 20:47:06.641163 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:47:06.640781    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wpxt6" podUID="cc763fcc-7c3b-4187-a9fa-3763f70cb349" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 20:47:07.594272 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:47:07.594237    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-6k5n7"
Sep 09 20:47:07.596530 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:47:07.596504    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-8l6p8"
Sep 09 20:47:07.716315 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:47:07.716257    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wpxt6" podStartSLOduration=3.393471169 podStartE2EDuration="5.716239026s" totalImagesPullingTime="2.322767857s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:47:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:47:05.688446165 +0000 UTC m=+387.316074053" watchObservedRunningTime="2026-09-09 20:47:07.716239026 +0000 UTC m=+389.343866898"
Sep 09 20:47:09.640466 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:47:09.640432    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wpxt6"
Sep 09 20:49:37.772643 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:49:37.772547    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-pkz5w"]
Sep 09 20:49:37.780087 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:49:37.780048    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-pkz5w"
Sep 09 20:49:37.784515 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:49:37.784489    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 09 20:49:37.784677 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:49:37.784496    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 09 20:49:37.786117 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:49:37.786074    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-qdbt7\""
Sep 09 20:49:37.786236 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:49:37.786074    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 09 20:49:37.792941 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:49:37.792920    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-pkz5w"]
Sep 09 20:49:37.880496 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:49:37.880460    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqv5n\" (UniqueName: \"kubernetes.io/projected/06ff6c9f-5dbb-4ca2-a82e-5c3681890936-kube-api-access-tqv5n\") pod \"postgres-69fb4594cf-pkz5w\" (UID: \"06ff6c9f-5dbb-4ca2-a82e-5c3681890936\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-pkz5w"
Sep 09 20:49:37.880496 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:49:37.880495    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/06ff6c9f-5dbb-4ca2-a82e-5c3681890936-data\") pod \"postgres-69fb4594cf-pkz5w\" (UID: \"06ff6c9f-5dbb-4ca2-a82e-5c3681890936\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-pkz5w"
Sep 09 20:49:37.981493 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:49:37.981455    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqv5n\" (UniqueName: \"kubernetes.io/projected/06ff6c9f-5dbb-4ca2-a82e-5c3681890936-kube-api-access-tqv5n\") pod \"postgres-69fb4594cf-pkz5w\" (UID: \"06ff6c9f-5dbb-4ca2-a82e-5c3681890936\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-pkz5w"
Sep 09 20:49:37.981493 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:49:37.981495    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/06ff6c9f-5dbb-4ca2-a82e-5c3681890936-data\") pod \"postgres-69fb4594cf-pkz5w\" (UID: \"06ff6c9f-5dbb-4ca2-a82e-5c3681890936\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-pkz5w"
Sep 09 20:49:37.981864 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:49:37.981848    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/06ff6c9f-5dbb-4ca2-a82e-5c3681890936-data\") pod \"postgres-69fb4594cf-pkz5w\" (UID: \"06ff6c9f-5dbb-4ca2-a82e-5c3681890936\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-pkz5w"
Sep 09 20:49:37.993078 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:49:37.993040    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqv5n\" (UniqueName: \"kubernetes.io/projected/06ff6c9f-5dbb-4ca2-a82e-5c3681890936-kube-api-access-tqv5n\") pod \"postgres-69fb4594cf-pkz5w\" (UID: \"06ff6c9f-5dbb-4ca2-a82e-5c3681890936\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-pkz5w"
Sep 09 20:49:38.093842 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:49:38.093744    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-pkz5w"
Sep 09 20:49:38.378777 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:49:38.378750    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-pkz5w"]
Sep 09 20:49:38.381515 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:49:38.381482    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06ff6c9f_5dbb_4ca2_a82e_5c3681890936.slice/crio-fe2c0ae973b1667f42111671cdbecee742d5150709422db043745e8631521cdd WatchSource:0}: Error finding container fe2c0ae973b1667f42111671cdbecee742d5150709422db043745e8631521cdd: Status 404 returned error can't find the container with id fe2c0ae973b1667f42111671cdbecee742d5150709422db043745e8631521cdd
Sep 09 20:49:39.369961 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:49:39.369920    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-pkz5w" event={"ID":"06ff6c9f-5dbb-4ca2-a82e-5c3681890936","Type":"ContainerStarted","Data":"fe2c0ae973b1667f42111671cdbecee742d5150709422db043745e8631521cdd"}
Sep 09 20:49:43.588979 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:49:43.588951    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 09 20:49:44.398452 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:49:44.398411    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-pkz5w" event={"ID":"06ff6c9f-5dbb-4ca2-a82e-5c3681890936","Type":"ContainerStarted","Data":"9d1b44a9dc75fe5d9db0f24fe255091a8487730e28e3d847ddabf494dec9acb9"}
Sep 09 20:49:44.398669 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:49:44.398595    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-pkz5w"
Sep 09 20:49:44.416576 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:49:44.416517    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-pkz5w" podStartSLOduration=2.213430565 podStartE2EDuration="7.416498742s" totalImagesPullingTime="5.203068177s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:49:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:49:44.414528793 +0000 UTC m=+546.042156668" watchObservedRunningTime="2026-09-09 20:49:44.416498742 +0000 UTC m=+546.044126615"
Sep 09 20:49:50.435154 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:49:50.435087    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-pkz5w"
Sep 09 20:50:38.783183 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:50:38.783148    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-wsv4v_2b1f9a6e-7a79-4e89-ac3f-4658d6d91550/console-operator/1.log"
Sep 09 20:50:38.784160 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:50:38.784142    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-wsv4v_2b1f9a6e-7a79-4e89-ac3f-4658d6d91550/console-operator/1.log"
Sep 09 20:50:38.790592 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:50:38.790560    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-75wj4_55b4fbae-29dc-41bf-a907-b1412b852cef/ovn-acl-logging/0.log"
Sep 09 20:50:38.791333 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:50:38.791311    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-75wj4_55b4fbae-29dc-41bf-a907-b1412b852cef/ovn-acl-logging/0.log"
Sep 09 20:51:19.634681 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:19.634636    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-2d7ms"]
Sep 09 20:51:19.638218 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:19.638193    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2d7ms"
Sep 09 20:51:19.640883 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:19.640851    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-szvm2\""
Sep 09 20:51:19.643892 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:19.643868    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 09 20:51:19.644495 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:19.644477    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 09 20:51:19.647937 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:19.647911    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 09 20:51:19.663941 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:19.663901    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-2d7ms"]
Sep 09 20:51:19.731064 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:19.731020    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5ef005c-d8f6-4d09-ac08-2fa7483c59e5-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2d7ms\" (UID: \"a5ef005c-d8f6-4d09-ac08-2fa7483c59e5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2d7ms"
Sep 09 20:51:19.731064 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:19.731067    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmktt\" (UniqueName: \"kubernetes.io/projected/a5ef005c-d8f6-4d09-ac08-2fa7483c59e5-kube-api-access-hmktt\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2d7ms\" (UID: \"a5ef005c-d8f6-4d09-ac08-2fa7483c59e5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2d7ms"
Sep 09 20:51:19.731293 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:19.731185    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5ef005c-d8f6-4d09-ac08-2fa7483c59e5-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2d7ms\" (UID: \"a5ef005c-d8f6-4d09-ac08-2fa7483c59e5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2d7ms"
Sep 09 20:51:19.731293 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:19.731279    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5ef005c-d8f6-4d09-ac08-2fa7483c59e5-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2d7ms\" (UID: \"a5ef005c-d8f6-4d09-ac08-2fa7483c59e5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2d7ms"
Sep 09 20:51:19.731364 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:19.731303    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5ef005c-d8f6-4d09-ac08-2fa7483c59e5-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2d7ms\" (UID: \"a5ef005c-d8f6-4d09-ac08-2fa7483c59e5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2d7ms"
Sep 09 20:51:19.731398 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:19.731366    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5ef005c-d8f6-4d09-ac08-2fa7483c59e5-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2d7ms\" (UID: \"a5ef005c-d8f6-4d09-ac08-2fa7483c59e5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2d7ms"
Sep 09 20:51:19.832316 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:19.832271    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5ef005c-d8f6-4d09-ac08-2fa7483c59e5-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2d7ms\" (UID: \"a5ef005c-d8f6-4d09-ac08-2fa7483c59e5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2d7ms"
Sep 09 20:51:19.832316 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:19.832320    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5ef005c-d8f6-4d09-ac08-2fa7483c59e5-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2d7ms\" (UID: \"a5ef005c-d8f6-4d09-ac08-2fa7483c59e5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2d7ms"
Sep 09 20:51:19.832552 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:19.832360    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5ef005c-d8f6-4d09-ac08-2fa7483c59e5-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2d7ms\" (UID: \"a5ef005c-d8f6-4d09-ac08-2fa7483c59e5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2d7ms"
Sep 09 20:51:19.832552 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:19.832483    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5ef005c-d8f6-4d09-ac08-2fa7483c59e5-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2d7ms\" (UID: \"a5ef005c-d8f6-4d09-ac08-2fa7483c59e5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2d7ms"
Sep 09 20:51:19.832552 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:19.832524    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmktt\" (UniqueName: \"kubernetes.io/projected/a5ef005c-d8f6-4d09-ac08-2fa7483c59e5-kube-api-access-hmktt\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2d7ms\" (UID: \"a5ef005c-d8f6-4d09-ac08-2fa7483c59e5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2d7ms"
Sep 09 20:51:19.832694 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:19.832598    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5ef005c-d8f6-4d09-ac08-2fa7483c59e5-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2d7ms\" (UID: \"a5ef005c-d8f6-4d09-ac08-2fa7483c59e5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2d7ms"
Sep 09 20:51:19.832837 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:19.832814    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5ef005c-d8f6-4d09-ac08-2fa7483c59e5-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2d7ms\" (UID: \"a5ef005c-d8f6-4d09-ac08-2fa7483c59e5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2d7ms"
Sep 09 20:51:19.832953 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:19.832851    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5ef005c-d8f6-4d09-ac08-2fa7483c59e5-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2d7ms\" (UID: \"a5ef005c-d8f6-4d09-ac08-2fa7483c59e5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2d7ms"
Sep 09 20:51:19.832953 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:19.832932    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5ef005c-d8f6-4d09-ac08-2fa7483c59e5-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2d7ms\" (UID: \"a5ef005c-d8f6-4d09-ac08-2fa7483c59e5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2d7ms"
Sep 09 20:51:19.834722 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:19.834702    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5ef005c-d8f6-4d09-ac08-2fa7483c59e5-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2d7ms\" (UID: \"a5ef005c-d8f6-4d09-ac08-2fa7483c59e5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2d7ms"
Sep 09 20:51:19.834960 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:19.834944    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5ef005c-d8f6-4d09-ac08-2fa7483c59e5-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2d7ms\" (UID: \"a5ef005c-d8f6-4d09-ac08-2fa7483c59e5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2d7ms"
Sep 09 20:51:19.843007 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:19.842984    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmktt\" (UniqueName: \"kubernetes.io/projected/a5ef005c-d8f6-4d09-ac08-2fa7483c59e5-kube-api-access-hmktt\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2d7ms\" (UID: \"a5ef005c-d8f6-4d09-ac08-2fa7483c59e5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2d7ms"
Sep 09 20:51:19.949358 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:19.949257    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2d7ms"
Sep 09 20:51:20.231565 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:20.231537    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-2d7ms"]
Sep 09 20:51:20.234938 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:51:20.234909    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5ef005c_d8f6_4d09_ac08_2fa7483c59e5.slice/crio-82db049a6b25e97a097a91b4ca87df1c0b878516d81805a33bfc7bef7e0561af WatchSource:0}: Error finding container 82db049a6b25e97a097a91b4ca87df1c0b878516d81805a33bfc7bef7e0561af: Status 404 returned error can't find the container with id 82db049a6b25e97a097a91b4ca87df1c0b878516d81805a33bfc7bef7e0561af
Sep 09 20:51:20.236448 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:20.236425    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 20:51:20.876191 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:20.876151    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2d7ms" event={"ID":"a5ef005c-d8f6-4d09-ac08-2fa7483c59e5","Type":"ContainerStarted","Data":"82db049a6b25e97a097a91b4ca87df1c0b878516d81805a33bfc7bef7e0561af"}
Sep 09 20:51:21.883253 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:21.883215    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2d7ms" event={"ID":"a5ef005c-d8f6-4d09-ac08-2fa7483c59e5","Type":"ContainerStarted","Data":"6cf4b3ff2c52301574b3b0a5639e2e72d23d8b371a0fa4843af8ca7938124295"}
Sep 09 20:51:21.904016 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:21.903960    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2d7ms" podStartSLOduration=1.916122055 podStartE2EDuration="2.903939769s" totalImagesPullingTime="987.817714ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:51:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:51:21.902993782 +0000 UTC m=+643.530621656" watchObservedRunningTime="2026-09-09 20:51:21.903939769 +0000 UTC m=+643.531567677"
Sep 09 20:51:29.949997 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:29.949953    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2d7ms"
Sep 09 20:51:29.950540 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:29.950036    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2d7ms"
Sep 09 20:51:29.965230 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:29.965190    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2d7ms"
Sep 09 20:51:30.937481 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:30.937445    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2d7ms"
Sep 09 20:51:50.400068 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:50.400029    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5rgbm"]
Sep 09 20:51:50.404313 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:50.404288    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5rgbm"
Sep 09 20:51:50.406959 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:50.406932    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Sep 09 20:51:50.415582 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:50.415543    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5rgbm"]
Sep 09 20:51:50.415964 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:50.415937    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3230d3ca-4f75-4a0d-92eb-7ba1b71cafaa-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5rgbm\" (UID: \"3230d3ca-4f75-4a0d-92eb-7ba1b71cafaa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5rgbm"
Sep 09 20:51:50.416060 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:50.415997    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3230d3ca-4f75-4a0d-92eb-7ba1b71cafaa-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5rgbm\" (UID: \"3230d3ca-4f75-4a0d-92eb-7ba1b71cafaa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5rgbm"
Sep 09 20:51:50.416060 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:50.416027    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrplt\" (UniqueName: \"kubernetes.io/projected/3230d3ca-4f75-4a0d-92eb-7ba1b71cafaa-kube-api-access-xrplt\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5rgbm\" (UID: \"3230d3ca-4f75-4a0d-92eb-7ba1b71cafaa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5rgbm"
Sep 09 20:51:50.416203 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:50.416133    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3230d3ca-4f75-4a0d-92eb-7ba1b71cafaa-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5rgbm\" (UID: \"3230d3ca-4f75-4a0d-92eb-7ba1b71cafaa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5rgbm"
Sep 09 20:51:50.416203 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:50.416153    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3230d3ca-4f75-4a0d-92eb-7ba1b71cafaa-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5rgbm\" (UID: \"3230d3ca-4f75-4a0d-92eb-7ba1b71cafaa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5rgbm"
Sep 09 20:51:50.416203 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:50.416191    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3230d3ca-4f75-4a0d-92eb-7ba1b71cafaa-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5rgbm\" (UID: \"3230d3ca-4f75-4a0d-92eb-7ba1b71cafaa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5rgbm"
Sep 09 20:51:50.516910 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:50.516863    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3230d3ca-4f75-4a0d-92eb-7ba1b71cafaa-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5rgbm\" (UID: \"3230d3ca-4f75-4a0d-92eb-7ba1b71cafaa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5rgbm"
Sep 09 20:51:50.516910 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:50.516911    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrplt\" (UniqueName: \"kubernetes.io/projected/3230d3ca-4f75-4a0d-92eb-7ba1b71cafaa-kube-api-access-xrplt\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5rgbm\" (UID: \"3230d3ca-4f75-4a0d-92eb-7ba1b71cafaa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5rgbm"
Sep 09 20:51:50.517217 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:50.516977    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3230d3ca-4f75-4a0d-92eb-7ba1b71cafaa-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5rgbm\" (UID: \"3230d3ca-4f75-4a0d-92eb-7ba1b71cafaa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5rgbm"
Sep 09 20:51:50.517217 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:50.516995    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3230d3ca-4f75-4a0d-92eb-7ba1b71cafaa-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5rgbm\" (UID: \"3230d3ca-4f75-4a0d-92eb-7ba1b71cafaa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5rgbm"
Sep 09 20:51:50.517217 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:50.517029    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3230d3ca-4f75-4a0d-92eb-7ba1b71cafaa-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5rgbm\" (UID: \"3230d3ca-4f75-4a0d-92eb-7ba1b71cafaa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5rgbm"
Sep 09 20:51:50.517217 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:50.517080    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3230d3ca-4f75-4a0d-92eb-7ba1b71cafaa-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5rgbm\" (UID: \"3230d3ca-4f75-4a0d-92eb-7ba1b71cafaa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5rgbm"
Sep 09 20:51:50.517443 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:50.517318    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3230d3ca-4f75-4a0d-92eb-7ba1b71cafaa-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5rgbm\" (UID: \"3230d3ca-4f75-4a0d-92eb-7ba1b71cafaa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5rgbm"
Sep 09 20:51:50.517443 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:50.517356    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3230d3ca-4f75-4a0d-92eb-7ba1b71cafaa-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5rgbm\" (UID: \"3230d3ca-4f75-4a0d-92eb-7ba1b71cafaa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5rgbm"
Sep 09 20:51:50.517552 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:50.517455    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3230d3ca-4f75-4a0d-92eb-7ba1b71cafaa-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5rgbm\" (UID: \"3230d3ca-4f75-4a0d-92eb-7ba1b71cafaa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5rgbm"
Sep 09 20:51:50.519385 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:50.519353    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3230d3ca-4f75-4a0d-92eb-7ba1b71cafaa-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5rgbm\" (UID: \"3230d3ca-4f75-4a0d-92eb-7ba1b71cafaa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5rgbm"
Sep 09 20:51:50.519744 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:50.519723    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3230d3ca-4f75-4a0d-92eb-7ba1b71cafaa-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5rgbm\" (UID: \"3230d3ca-4f75-4a0d-92eb-7ba1b71cafaa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5rgbm"
Sep 09 20:51:50.525551 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:50.525529    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrplt\" (UniqueName: \"kubernetes.io/projected/3230d3ca-4f75-4a0d-92eb-7ba1b71cafaa-kube-api-access-xrplt\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5rgbm\" (UID: \"3230d3ca-4f75-4a0d-92eb-7ba1b71cafaa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5rgbm"
Sep 09 20:51:50.716644 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:50.716537    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5rgbm"
Sep 09 20:51:50.994750 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:50.994711    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5rgbm"]
Sep 09 20:51:50.997935 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:51:50.997898    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3230d3ca_4f75_4a0d_92eb_7ba1b71cafaa.slice/crio-81284011c63a52507069918a756a192129c6ceff542739cb8423e6a4820c7505 WatchSource:0}: Error finding container 81284011c63a52507069918a756a192129c6ceff542739cb8423e6a4820c7505: Status 404 returned error can't find the container with id 81284011c63a52507069918a756a192129c6ceff542739cb8423e6a4820c7505
Sep 09 20:51:51.020677 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:51.020641    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5rgbm" event={"ID":"3230d3ca-4f75-4a0d-92eb-7ba1b71cafaa","Type":"ContainerStarted","Data":"81284011c63a52507069918a756a192129c6ceff542739cb8423e6a4820c7505"}
Sep 09 20:51:52.026533 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:52.026486    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5rgbm" event={"ID":"3230d3ca-4f75-4a0d-92eb-7ba1b71cafaa","Type":"ContainerStarted","Data":"5296ffc1efb0e050b6b404ad84f7e8b0b185eb781a0d18f803bea544086ce870"}
Sep 09 20:51:52.047537 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:52.047481    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5rgbm" podStartSLOduration=1.867809368 podStartE2EDuration="2.047460086s" totalImagesPullingTime="179.650718ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:51:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:51:52.045677702 +0000 UTC m=+673.673305575" watchObservedRunningTime="2026-09-09 20:51:52.047460086 +0000 UTC m=+673.675087959"
Sep 09 20:51:59.594340 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:59.594300    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-clx96"]
Sep 09 20:51:59.597059 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:59.597036    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clx96"
Sep 09 20:51:59.600009 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:59.599981    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Sep 09 20:51:59.610294 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:59.610264    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-clx96"]
Sep 09 20:51:59.697835 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:59.697796    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8afcba57-faa6-4184-91d0-a8b97d76f495-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-clx96\" (UID: \"8afcba57-faa6-4184-91d0-a8b97d76f495\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clx96"
Sep 09 20:51:59.698024 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:59.697841    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8afcba57-faa6-4184-91d0-a8b97d76f495-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-clx96\" (UID: \"8afcba57-faa6-4184-91d0-a8b97d76f495\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clx96"
Sep 09 20:51:59.698024 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:59.697916    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8afcba57-faa6-4184-91d0-a8b97d76f495-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-clx96\" (UID: \"8afcba57-faa6-4184-91d0-a8b97d76f495\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clx96"
Sep 09 20:51:59.698024 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:59.697978    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8afcba57-faa6-4184-91d0-a8b97d76f495-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-clx96\" (UID: \"8afcba57-faa6-4184-91d0-a8b97d76f495\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clx96"
Sep 09 20:51:59.698024 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:59.698013    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8afcba57-faa6-4184-91d0-a8b97d76f495-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-clx96\" (UID: \"8afcba57-faa6-4184-91d0-a8b97d76f495\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clx96"
Sep 09 20:51:59.698191 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:59.698036    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vkmz\" (UniqueName: \"kubernetes.io/projected/8afcba57-faa6-4184-91d0-a8b97d76f495-kube-api-access-9vkmz\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-clx96\" (UID: \"8afcba57-faa6-4184-91d0-a8b97d76f495\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clx96"
Sep 09 20:51:59.799032 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:59.798990    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8afcba57-faa6-4184-91d0-a8b97d76f495-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-clx96\" (UID: \"8afcba57-faa6-4184-91d0-a8b97d76f495\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clx96"
Sep 09 20:51:59.799032 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:59.799042    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8afcba57-faa6-4184-91d0-a8b97d76f495-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-clx96\" (UID: \"8afcba57-faa6-4184-91d0-a8b97d76f495\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clx96"
Sep 09 20:51:59.799282 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:59.799069    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8afcba57-faa6-4184-91d0-a8b97d76f495-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-clx96\" (UID: \"8afcba57-faa6-4184-91d0-a8b97d76f495\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clx96"
Sep 09 20:51:59.799282 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:59.799097    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vkmz\" (UniqueName: \"kubernetes.io/projected/8afcba57-faa6-4184-91d0-a8b97d76f495-kube-api-access-9vkmz\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-clx96\" (UID: \"8afcba57-faa6-4184-91d0-a8b97d76f495\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clx96"
Sep 09 20:51:59.799282 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:59.799199    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8afcba57-faa6-4184-91d0-a8b97d76f495-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-clx96\" (UID: \"8afcba57-faa6-4184-91d0-a8b97d76f495\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clx96"
Sep 09 20:51:59.799282 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:59.799234    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8afcba57-faa6-4184-91d0-a8b97d76f495-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-clx96\" (UID: \"8afcba57-faa6-4184-91d0-a8b97d76f495\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clx96"
Sep 09 20:51:59.799557 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:59.799522    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8afcba57-faa6-4184-91d0-a8b97d76f495-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-clx96\" (UID: \"8afcba57-faa6-4184-91d0-a8b97d76f495\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clx96"
Sep 09 20:51:59.799609 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:59.799568    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8afcba57-faa6-4184-91d0-a8b97d76f495-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-clx96\" (UID: \"8afcba57-faa6-4184-91d0-a8b97d76f495\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clx96"
Sep 09 20:51:59.799645 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:59.799623    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8afcba57-faa6-4184-91d0-a8b97d76f495-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-clx96\" (UID: \"8afcba57-faa6-4184-91d0-a8b97d76f495\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clx96"
Sep 09 20:51:59.801627 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:59.801594    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8afcba57-faa6-4184-91d0-a8b97d76f495-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-clx96\" (UID: \"8afcba57-faa6-4184-91d0-a8b97d76f495\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clx96"
Sep 09 20:51:59.801815 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:59.801797    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8afcba57-faa6-4184-91d0-a8b97d76f495-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-clx96\" (UID: \"8afcba57-faa6-4184-91d0-a8b97d76f495\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clx96"
Sep 09 20:51:59.809941 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:59.809909    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vkmz\" (UniqueName: \"kubernetes.io/projected/8afcba57-faa6-4184-91d0-a8b97d76f495-kube-api-access-9vkmz\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-clx96\" (UID: \"8afcba57-faa6-4184-91d0-a8b97d76f495\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clx96"
Sep 09 20:51:59.908694 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:51:59.908600    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clx96"
Sep 09 20:52:00.185194 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:52:00.185166    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-clx96"]
Sep 09 20:52:00.187493 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:52:00.187461    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8afcba57_faa6_4184_91d0_a8b97d76f495.slice/crio-d1febcfca9c292681e33387fa86a13e18cdc2083a6ab24134af01508f692ecf7 WatchSource:0}: Error finding container d1febcfca9c292681e33387fa86a13e18cdc2083a6ab24134af01508f692ecf7: Status 404 returned error can't find the container with id d1febcfca9c292681e33387fa86a13e18cdc2083a6ab24134af01508f692ecf7
Sep 09 20:52:00.717583 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:52:00.717484    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5rgbm"
Sep 09 20:52:00.717583 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:52:00.717550    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5rgbm"
Sep 09 20:52:00.730684 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:52:00.730650    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5rgbm"
Sep 09 20:52:01.072417 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:52:01.072377    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clx96" event={"ID":"8afcba57-faa6-4184-91d0-a8b97d76f495","Type":"ContainerStarted","Data":"a0fa0d8201994338396ee4e8a80cd7085e13d94a5484cf287272547827c959c0"}
Sep 09 20:52:01.072620 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:52:01.072439    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clx96" event={"ID":"8afcba57-faa6-4184-91d0-a8b97d76f495","Type":"ContainerStarted","Data":"d1febcfca9c292681e33387fa86a13e18cdc2083a6ab24134af01508f692ecf7"}
Sep 09 20:52:01.085461 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:52:01.085425    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5rgbm"
Sep 09 20:52:01.098077 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:52:01.098011    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clx96" podStartSLOduration=1.90143755 podStartE2EDuration="2.097991374s" totalImagesPullingTime="196.553824ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:51:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:52:01.096096134 +0000 UTC m=+682.723724009" watchObservedRunningTime="2026-09-09 20:52:01.097991374 +0000 UTC m=+682.725619253"
Sep 09 20:52:09.909559 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:52:09.909511    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clx96"
Sep 09 20:52:09.909977 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:52:09.909595    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clx96"
Sep 09 20:52:09.922504 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:52:09.922469    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clx96"
Sep 09 20:52:10.128629 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:52:10.128597    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clx96"
Sep 09 20:53:50.628198 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:50.628155    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd"]
Sep 09 20:53:50.631028 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:50.631004    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd"
Sep 09 20:53:50.635884 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:50.635861    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-d36b91-openshift-default-dockercfg-rq5bt\""
Sep 09 20:53:50.648032 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:50.648002    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd"]
Sep 09 20:53:50.711517 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:50.711480    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk"]
Sep 09 20:53:50.713875 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:50.713855    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk"
Sep 09 20:53:50.717912 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:50.717890    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-47e41c-openshift-default-dockercfg-lcsn5\""
Sep 09 20:53:50.728190 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:50.728162    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3dff37dd-436f-470c-9b0e-71ce7cd1d9f2-istio-envoy\") pod \"e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd\" (UID: \"3dff37dd-436f-470c-9b0e-71ce7cd1d9f2\") " pod="openshift-ingress/e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd"
Sep 09 20:53:50.728305 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:50.728224    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3dff37dd-436f-470c-9b0e-71ce7cd1d9f2-istio-token\") pod \"e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd\" (UID: \"3dff37dd-436f-470c-9b0e-71ce7cd1d9f2\") " pod="openshift-ingress/e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd"
Sep 09 20:53:50.728305 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:50.728281    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3dff37dd-436f-470c-9b0e-71ce7cd1d9f2-workload-certs\") pod \"e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd\" (UID: \"3dff37dd-436f-470c-9b0e-71ce7cd1d9f2\") " pod="openshift-ingress/e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd"
Sep 09 20:53:50.728413 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:50.728353    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3dff37dd-436f-470c-9b0e-71ce7cd1d9f2-istio-podinfo\") pod \"e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd\" (UID: \"3dff37dd-436f-470c-9b0e-71ce7cd1d9f2\") " pod="openshift-ingress/e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd"
Sep 09 20:53:50.728465 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:50.728434    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jxvb\" (UniqueName: \"kubernetes.io/projected/3dff37dd-436f-470c-9b0e-71ce7cd1d9f2-kube-api-access-8jxvb\") pod \"e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd\" (UID: \"3dff37dd-436f-470c-9b0e-71ce7cd1d9f2\") " pod="openshift-ingress/e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd"
Sep 09 20:53:50.728520 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:50.728465    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3dff37dd-436f-470c-9b0e-71ce7cd1d9f2-credential-socket\") pod \"e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd\" (UID: \"3dff37dd-436f-470c-9b0e-71ce7cd1d9f2\") " pod="openshift-ingress/e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd"
Sep 09 20:53:50.728828 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:50.728713    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3dff37dd-436f-470c-9b0e-71ce7cd1d9f2-workload-socket\") pod \"e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd\" (UID: \"3dff37dd-436f-470c-9b0e-71ce7cd1d9f2\") " pod="openshift-ingress/e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd"
Sep 09 20:53:50.728828 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:50.728764    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3dff37dd-436f-470c-9b0e-71ce7cd1d9f2-istiod-ca-cert\") pod \"e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd\" (UID: \"3dff37dd-436f-470c-9b0e-71ce7cd1d9f2\") " pod="openshift-ingress/e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd"
Sep 09 20:53:50.728828 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:50.728806    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3dff37dd-436f-470c-9b0e-71ce7cd1d9f2-istio-data\") pod \"e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd\" (UID: \"3dff37dd-436f-470c-9b0e-71ce7cd1d9f2\") " pod="openshift-ingress/e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd"
Sep 09 20:53:50.731038 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:50.731011    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk"]
Sep 09 20:53:50.829542 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:50.829506    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07f6f4a7-5d41-4db5-9c4a-eceaf6e96903-istio-data\") pod \"e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk\" (UID: \"07f6f4a7-5d41-4db5-9c4a-eceaf6e96903\") " pod="openshift-ingress/e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk"
Sep 09 20:53:50.829708 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:50.829564    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3dff37dd-436f-470c-9b0e-71ce7cd1d9f2-istio-envoy\") pod \"e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd\" (UID: \"3dff37dd-436f-470c-9b0e-71ce7cd1d9f2\") " pod="openshift-ingress/e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd"
Sep 09 20:53:50.829708 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:50.829625    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3dff37dd-436f-470c-9b0e-71ce7cd1d9f2-istio-token\") pod \"e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd\" (UID: \"3dff37dd-436f-470c-9b0e-71ce7cd1d9f2\") " pod="openshift-ingress/e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd"
Sep 09 20:53:50.829708 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:50.829648    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07f6f4a7-5d41-4db5-9c4a-eceaf6e96903-istio-envoy\") pod \"e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk\" (UID: \"07f6f4a7-5d41-4db5-9c4a-eceaf6e96903\") " pod="openshift-ingress/e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk"
Sep 09 20:53:50.829708 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:50.829684    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07f6f4a7-5d41-4db5-9c4a-eceaf6e96903-workload-certs\") pod \"e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk\" (UID: \"07f6f4a7-5d41-4db5-9c4a-eceaf6e96903\") " pod="openshift-ingress/e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk"
Sep 09 20:53:50.829915 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:50.829708    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07f6f4a7-5d41-4db5-9c4a-eceaf6e96903-istio-token\") pod \"e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk\" (UID: \"07f6f4a7-5d41-4db5-9c4a-eceaf6e96903\") " pod="openshift-ingress/e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk"
Sep 09 20:53:50.829915 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:50.829762    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkcvr\" (UniqueName: \"kubernetes.io/projected/07f6f4a7-5d41-4db5-9c4a-eceaf6e96903-kube-api-access-bkcvr\") pod \"e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk\" (UID: \"07f6f4a7-5d41-4db5-9c4a-eceaf6e96903\") " pod="openshift-ingress/e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk"
Sep 09 20:53:50.829915 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:50.829825    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3dff37dd-436f-470c-9b0e-71ce7cd1d9f2-workload-certs\") pod \"e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd\" (UID: \"3dff37dd-436f-470c-9b0e-71ce7cd1d9f2\") " pod="openshift-ingress/e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd"
Sep 09 20:53:50.829915 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:50.829862    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3dff37dd-436f-470c-9b0e-71ce7cd1d9f2-istio-podinfo\") pod \"e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd\" (UID: \"3dff37dd-436f-470c-9b0e-71ce7cd1d9f2\") " pod="openshift-ingress/e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd"
Sep 09 20:53:50.829915 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:50.829895    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07f6f4a7-5d41-4db5-9c4a-eceaf6e96903-istiod-ca-cert\") pod \"e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk\" (UID: \"07f6f4a7-5d41-4db5-9c4a-eceaf6e96903\") " pod="openshift-ingress/e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk"
Sep 09 20:53:50.830191 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:50.829948    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jxvb\" (UniqueName: \"kubernetes.io/projected/3dff37dd-436f-470c-9b0e-71ce7cd1d9f2-kube-api-access-8jxvb\") pod \"e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd\" (UID: \"3dff37dd-436f-470c-9b0e-71ce7cd1d9f2\") " pod="openshift-ingress/e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd"
Sep 09 20:53:50.830191 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:50.830008    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3dff37dd-436f-470c-9b0e-71ce7cd1d9f2-credential-socket\") pod \"e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd\" (UID: \"3dff37dd-436f-470c-9b0e-71ce7cd1d9f2\") " pod="openshift-ingress/e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd"
Sep 09 20:53:50.830191 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:50.830048    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07f6f4a7-5d41-4db5-9c4a-eceaf6e96903-credential-socket\") pod \"e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk\" (UID: \"07f6f4a7-5d41-4db5-9c4a-eceaf6e96903\") " pod="openshift-ingress/e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk"
Sep 09 20:53:50.830191 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:50.830095    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3dff37dd-436f-470c-9b0e-71ce7cd1d9f2-workload-socket\") pod \"e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd\" (UID: \"3dff37dd-436f-470c-9b0e-71ce7cd1d9f2\") " pod="openshift-ingress/e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd"
Sep 09 20:53:50.830191 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:50.830165    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07f6f4a7-5d41-4db5-9c4a-eceaf6e96903-workload-socket\") pod \"e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk\" (UID: \"07f6f4a7-5d41-4db5-9c4a-eceaf6e96903\") " pod="openshift-ingress/e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk"
Sep 09 20:53:50.830428 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:50.830203    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3dff37dd-436f-470c-9b0e-71ce7cd1d9f2-workload-certs\") pod \"e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd\" (UID: \"3dff37dd-436f-470c-9b0e-71ce7cd1d9f2\") " pod="openshift-ingress/e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd"
Sep 09 20:53:50.830428 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:50.830202    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07f6f4a7-5d41-4db5-9c4a-eceaf6e96903-istio-podinfo\") pod \"e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk\" (UID: \"07f6f4a7-5d41-4db5-9c4a-eceaf6e96903\") " pod="openshift-ingress/e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk"
Sep 09 20:53:50.830428 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:50.830268    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3dff37dd-436f-470c-9b0e-71ce7cd1d9f2-istiod-ca-cert\") pod \"e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd\" (UID: \"3dff37dd-436f-470c-9b0e-71ce7cd1d9f2\") " pod="openshift-ingress/e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd"
Sep 09 20:53:50.830428 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:50.830325    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3dff37dd-436f-470c-9b0e-71ce7cd1d9f2-istio-data\") pod \"e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd\" (UID: \"3dff37dd-436f-470c-9b0e-71ce7cd1d9f2\") " pod="openshift-ingress/e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd"
Sep 09 20:53:50.830428 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:50.830352    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3dff37dd-436f-470c-9b0e-71ce7cd1d9f2-credential-socket\") pod \"e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd\" (UID: \"3dff37dd-436f-470c-9b0e-71ce7cd1d9f2\") " pod="openshift-ingress/e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd"
Sep 09 20:53:50.830678 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:50.830461    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3dff37dd-436f-470c-9b0e-71ce7cd1d9f2-workload-socket\") pod \"e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd\" (UID: \"3dff37dd-436f-470c-9b0e-71ce7cd1d9f2\") " pod="openshift-ingress/e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd"
Sep 09 20:53:50.830678 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:50.830655    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3dff37dd-436f-470c-9b0e-71ce7cd1d9f2-istio-data\") pod \"e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd\" (UID: \"3dff37dd-436f-470c-9b0e-71ce7cd1d9f2\") " pod="openshift-ingress/e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd"
Sep 09 20:53:50.830844 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:50.830825    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3dff37dd-436f-470c-9b0e-71ce7cd1d9f2-istiod-ca-cert\") pod \"e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd\" (UID: \"3dff37dd-436f-470c-9b0e-71ce7cd1d9f2\") " pod="openshift-ingress/e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd"
Sep 09 20:53:50.832357 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:50.832332    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3dff37dd-436f-470c-9b0e-71ce7cd1d9f2-istio-envoy\") pod \"e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd\" (UID: \"3dff37dd-436f-470c-9b0e-71ce7cd1d9f2\") " pod="openshift-ingress/e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd"
Sep 09 20:53:50.832649 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:50.832627    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3dff37dd-436f-470c-9b0e-71ce7cd1d9f2-istio-podinfo\") pod \"e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd\" (UID: \"3dff37dd-436f-470c-9b0e-71ce7cd1d9f2\") " pod="openshift-ingress/e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd"
Sep 09 20:53:50.841852 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:50.841826    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3dff37dd-436f-470c-9b0e-71ce7cd1d9f2-istio-token\") pod \"e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd\" (UID: \"3dff37dd-436f-470c-9b0e-71ce7cd1d9f2\") " pod="openshift-ingress/e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd"
Sep 09 20:53:50.842175 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:50.842154    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jxvb\" (UniqueName: \"kubernetes.io/projected/3dff37dd-436f-470c-9b0e-71ce7cd1d9f2-kube-api-access-8jxvb\") pod \"e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd\" (UID: \"3dff37dd-436f-470c-9b0e-71ce7cd1d9f2\") " pod="openshift-ingress/e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd"
Sep 09 20:53:50.931483 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:50.931394    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07f6f4a7-5d41-4db5-9c4a-eceaf6e96903-credential-socket\") pod \"e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk\" (UID: \"07f6f4a7-5d41-4db5-9c4a-eceaf6e96903\") " pod="openshift-ingress/e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk"
Sep 09 20:53:50.931483 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:50.931441    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07f6f4a7-5d41-4db5-9c4a-eceaf6e96903-workload-socket\") pod \"e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk\" (UID: \"07f6f4a7-5d41-4db5-9c4a-eceaf6e96903\") " pod="openshift-ingress/e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk"
Sep 09 20:53:50.931483 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:50.931465    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07f6f4a7-5d41-4db5-9c4a-eceaf6e96903-istio-podinfo\") pod \"e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk\" (UID: \"07f6f4a7-5d41-4db5-9c4a-eceaf6e96903\") " pod="openshift-ingress/e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk"
Sep 09 20:53:50.931762 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:50.931541    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07f6f4a7-5d41-4db5-9c4a-eceaf6e96903-istio-data\") pod \"e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk\" (UID: \"07f6f4a7-5d41-4db5-9c4a-eceaf6e96903\") " pod="openshift-ingress/e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk"
Sep 09 20:53:50.931762 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:50.931599    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07f6f4a7-5d41-4db5-9c4a-eceaf6e96903-istio-envoy\") pod \"e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk\" (UID: \"07f6f4a7-5d41-4db5-9c4a-eceaf6e96903\") " pod="openshift-ingress/e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk"
Sep 09 20:53:50.931762 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:50.931642    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07f6f4a7-5d41-4db5-9c4a-eceaf6e96903-workload-certs\") pod \"e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk\" (UID: \"07f6f4a7-5d41-4db5-9c4a-eceaf6e96903\") " pod="openshift-ingress/e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk"
Sep 09 20:53:50.931762 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:50.931663    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07f6f4a7-5d41-4db5-9c4a-eceaf6e96903-istio-token\") pod \"e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk\" (UID: \"07f6f4a7-5d41-4db5-9c4a-eceaf6e96903\") " pod="openshift-ingress/e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk"
Sep 09 20:53:50.931762 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:50.931692    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkcvr\" (UniqueName: \"kubernetes.io/projected/07f6f4a7-5d41-4db5-9c4a-eceaf6e96903-kube-api-access-bkcvr\") pod \"e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk\" (UID: \"07f6f4a7-5d41-4db5-9c4a-eceaf6e96903\") " pod="openshift-ingress/e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk"
Sep 09 20:53:50.931762 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:50.931743    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07f6f4a7-5d41-4db5-9c4a-eceaf6e96903-istiod-ca-cert\") pod \"e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk\" (UID: \"07f6f4a7-5d41-4db5-9c4a-eceaf6e96903\") " pod="openshift-ingress/e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk"
Sep 09 20:53:50.932057 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:50.931831    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07f6f4a7-5d41-4db5-9c4a-eceaf6e96903-workload-socket\") pod \"e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk\" (UID: \"07f6f4a7-5d41-4db5-9c4a-eceaf6e96903\") " pod="openshift-ingress/e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk"
Sep 09 20:53:50.932057 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:50.931868    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07f6f4a7-5d41-4db5-9c4a-eceaf6e96903-credential-socket\") pod \"e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk\" (UID: \"07f6f4a7-5d41-4db5-9c4a-eceaf6e96903\") " pod="openshift-ingress/e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk"
Sep 09 20:53:50.932057 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:50.931890    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07f6f4a7-5d41-4db5-9c4a-eceaf6e96903-istio-data\") pod \"e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk\" (UID: \"07f6f4a7-5d41-4db5-9c4a-eceaf6e96903\") " pod="openshift-ingress/e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk"
Sep 09 20:53:50.932259 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:50.932190    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07f6f4a7-5d41-4db5-9c4a-eceaf6e96903-workload-certs\") pod \"e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk\" (UID: \"07f6f4a7-5d41-4db5-9c4a-eceaf6e96903\") " pod="openshift-ingress/e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk"
Sep 09 20:53:50.932477 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:50.932453    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07f6f4a7-5d41-4db5-9c4a-eceaf6e96903-istiod-ca-cert\") pod \"e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk\" (UID: \"07f6f4a7-5d41-4db5-9c4a-eceaf6e96903\") " pod="openshift-ingress/e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk"
Sep 09 20:53:50.934068 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:50.934038    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07f6f4a7-5d41-4db5-9c4a-eceaf6e96903-istio-envoy\") pod \"e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk\" (UID: \"07f6f4a7-5d41-4db5-9c4a-eceaf6e96903\") " pod="openshift-ingress/e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk"
Sep 09 20:53:50.934334 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:50.934312    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07f6f4a7-5d41-4db5-9c4a-eceaf6e96903-istio-podinfo\") pod \"e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk\" (UID: \"07f6f4a7-5d41-4db5-9c4a-eceaf6e96903\") " pod="openshift-ingress/e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk"
Sep 09 20:53:50.941697 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:50.941669    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07f6f4a7-5d41-4db5-9c4a-eceaf6e96903-istio-token\") pod \"e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk\" (UID: \"07f6f4a7-5d41-4db5-9c4a-eceaf6e96903\") " pod="openshift-ingress/e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk"
Sep 09 20:53:50.941852 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:50.941835    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd"
Sep 09 20:53:50.942068 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:50.942047    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkcvr\" (UniqueName: \"kubernetes.io/projected/07f6f4a7-5d41-4db5-9c4a-eceaf6e96903-kube-api-access-bkcvr\") pod \"e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk\" (UID: \"07f6f4a7-5d41-4db5-9c4a-eceaf6e96903\") " pod="openshift-ingress/e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk"
Sep 09 20:53:51.025604 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:51.025560    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk"
Sep 09 20:53:51.219739 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:51.219656    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd"]
Sep 09 20:53:51.222624 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:53:51.222592    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3dff37dd_436f_470c_9b0e_71ce7cd1d9f2.slice/crio-708a43949ce14176947444d6165891ecb544ec4d74903a9119b0e67d52ef4cb3 WatchSource:0}: Error finding container 708a43949ce14176947444d6165891ecb544ec4d74903a9119b0e67d52ef4cb3: Status 404 returned error can't find the container with id 708a43949ce14176947444d6165891ecb544ec4d74903a9119b0e67d52ef4cb3
Sep 09 20:53:51.224749 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:51.224714    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:53:51.224816 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:51.224785    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:53:51.224856 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:51.224813    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:53:51.319692 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:51.319657    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk"]
Sep 09 20:53:51.320737 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:53:51.320696    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07f6f4a7_5d41_4db5_9c4a_eceaf6e96903.slice/crio-9a03ad214b0007b089a670bb18ea5b8f3551336e6bc46388de39014779ab95ed WatchSource:0}: Error finding container 9a03ad214b0007b089a670bb18ea5b8f3551336e6bc46388de39014779ab95ed: Status 404 returned error can't find the container with id 9a03ad214b0007b089a670bb18ea5b8f3551336e6bc46388de39014779ab95ed
Sep 09 20:53:51.323351 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:51.323310    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:53:51.323445 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:51.323395    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:53:51.323445 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:51.323437    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:53:51.611737 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:51.611687    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd" event={"ID":"3dff37dd-436f-470c-9b0e-71ce7cd1d9f2","Type":"ContainerStarted","Data":"cce5895e33621c95c8291dab949ab8ace88c6c68e91bbea855fe67bfb2ec24a4"}
Sep 09 20:53:51.611940 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:51.611827    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd" event={"ID":"3dff37dd-436f-470c-9b0e-71ce7cd1d9f2","Type":"ContainerStarted","Data":"708a43949ce14176947444d6165891ecb544ec4d74903a9119b0e67d52ef4cb3"}
Sep 09 20:53:51.613680 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:51.613655    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk" event={"ID":"07f6f4a7-5d41-4db5-9c4a-eceaf6e96903","Type":"ContainerStarted","Data":"3387a184712e50dcad798672842de44a6fef191ddf2ece416e7abcb1cc6d7cc7"}
Sep 09 20:53:51.613794 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:51.613701    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk" event={"ID":"07f6f4a7-5d41-4db5-9c4a-eceaf6e96903","Type":"ContainerStarted","Data":"9a03ad214b0007b089a670bb18ea5b8f3551336e6bc46388de39014779ab95ed"}
Sep 09 20:53:51.645401 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:51.645332    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd" podStartSLOduration=1.645309616 podStartE2EDuration="1.645309616s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:53:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:53:51.64516509 +0000 UTC m=+793.272792963" watchObservedRunningTime="2026-09-09 20:53:51.645309616 +0000 UTC m=+793.272937491"
Sep 09 20:53:51.675719 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:51.675661    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk" podStartSLOduration=1.67564096 podStartE2EDuration="1.67564096s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:53:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:53:51.674799875 +0000 UTC m=+793.302427761" watchObservedRunningTime="2026-09-09 20:53:51.67564096 +0000 UTC m=+793.303268834"
Sep 09 20:53:51.942325 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:51.942239    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd"
Sep 09 20:53:52.025847 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:52.025811    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk"
Sep 09 20:53:52.030968 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:52.030942    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk"
Sep 09 20:53:52.618580 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:52.618550    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk"
Sep 09 20:53:52.619576 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:52.619557    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk"
Sep 09 20:53:52.947816 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:52.947734    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd"
Sep 09 20:53:53.624568 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:53.624537    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd"
Sep 09 20:53:53.625577 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:53:53.625560    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd"
Sep 09 20:54:00.774348 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:00.774316    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss"]
Sep 09 20:54:00.777602 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:00.777584    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss"
Sep 09 20:54:00.784743 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:00.784647    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-autores-864e4e-openshift-default-dockercfg-s9g2g\""
Sep 09 20:54:00.932846 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:00.932762    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92d54f63-73ce-4d1f-a0df-c0176265fb0f-istio-podinfo\") pod \"e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss\" (UID: \"92d54f63-73ce-4d1f-a0df-c0176265fb0f\") " pod="openshift-ingress/e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss"
Sep 09 20:54:00.932846 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:00.932805    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92d54f63-73ce-4d1f-a0df-c0176265fb0f-istiod-ca-cert\") pod \"e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss\" (UID: \"92d54f63-73ce-4d1f-a0df-c0176265fb0f\") " pod="openshift-ingress/e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss"
Sep 09 20:54:00.932846 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:00.932839    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26vmz\" (UniqueName: \"kubernetes.io/projected/92d54f63-73ce-4d1f-a0df-c0176265fb0f-kube-api-access-26vmz\") pod \"e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss\" (UID: \"92d54f63-73ce-4d1f-a0df-c0176265fb0f\") " pod="openshift-ingress/e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss"
Sep 09 20:54:00.933154 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:00.932867    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92d54f63-73ce-4d1f-a0df-c0176265fb0f-istio-data\") pod \"e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss\" (UID: \"92d54f63-73ce-4d1f-a0df-c0176265fb0f\") " pod="openshift-ingress/e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss"
Sep 09 20:54:00.933154 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:00.932899    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92d54f63-73ce-4d1f-a0df-c0176265fb0f-istio-envoy\") pod \"e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss\" (UID: \"92d54f63-73ce-4d1f-a0df-c0176265fb0f\") " pod="openshift-ingress/e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss"
Sep 09 20:54:00.933154 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:00.932928    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92d54f63-73ce-4d1f-a0df-c0176265fb0f-workload-socket\") pod \"e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss\" (UID: \"92d54f63-73ce-4d1f-a0df-c0176265fb0f\") " pod="openshift-ingress/e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss"
Sep 09 20:54:00.933154 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:00.933008    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92d54f63-73ce-4d1f-a0df-c0176265fb0f-istio-token\") pod \"e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss\" (UID: \"92d54f63-73ce-4d1f-a0df-c0176265fb0f\") " pod="openshift-ingress/e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss"
Sep 09 20:54:00.933154 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:00.933066    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92d54f63-73ce-4d1f-a0df-c0176265fb0f-credential-socket\") pod \"e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss\" (UID: \"92d54f63-73ce-4d1f-a0df-c0176265fb0f\") " pod="openshift-ingress/e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss"
Sep 09 20:54:00.933154 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:00.933092    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92d54f63-73ce-4d1f-a0df-c0176265fb0f-workload-certs\") pod \"e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss\" (UID: \"92d54f63-73ce-4d1f-a0df-c0176265fb0f\") " pod="openshift-ingress/e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss"
Sep 09 20:54:00.982535 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:00.982497    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss"]
Sep 09 20:54:01.034231 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:01.034139    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92d54f63-73ce-4d1f-a0df-c0176265fb0f-credential-socket\") pod \"e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss\" (UID: \"92d54f63-73ce-4d1f-a0df-c0176265fb0f\") " pod="openshift-ingress/e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss"
Sep 09 20:54:01.034231 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:01.034174    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92d54f63-73ce-4d1f-a0df-c0176265fb0f-workload-certs\") pod \"e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss\" (UID: \"92d54f63-73ce-4d1f-a0df-c0176265fb0f\") " pod="openshift-ingress/e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss"
Sep 09 20:54:01.034231 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:01.034225    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92d54f63-73ce-4d1f-a0df-c0176265fb0f-istio-podinfo\") pod \"e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss\" (UID: \"92d54f63-73ce-4d1f-a0df-c0176265fb0f\") " pod="openshift-ingress/e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss"
Sep 09 20:54:01.034593 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:01.034251    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92d54f63-73ce-4d1f-a0df-c0176265fb0f-istiod-ca-cert\") pod \"e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss\" (UID: \"92d54f63-73ce-4d1f-a0df-c0176265fb0f\") " pod="openshift-ingress/e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss"
Sep 09 20:54:01.034593 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:01.034278    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26vmz\" (UniqueName: \"kubernetes.io/projected/92d54f63-73ce-4d1f-a0df-c0176265fb0f-kube-api-access-26vmz\") pod \"e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss\" (UID: \"92d54f63-73ce-4d1f-a0df-c0176265fb0f\") " pod="openshift-ingress/e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss"
Sep 09 20:54:01.034593 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:01.034306    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92d54f63-73ce-4d1f-a0df-c0176265fb0f-istio-data\") pod \"e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss\" (UID: \"92d54f63-73ce-4d1f-a0df-c0176265fb0f\") " pod="openshift-ingress/e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss"
Sep 09 20:54:01.034593 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:01.034350    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92d54f63-73ce-4d1f-a0df-c0176265fb0f-istio-envoy\") pod \"e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss\" (UID: \"92d54f63-73ce-4d1f-a0df-c0176265fb0f\") " pod="openshift-ingress/e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss"
Sep 09 20:54:01.034593 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:01.034389    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92d54f63-73ce-4d1f-a0df-c0176265fb0f-workload-socket\") pod \"e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss\" (UID: \"92d54f63-73ce-4d1f-a0df-c0176265fb0f\") " pod="openshift-ingress/e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss"
Sep 09 20:54:01.034593 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:01.034458    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92d54f63-73ce-4d1f-a0df-c0176265fb0f-istio-token\") pod \"e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss\" (UID: \"92d54f63-73ce-4d1f-a0df-c0176265fb0f\") " pod="openshift-ingress/e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss"
Sep 09 20:54:01.034867 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:01.034684    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92d54f63-73ce-4d1f-a0df-c0176265fb0f-credential-socket\") pod \"e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss\" (UID: \"92d54f63-73ce-4d1f-a0df-c0176265fb0f\") " pod="openshift-ingress/e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss"
Sep 09 20:54:01.034926 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:01.034867    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92d54f63-73ce-4d1f-a0df-c0176265fb0f-istio-data\") pod \"e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss\" (UID: \"92d54f63-73ce-4d1f-a0df-c0176265fb0f\") " pod="openshift-ingress/e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss"
Sep 09 20:54:01.034979 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:01.034963    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92d54f63-73ce-4d1f-a0df-c0176265fb0f-workload-certs\") pod \"e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss\" (UID: \"92d54f63-73ce-4d1f-a0df-c0176265fb0f\") " pod="openshift-ingress/e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss"
Sep 09 20:54:01.035166 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:01.035138    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92d54f63-73ce-4d1f-a0df-c0176265fb0f-workload-socket\") pod \"e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss\" (UID: \"92d54f63-73ce-4d1f-a0df-c0176265fb0f\") " pod="openshift-ingress/e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss"
Sep 09 20:54:01.035252 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:01.035209    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92d54f63-73ce-4d1f-a0df-c0176265fb0f-istiod-ca-cert\") pod \"e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss\" (UID: \"92d54f63-73ce-4d1f-a0df-c0176265fb0f\") " pod="openshift-ingress/e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss"
Sep 09 20:54:01.036861 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:01.036840    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92d54f63-73ce-4d1f-a0df-c0176265fb0f-istio-podinfo\") pod \"e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss\" (UID: \"92d54f63-73ce-4d1f-a0df-c0176265fb0f\") " pod="openshift-ingress/e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss"
Sep 09 20:54:01.036964 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:01.036887    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92d54f63-73ce-4d1f-a0df-c0176265fb0f-istio-envoy\") pod \"e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss\" (UID: \"92d54f63-73ce-4d1f-a0df-c0176265fb0f\") " pod="openshift-ingress/e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss"
Sep 09 20:54:01.045713 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:01.045691    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-26vmz\" (UniqueName: \"kubernetes.io/projected/92d54f63-73ce-4d1f-a0df-c0176265fb0f-kube-api-access-26vmz\") pod \"e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss\" (UID: \"92d54f63-73ce-4d1f-a0df-c0176265fb0f\") " pod="openshift-ingress/e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss"
Sep 09 20:54:01.049192 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:01.049168    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92d54f63-73ce-4d1f-a0df-c0176265fb0f-istio-token\") pod \"e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss\" (UID: \"92d54f63-73ce-4d1f-a0df-c0176265fb0f\") " pod="openshift-ingress/e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss"
Sep 09 20:54:01.088519 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:01.088496    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss"
Sep 09 20:54:01.397907 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:01.397881    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss"]
Sep 09 20:54:01.400597 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:54:01.400568    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92d54f63_73ce_4d1f_a0df_c0176265fb0f.slice/crio-5c0e1d20867b5c3cb14a9c3523df2eb52b487ff7f149afb35b792a27c5331acf WatchSource:0}: Error finding container 5c0e1d20867b5c3cb14a9c3523df2eb52b487ff7f149afb35b792a27c5331acf: Status 404 returned error can't find the container with id 5c0e1d20867b5c3cb14a9c3523df2eb52b487ff7f149afb35b792a27c5331acf
Sep 09 20:54:01.402885 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:01.402851    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:54:01.402944 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:01.402923    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:54:01.402980 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:01.402957    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:54:01.665541 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:01.665450    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss" event={"ID":"92d54f63-73ce-4d1f-a0df-c0176265fb0f","Type":"ContainerStarted","Data":"161ac86d91fcab7e0c69dbe55bbfc7fe88717654d867b3e6c97699cd72955648"}
Sep 09 20:54:01.665541 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:01.665533    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss" event={"ID":"92d54f63-73ce-4d1f-a0df-c0176265fb0f","Type":"ContainerStarted","Data":"5c0e1d20867b5c3cb14a9c3523df2eb52b487ff7f149afb35b792a27c5331acf"}
Sep 09 20:54:01.701884 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:01.701832    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss" podStartSLOduration=1.7018133340000001 podStartE2EDuration="1.701813334s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:54:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:54:01.699001717 +0000 UTC m=+803.326629591" watchObservedRunningTime="2026-09-09 20:54:01.701813334 +0000 UTC m=+803.329441207"
Sep 09 20:54:02.088845 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:02.088815    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss"
Sep 09 20:54:02.094054 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:02.094026    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss"
Sep 09 20:54:02.671429 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:02.671397    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss"
Sep 09 20:54:02.672292 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:02.672273    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss"
Sep 09 20:54:04.035280 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:04.035244    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m"]
Sep 09 20:54:04.037824 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:04.037807    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m"
Sep 09 20:54:04.046539 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:04.046514    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-04cfc947-openshift-default-dockercfg-8b9q8\""
Sep 09 20:54:04.062394 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:04.062362    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fw452\" (UniqueName: \"kubernetes.io/projected/d39d8f5a-1c5c-4b84-a4de-d65511571543-kube-api-access-fw452\") pod \"e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m\" (UID: \"d39d8f5a-1c5c-4b84-a4de-d65511571543\") " pod="openshift-ingress/e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m"
Sep 09 20:54:04.062524 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:04.062416    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d39d8f5a-1c5c-4b84-a4de-d65511571543-workload-socket\") pod \"e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m\" (UID: \"d39d8f5a-1c5c-4b84-a4de-d65511571543\") " pod="openshift-ingress/e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m"
Sep 09 20:54:04.062524 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:04.062476    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d39d8f5a-1c5c-4b84-a4de-d65511571543-workload-certs\") pod \"e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m\" (UID: \"d39d8f5a-1c5c-4b84-a4de-d65511571543\") " pod="openshift-ingress/e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m"
Sep 09 20:54:04.062524 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:04.062512    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d39d8f5a-1c5c-4b84-a4de-d65511571543-istiod-ca-cert\") pod \"e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m\" (UID: \"d39d8f5a-1c5c-4b84-a4de-d65511571543\") " pod="openshift-ingress/e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m"
Sep 09 20:54:04.062683 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:04.062616    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d39d8f5a-1c5c-4b84-a4de-d65511571543-istio-token\") pod \"e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m\" (UID: \"d39d8f5a-1c5c-4b84-a4de-d65511571543\") " pod="openshift-ingress/e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m"
Sep 09 20:54:04.062733 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:04.062686    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d39d8f5a-1c5c-4b84-a4de-d65511571543-credential-socket\") pod \"e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m\" (UID: \"d39d8f5a-1c5c-4b84-a4de-d65511571543\") " pod="openshift-ingress/e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m"
Sep 09 20:54:04.062783 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:04.062750    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d39d8f5a-1c5c-4b84-a4de-d65511571543-istio-envoy\") pod \"e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m\" (UID: \"d39d8f5a-1c5c-4b84-a4de-d65511571543\") " pod="openshift-ingress/e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m"
Sep 09 20:54:04.062852 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:04.062830    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d39d8f5a-1c5c-4b84-a4de-d65511571543-istio-podinfo\") pod \"e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m\" (UID: \"d39d8f5a-1c5c-4b84-a4de-d65511571543\") " pod="openshift-ingress/e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m"
Sep 09 20:54:04.062902 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:04.062872    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d39d8f5a-1c5c-4b84-a4de-d65511571543-istio-data\") pod \"e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m\" (UID: \"d39d8f5a-1c5c-4b84-a4de-d65511571543\") " pod="openshift-ingress/e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m"
Sep 09 20:54:04.102160 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:04.102127    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m"]
Sep 09 20:54:04.164769 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:04.164641    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fw452\" (UniqueName: \"kubernetes.io/projected/d39d8f5a-1c5c-4b84-a4de-d65511571543-kube-api-access-fw452\") pod \"e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m\" (UID: \"d39d8f5a-1c5c-4b84-a4de-d65511571543\") " pod="openshift-ingress/e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m"
Sep 09 20:54:04.164769 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:04.164704    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d39d8f5a-1c5c-4b84-a4de-d65511571543-workload-socket\") pod \"e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m\" (UID: \"d39d8f5a-1c5c-4b84-a4de-d65511571543\") " pod="openshift-ingress/e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m"
Sep 09 20:54:04.164769 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:04.164734    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d39d8f5a-1c5c-4b84-a4de-d65511571543-workload-certs\") pod \"e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m\" (UID: \"d39d8f5a-1c5c-4b84-a4de-d65511571543\") " pod="openshift-ingress/e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m"
Sep 09 20:54:04.164769 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:04.164763    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d39d8f5a-1c5c-4b84-a4de-d65511571543-istiod-ca-cert\") pod \"e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m\" (UID: \"d39d8f5a-1c5c-4b84-a4de-d65511571543\") " pod="openshift-ingress/e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m"
Sep 09 20:54:04.165163 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:04.164850    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d39d8f5a-1c5c-4b84-a4de-d65511571543-istio-token\") pod \"e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m\" (UID: \"d39d8f5a-1c5c-4b84-a4de-d65511571543\") " pod="openshift-ingress/e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m"
Sep 09 20:54:04.165163 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:04.164901    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d39d8f5a-1c5c-4b84-a4de-d65511571543-credential-socket\") pod \"e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m\" (UID: \"d39d8f5a-1c5c-4b84-a4de-d65511571543\") " pod="openshift-ingress/e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m"
Sep 09 20:54:04.165163 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:04.164962    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d39d8f5a-1c5c-4b84-a4de-d65511571543-istio-envoy\") pod \"e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m\" (UID: \"d39d8f5a-1c5c-4b84-a4de-d65511571543\") " pod="openshift-ingress/e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m"
Sep 09 20:54:04.165163 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:04.165011    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d39d8f5a-1c5c-4b84-a4de-d65511571543-istio-podinfo\") pod \"e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m\" (UID: \"d39d8f5a-1c5c-4b84-a4de-d65511571543\") " pod="openshift-ingress/e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m"
Sep 09 20:54:04.165163 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:04.165038    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d39d8f5a-1c5c-4b84-a4de-d65511571543-istio-data\") pod \"e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m\" (UID: \"d39d8f5a-1c5c-4b84-a4de-d65511571543\") " pod="openshift-ingress/e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m"
Sep 09 20:54:04.165606 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:04.165532    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d39d8f5a-1c5c-4b84-a4de-d65511571543-istio-data\") pod \"e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m\" (UID: \"d39d8f5a-1c5c-4b84-a4de-d65511571543\") " pod="openshift-ingress/e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m"
Sep 09 20:54:04.166386 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:04.166344    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d39d8f5a-1c5c-4b84-a4de-d65511571543-workload-socket\") pod \"e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m\" (UID: \"d39d8f5a-1c5c-4b84-a4de-d65511571543\") " pod="openshift-ingress/e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m"
Sep 09 20:54:04.166637 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:04.166602    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d39d8f5a-1c5c-4b84-a4de-d65511571543-workload-certs\") pod \"e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m\" (UID: \"d39d8f5a-1c5c-4b84-a4de-d65511571543\") " pod="openshift-ingress/e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m"
Sep 09 20:54:04.167268 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:04.167231    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d39d8f5a-1c5c-4b84-a4de-d65511571543-istiod-ca-cert\") pod \"e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m\" (UID: \"d39d8f5a-1c5c-4b84-a4de-d65511571543\") " pod="openshift-ingress/e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m"
Sep 09 20:54:04.168183 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:04.168093    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d39d8f5a-1c5c-4b84-a4de-d65511571543-credential-socket\") pod \"e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m\" (UID: \"d39d8f5a-1c5c-4b84-a4de-d65511571543\") " pod="openshift-ingress/e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m"
Sep 09 20:54:04.171859 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:04.171836    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d39d8f5a-1c5c-4b84-a4de-d65511571543-istio-podinfo\") pod \"e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m\" (UID: \"d39d8f5a-1c5c-4b84-a4de-d65511571543\") " pod="openshift-ingress/e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m"
Sep 09 20:54:04.185507 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:04.185469    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d39d8f5a-1c5c-4b84-a4de-d65511571543-istio-envoy\") pod \"e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m\" (UID: \"d39d8f5a-1c5c-4b84-a4de-d65511571543\") " pod="openshift-ingress/e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m"
Sep 09 20:54:04.187937 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:04.187906    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fw452\" (UniqueName: \"kubernetes.io/projected/d39d8f5a-1c5c-4b84-a4de-d65511571543-kube-api-access-fw452\") pod \"e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m\" (UID: \"d39d8f5a-1c5c-4b84-a4de-d65511571543\") " pod="openshift-ingress/e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m"
Sep 09 20:54:04.198858 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:04.198833    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d39d8f5a-1c5c-4b84-a4de-d65511571543-istio-token\") pod \"e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m\" (UID: \"d39d8f5a-1c5c-4b84-a4de-d65511571543\") " pod="openshift-ingress/e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m"
Sep 09 20:54:04.349590 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:04.349502    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m"
Sep 09 20:54:04.670336 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:04.670239    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m"]
Sep 09 20:54:04.674617 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:54:04.674591    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd39d8f5a_1c5c_4b84_a4de_d65511571543.slice/crio-9214c21c0ff129de76db45ddd2c43668e715be1e2d9ace123d78466d2e199b36 WatchSource:0}: Error finding container 9214c21c0ff129de76db45ddd2c43668e715be1e2d9ace123d78466d2e199b36: Status 404 returned error can't find the container with id 9214c21c0ff129de76db45ddd2c43668e715be1e2d9ace123d78466d2e199b36
Sep 09 20:54:04.676802 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:04.676772    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:54:04.676879 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:04.676837    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:54:04.676879 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:04.676866    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:54:04.682325 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:04.682303    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m" event={"ID":"d39d8f5a-1c5c-4b84-a4de-d65511571543","Type":"ContainerStarted","Data":"9214c21c0ff129de76db45ddd2c43668e715be1e2d9ace123d78466d2e199b36"}
Sep 09 20:54:05.696816 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:05.696730    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m" event={"ID":"d39d8f5a-1c5c-4b84-a4de-d65511571543","Type":"ContainerStarted","Data":"cdb0267ebb031e24fc95797a4d62c20b38269e48c451dc33e012bb131160dbb6"}
Sep 09 20:54:05.753522 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:05.753431    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m" podStartSLOduration=2.753406216 podStartE2EDuration="2.753406216s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:54:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:54:05.748693914 +0000 UTC m=+807.376321812" watchObservedRunningTime="2026-09-09 20:54:05.753406216 +0000 UTC m=+807.381034092"
Sep 09 20:54:06.350616 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:06.350572    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m"
Sep 09 20:54:06.356387 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:06.356360    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m"
Sep 09 20:54:06.703883 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:06.703789    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m"
Sep 09 20:54:06.707504 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:06.707477    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m"
Sep 09 20:54:09.363001 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:09.362964    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-04cfc947-f65f64b46-xlthl"]
Sep 09 20:54:09.366096 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:09.366066    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-04cfc947-f65f64b46-xlthl"
Sep 09 20:54:09.369503 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:09.369484    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-04cfc947\""
Sep 09 20:54:09.369702 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:09.369677    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-04cfc947-dockercfg-8rf78\""
Sep 09 20:54:09.385273 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:09.385250    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-04cfc947-f65f64b46-xlthl"]
Sep 09 20:54:09.435178 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:09.435139    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfs72\" (UniqueName: \"kubernetes.io/projected/4ca83c6a-7477-4965-84af-aee691cebab4-kube-api-access-kfs72\") pod \"payload-pre-processing-e2e-ait-04cfc947-f65f64b46-xlthl\" (UID: \"4ca83c6a-7477-4965-84af-aee691cebab4\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-04cfc947-f65f64b46-xlthl"
Sep 09 20:54:09.436419 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:09.436392    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ca83c6a-7477-4965-84af-aee691cebab4-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-04cfc947-f65f64b46-xlthl\" (UID: \"4ca83c6a-7477-4965-84af-aee691cebab4\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-04cfc947-f65f64b46-xlthl"
Sep 09 20:54:09.537366 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:09.537308    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ca83c6a-7477-4965-84af-aee691cebab4-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-04cfc947-f65f64b46-xlthl\" (UID: \"4ca83c6a-7477-4965-84af-aee691cebab4\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-04cfc947-f65f64b46-xlthl"
Sep 09 20:54:09.537544 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:09.537482    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfs72\" (UniqueName: \"kubernetes.io/projected/4ca83c6a-7477-4965-84af-aee691cebab4-kube-api-access-kfs72\") pod \"payload-pre-processing-e2e-ait-04cfc947-f65f64b46-xlthl\" (UID: \"4ca83c6a-7477-4965-84af-aee691cebab4\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-04cfc947-f65f64b46-xlthl"
Sep 09 20:54:09.538557 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:09.538527    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ca83c6a-7477-4965-84af-aee691cebab4-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-04cfc947-f65f64b46-xlthl\" (UID: \"4ca83c6a-7477-4965-84af-aee691cebab4\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-04cfc947-f65f64b46-xlthl"
Sep 09 20:54:09.548794 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:09.548762    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfs72\" (UniqueName: \"kubernetes.io/projected/4ca83c6a-7477-4965-84af-aee691cebab4-kube-api-access-kfs72\") pod \"payload-pre-processing-e2e-ait-04cfc947-f65f64b46-xlthl\" (UID: \"4ca83c6a-7477-4965-84af-aee691cebab4\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-04cfc947-f65f64b46-xlthl"
Sep 09 20:54:09.677470 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:09.677364    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-04cfc947-f65f64b46-xlthl"
Sep 09 20:54:09.988854 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:09.988653    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-04cfc947-f65f64b46-xlthl"]
Sep 09 20:54:09.991071 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:54:09.991044    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ca83c6a_7477_4965_84af_aee691cebab4.slice/crio-e3b72e589540333c18251f1d524b8b98302ac735706990815f4cdaafab65c8c1 WatchSource:0}: Error finding container e3b72e589540333c18251f1d524b8b98302ac735706990815f4cdaafab65c8c1: Status 404 returned error can't find the container with id e3b72e589540333c18251f1d524b8b98302ac735706990815f4cdaafab65c8c1
Sep 09 20:54:10.751469 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:10.751414    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-04cfc947-f65f64b46-xlthl" event={"ID":"4ca83c6a-7477-4965-84af-aee691cebab4","Type":"ContainerStarted","Data":"e3b72e589540333c18251f1d524b8b98302ac735706990815f4cdaafab65c8c1"}
Sep 09 20:54:13.769434 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:13.769402    2581 generic.go:356] "Generic (PLEG): container finished" podID="07f6f4a7-5d41-4db5-9c4a-eceaf6e96903" containerID="3387a184712e50dcad798672842de44a6fef191ddf2ece416e7abcb1cc6d7cc7" exitCode=0
Sep 09 20:54:13.769814 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:13.769477    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk" event={"ID":"07f6f4a7-5d41-4db5-9c4a-eceaf6e96903","Type":"ContainerDied","Data":"3387a184712e50dcad798672842de44a6fef191ddf2ece416e7abcb1cc6d7cc7"}
Sep 09 20:54:13.769964 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:13.769950    2581 scope.go:117] "RemoveContainer" containerID="3387a184712e50dcad798672842de44a6fef191ddf2ece416e7abcb1cc6d7cc7"
Sep 09 20:54:13.771268 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:13.771238    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:54:13.771380 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:13.771318    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:54:13.771380 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:13.771358    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:54:13.771475 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:13.771391    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-04cfc947-f65f64b46-xlthl" event={"ID":"4ca83c6a-7477-4965-84af-aee691cebab4","Type":"ContainerStarted","Data":"12ce9934cc031cfc8c02df44bd8a83ce06f6b2500fa97163fbb3b850708e0d86"}
Sep 09 20:54:13.771475 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:13.771472    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-04cfc947-f65f64b46-xlthl"
Sep 09 20:54:13.773154 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:13.773134    2581 generic.go:356] "Generic (PLEG): container finished" podID="3dff37dd-436f-470c-9b0e-71ce7cd1d9f2" containerID="cce5895e33621c95c8291dab949ab8ace88c6c68e91bbea855fe67bfb2ec24a4" exitCode=0
Sep 09 20:54:13.773255 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:13.773166    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd" event={"ID":"3dff37dd-436f-470c-9b0e-71ce7cd1d9f2","Type":"ContainerDied","Data":"cce5895e33621c95c8291dab949ab8ace88c6c68e91bbea855fe67bfb2ec24a4"}
Sep 09 20:54:13.773672 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:13.773555    2581 scope.go:117] "RemoveContainer" containerID="cce5895e33621c95c8291dab949ab8ace88c6c68e91bbea855fe67bfb2ec24a4"
Sep 09 20:54:13.774615 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:13.774591    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:54:13.774669 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:13.774651    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:54:13.774711 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:13.774699    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:54:13.839836 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:13.839768    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-04cfc947-f65f64b46-xlthl" podStartSLOduration=2.011769148 podStartE2EDuration="4.839747583s" totalImagesPullingTime="2.827978435s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:54:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:54:13.837418669 +0000 UTC m=+815.465046543" watchObservedRunningTime="2026-09-09 20:54:13.839747583 +0000 UTC m=+815.467375456"
Sep 09 20:54:14.781255 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:14.781218    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd" event={"ID":"3dff37dd-436f-470c-9b0e-71ce7cd1d9f2","Type":"ContainerStarted","Data":"c5452b45a75f014bf445627945b7a155cc7df5e54c8f348d672d239af406a0c8"}
Sep 09 20:54:14.783626 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:14.783598    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk" event={"ID":"07f6f4a7-5d41-4db5-9c4a-eceaf6e96903","Type":"ContainerStarted","Data":"6abe8eb1096f66f61cc06337e07f88b9740936d05351dc7f6b9d06a568fc7008"}
Sep 09 20:54:14.942635 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:14.942586    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd"
Sep 09 20:54:14.943305 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:14.942898    2581 patch_prober.go:28] interesting pod/e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body=
Sep 09 20:54:14.943305 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:14.942966    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd" podUID="3dff37dd-436f-470c-9b0e-71ce7cd1d9f2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused"
Sep 09 20:54:15.025907 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:15.025798    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk"
Sep 09 20:54:15.026156 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:15.026125    2581 patch_prober.go:28] interesting pod/e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body=
Sep 09 20:54:15.026252 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:15.026188    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk" podUID="07f6f4a7-5d41-4db5-9c4a-eceaf6e96903" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused"
Sep 09 20:54:15.942970 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:15.942925    2581 patch_prober.go:28] interesting pod/e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body=
Sep 09 20:54:15.943541 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:15.942998    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd" podUID="3dff37dd-436f-470c-9b0e-71ce7cd1d9f2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused"
Sep 09 20:54:16.026677 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:16.026619    2581 patch_prober.go:28] interesting pod/e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body=
Sep 09 20:54:16.026886 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:16.026703    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk" podUID="07f6f4a7-5d41-4db5-9c4a-eceaf6e96903" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused"
Sep 09 20:54:16.942825 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:16.942775    2581 patch_prober.go:28] interesting pod/e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body=
Sep 09 20:54:16.943050 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:16.942849    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd" podUID="3dff37dd-436f-470c-9b0e-71ce7cd1d9f2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused"
Sep 09 20:54:17.026278 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:17.026229    2581 patch_prober.go:28] interesting pod/e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body=
Sep 09 20:54:17.026481 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:17.026355    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk" podUID="07f6f4a7-5d41-4db5-9c4a-eceaf6e96903" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused"
Sep 09 20:54:17.942901 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:17.942860    2581 patch_prober.go:28] interesting pod/e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body=
Sep 09 20:54:17.943237 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:17.943203    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd" podUID="3dff37dd-436f-470c-9b0e-71ce7cd1d9f2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused"
Sep 09 20:54:18.027690 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:18.027651    2581 patch_prober.go:28] interesting pod/e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body=
Sep 09 20:54:18.027995 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:18.027951    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk" podUID="07f6f4a7-5d41-4db5-9c4a-eceaf6e96903" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused"
Sep 09 20:54:18.942850 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:18.942810    2581 patch_prober.go:28] interesting pod/e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body=
Sep 09 20:54:18.943180 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:18.943144    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd" podUID="3dff37dd-436f-470c-9b0e-71ce7cd1d9f2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused"
Sep 09 20:54:19.026335 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:19.026298    2581 patch_prober.go:28] interesting pod/e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body=
Sep 09 20:54:19.026925 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:19.026889    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk" podUID="07f6f4a7-5d41-4db5-9c4a-eceaf6e96903" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused"
Sep 09 20:54:19.944743 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:19.944702    2581 patch_prober.go:28] interesting pod/e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body=
Sep 09 20:54:19.945033 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:19.945000    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd" podUID="3dff37dd-436f-470c-9b0e-71ce7cd1d9f2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused"
Sep 09 20:54:20.036018 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:20.035981    2581 patch_prober.go:28] interesting pod/e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body=
Sep 09 20:54:20.036677 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:20.036627    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk" podUID="07f6f4a7-5d41-4db5-9c4a-eceaf6e96903" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused"
Sep 09 20:54:21.042600 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:21.042556    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd"
Sep 09 20:54:21.044972 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:21.044949    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd"
Sep 09 20:54:21.049408 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:21.049387    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd"
Sep 09 20:54:21.427007 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:21.426910    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk"
Sep 09 20:54:21.428207 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:21.428127    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk"
Sep 09 20:54:21.435974 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:21.435943    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk"
Sep 09 20:54:24.742118 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:24.742076    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx"]
Sep 09 20:54:24.750453 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:24.750428    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx"
Sep 09 20:54:24.756823 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:24.756797    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-423e6a-openshift-default-dockercfg-2j8ft\""
Sep 09 20:54:24.786026 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:24.785994    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-04cfc947-f65f64b46-xlthl"
Sep 09 20:54:24.802172 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:24.802137    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67436bca-5ef3-400f-9892-6d1e3403349b-credential-socket\") pod \"e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx\" (UID: \"67436bca-5ef3-400f-9892-6d1e3403349b\") " pod="openshift-ingress/e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx"
Sep 09 20:54:24.802355 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:24.802203    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67436bca-5ef3-400f-9892-6d1e3403349b-istio-envoy\") pod \"e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx\" (UID: \"67436bca-5ef3-400f-9892-6d1e3403349b\") " pod="openshift-ingress/e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx"
Sep 09 20:54:24.802355 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:24.802245    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67436bca-5ef3-400f-9892-6d1e3403349b-istio-podinfo\") pod \"e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx\" (UID: \"67436bca-5ef3-400f-9892-6d1e3403349b\") " pod="openshift-ingress/e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx"
Sep 09 20:54:24.802355 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:24.802275    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67436bca-5ef3-400f-9892-6d1e3403349b-workload-socket\") pod \"e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx\" (UID: \"67436bca-5ef3-400f-9892-6d1e3403349b\") " pod="openshift-ingress/e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx"
Sep 09 20:54:24.802355 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:24.802321    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4ft7\" (UniqueName: \"kubernetes.io/projected/67436bca-5ef3-400f-9892-6d1e3403349b-kube-api-access-k4ft7\") pod \"e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx\" (UID: \"67436bca-5ef3-400f-9892-6d1e3403349b\") " pod="openshift-ingress/e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx"
Sep 09 20:54:24.802626 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:24.802371    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67436bca-5ef3-400f-9892-6d1e3403349b-istio-token\") pod \"e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx\" (UID: \"67436bca-5ef3-400f-9892-6d1e3403349b\") " pod="openshift-ingress/e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx"
Sep 09 20:54:24.802626 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:24.802425    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67436bca-5ef3-400f-9892-6d1e3403349b-istiod-ca-cert\") pod \"e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx\" (UID: \"67436bca-5ef3-400f-9892-6d1e3403349b\") " pod="openshift-ingress/e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx"
Sep 09 20:54:24.802626 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:24.802495    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67436bca-5ef3-400f-9892-6d1e3403349b-workload-certs\") pod \"e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx\" (UID: \"67436bca-5ef3-400f-9892-6d1e3403349b\") " pod="openshift-ingress/e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx"
Sep 09 20:54:24.802626 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:24.802593    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67436bca-5ef3-400f-9892-6d1e3403349b-istio-data\") pod \"e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx\" (UID: \"67436bca-5ef3-400f-9892-6d1e3403349b\") " pod="openshift-ingress/e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx"
Sep 09 20:54:24.871806 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:24.871764    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx"]
Sep 09 20:54:24.904953 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:24.903793    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67436bca-5ef3-400f-9892-6d1e3403349b-istio-envoy\") pod \"e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx\" (UID: \"67436bca-5ef3-400f-9892-6d1e3403349b\") " pod="openshift-ingress/e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx"
Sep 09 20:54:24.904953 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:24.903867    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67436bca-5ef3-400f-9892-6d1e3403349b-istio-podinfo\") pod \"e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx\" (UID: \"67436bca-5ef3-400f-9892-6d1e3403349b\") " pod="openshift-ingress/e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx"
Sep 09 20:54:24.904953 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:24.903915    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67436bca-5ef3-400f-9892-6d1e3403349b-workload-socket\") pod \"e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx\" (UID: \"67436bca-5ef3-400f-9892-6d1e3403349b\") " pod="openshift-ingress/e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx"
Sep 09 20:54:24.904953 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:24.903980    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4ft7\" (UniqueName: \"kubernetes.io/projected/67436bca-5ef3-400f-9892-6d1e3403349b-kube-api-access-k4ft7\") pod \"e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx\" (UID: \"67436bca-5ef3-400f-9892-6d1e3403349b\") " pod="openshift-ingress/e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx"
Sep 09 20:54:24.904953 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:24.904032    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67436bca-5ef3-400f-9892-6d1e3403349b-istio-token\") pod \"e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx\" (UID: \"67436bca-5ef3-400f-9892-6d1e3403349b\") " pod="openshift-ingress/e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx"
Sep 09 20:54:24.904953 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:24.904083    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67436bca-5ef3-400f-9892-6d1e3403349b-istiod-ca-cert\") pod \"e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx\" (UID: \"67436bca-5ef3-400f-9892-6d1e3403349b\") " pod="openshift-ingress/e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx"
Sep 09 20:54:24.904953 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:24.904229    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67436bca-5ef3-400f-9892-6d1e3403349b-workload-certs\") pod \"e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx\" (UID: \"67436bca-5ef3-400f-9892-6d1e3403349b\") " pod="openshift-ingress/e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx"
Sep 09 20:54:24.904953 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:24.904267    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67436bca-5ef3-400f-9892-6d1e3403349b-istio-data\") pod \"e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx\" (UID: \"67436bca-5ef3-400f-9892-6d1e3403349b\") " pod="openshift-ingress/e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx"
Sep 09 20:54:24.904953 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:24.904295    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67436bca-5ef3-400f-9892-6d1e3403349b-credential-socket\") pod \"e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx\" (UID: \"67436bca-5ef3-400f-9892-6d1e3403349b\") " pod="openshift-ingress/e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx"
Sep 09 20:54:24.904953 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:24.904682    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67436bca-5ef3-400f-9892-6d1e3403349b-workload-certs\") pod \"e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx\" (UID: \"67436bca-5ef3-400f-9892-6d1e3403349b\") " pod="openshift-ingress/e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx"
Sep 09 20:54:24.905661 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:24.905048    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67436bca-5ef3-400f-9892-6d1e3403349b-workload-socket\") pod \"e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx\" (UID: \"67436bca-5ef3-400f-9892-6d1e3403349b\") " pod="openshift-ingress/e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx"
Sep 09 20:54:24.905661 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:24.905149    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67436bca-5ef3-400f-9892-6d1e3403349b-credential-socket\") pod \"e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx\" (UID: \"67436bca-5ef3-400f-9892-6d1e3403349b\") " pod="openshift-ingress/e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx"
Sep 09 20:54:24.905661 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:24.905272    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67436bca-5ef3-400f-9892-6d1e3403349b-istiod-ca-cert\") pod \"e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx\" (UID: \"67436bca-5ef3-400f-9892-6d1e3403349b\") " pod="openshift-ingress/e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx"
Sep 09 20:54:24.905956 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:24.905907    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67436bca-5ef3-400f-9892-6d1e3403349b-istio-data\") pod \"e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx\" (UID: \"67436bca-5ef3-400f-9892-6d1e3403349b\") " pod="openshift-ingress/e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx"
Sep 09 20:54:24.924719 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:24.912523    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67436bca-5ef3-400f-9892-6d1e3403349b-istio-envoy\") pod \"e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx\" (UID: \"67436bca-5ef3-400f-9892-6d1e3403349b\") " pod="openshift-ingress/e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx"
Sep 09 20:54:24.924719 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:24.913263    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67436bca-5ef3-400f-9892-6d1e3403349b-istio-podinfo\") pod \"e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx\" (UID: \"67436bca-5ef3-400f-9892-6d1e3403349b\") " pod="openshift-ingress/e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx"
Sep 09 20:54:24.926169 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:24.925557    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4ft7\" (UniqueName: \"kubernetes.io/projected/67436bca-5ef3-400f-9892-6d1e3403349b-kube-api-access-k4ft7\") pod \"e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx\" (UID: \"67436bca-5ef3-400f-9892-6d1e3403349b\") " pod="openshift-ingress/e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx"
Sep 09 20:54:24.932261 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:24.932229    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67436bca-5ef3-400f-9892-6d1e3403349b-istio-token\") pod \"e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx\" (UID: \"67436bca-5ef3-400f-9892-6d1e3403349b\") " pod="openshift-ingress/e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx"
Sep 09 20:54:25.062269 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:25.062233    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx"
Sep 09 20:54:25.352271 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:25.352247    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx"]
Sep 09 20:54:25.353799 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:54:25.353769    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67436bca_5ef3_400f_9892_6d1e3403349b.slice/crio-5c0882badf99b1d7f2b9b63f2ca40a82fc93dc3e86b565056ec453d5d0d0231b WatchSource:0}: Error finding container 5c0882badf99b1d7f2b9b63f2ca40a82fc93dc3e86b565056ec453d5d0d0231b: Status 404 returned error can't find the container with id 5c0882badf99b1d7f2b9b63f2ca40a82fc93dc3e86b565056ec453d5d0d0231b
Sep 09 20:54:25.356362 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:25.356326    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:54:25.356444 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:25.356394    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:54:25.356444 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:25.356426    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:54:25.880637 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:25.880601    2581 generic.go:356] "Generic (PLEG): container finished" podID="92d54f63-73ce-4d1f-a0df-c0176265fb0f" containerID="161ac86d91fcab7e0c69dbe55bbfc7fe88717654d867b3e6c97699cd72955648" exitCode=0
Sep 09 20:54:25.881058 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:25.880672    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss" event={"ID":"92d54f63-73ce-4d1f-a0df-c0176265fb0f","Type":"ContainerDied","Data":"161ac86d91fcab7e0c69dbe55bbfc7fe88717654d867b3e6c97699cd72955648"}
Sep 09 20:54:25.881191 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:25.881171    2581 scope.go:117] "RemoveContainer" containerID="161ac86d91fcab7e0c69dbe55bbfc7fe88717654d867b3e6c97699cd72955648"
Sep 09 20:54:25.882397 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:25.882362    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:54:25.882500 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:25.882418    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx" event={"ID":"67436bca-5ef3-400f-9892-6d1e3403349b","Type":"ContainerStarted","Data":"81fa1fe91f1684b322720d3bdeed556f730e4e2a53df649adcb43beaa9c74660"}
Sep 09 20:54:25.882500 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:25.882448    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:54:25.882500 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:25.882462    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx" event={"ID":"67436bca-5ef3-400f-9892-6d1e3403349b","Type":"ContainerStarted","Data":"5c0882badf99b1d7f2b9b63f2ca40a82fc93dc3e86b565056ec453d5d0d0231b"}
Sep 09 20:54:25.882500 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:25.882490    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:54:25.884017 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:25.884000    2581 generic.go:356] "Generic (PLEG): container finished" podID="d39d8f5a-1c5c-4b84-a4de-d65511571543" containerID="cdb0267ebb031e24fc95797a4d62c20b38269e48c451dc33e012bb131160dbb6" exitCode=0
Sep 09 20:54:25.884077 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:25.884027    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m" event={"ID":"d39d8f5a-1c5c-4b84-a4de-d65511571543","Type":"ContainerDied","Data":"cdb0267ebb031e24fc95797a4d62c20b38269e48c451dc33e012bb131160dbb6"}
Sep 09 20:54:25.884356 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:25.884344    2581 scope.go:117] "RemoveContainer" containerID="cdb0267ebb031e24fc95797a4d62c20b38269e48c451dc33e012bb131160dbb6"
Sep 09 20:54:25.885389 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:25.885366    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:54:25.885444 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:25.885430    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:54:25.885479 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:25.885468    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:54:25.987537 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:25.987370    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx" podStartSLOduration=1.987345709 podStartE2EDuration="1.987345709s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:54:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:54:25.981123578 +0000 UTC m=+827.608751448" watchObservedRunningTime="2026-09-09 20:54:25.987345709 +0000 UTC m=+827.614973588"
Sep 09 20:54:26.062474 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:26.062439    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx"
Sep 09 20:54:26.068845 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:26.068659    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx"
Sep 09 20:54:26.892309 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:26.892272    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss" event={"ID":"92d54f63-73ce-4d1f-a0df-c0176265fb0f","Type":"ContainerStarted","Data":"ac347d12a46d5e9eaa809a19beafaf25bc19c0c03309650ab1ad5573efbfe027"}
Sep 09 20:54:26.895314 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:26.895279    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m" event={"ID":"d39d8f5a-1c5c-4b84-a4de-d65511571543","Type":"ContainerStarted","Data":"b26d28db32c19ba59481b451abcfa840b9ce1d53b2404675e32c159fc90f4932"}
Sep 09 20:54:26.895529 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:26.895511    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx"
Sep 09 20:54:26.896648 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:26.896627    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx"
Sep 09 20:54:27.089465 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:27.089429    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss"
Sep 09 20:54:27.089797 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:27.089771    2581 patch_prober.go:28] interesting pod/e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Sep 09 20:54:27.089866 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:27.089841    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss" podUID="92d54f63-73ce-4d1f-a0df-c0176265fb0f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Sep 09 20:54:27.350439 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:27.350380    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m"
Sep 09 20:54:27.350925 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:27.350896    2581 patch_prober.go:28] interesting pod/e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body=
Sep 09 20:54:27.351061 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:27.350958    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m" podUID="d39d8f5a-1c5c-4b84-a4de-d65511571543" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused"
Sep 09 20:54:28.089733 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:28.089699    2581 patch_prober.go:28] interesting pod/e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Sep 09 20:54:28.090110 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:28.089754    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss" podUID="92d54f63-73ce-4d1f-a0df-c0176265fb0f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Sep 09 20:54:28.350377 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:28.350282    2581 patch_prober.go:28] interesting pod/e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body=
Sep 09 20:54:28.350377 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:28.350364    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m" podUID="d39d8f5a-1c5c-4b84-a4de-d65511571543" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused"
Sep 09 20:54:29.089374 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:29.089323    2581 patch_prober.go:28] interesting pod/e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Sep 09 20:54:29.089541 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:29.089393    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss" podUID="92d54f63-73ce-4d1f-a0df-c0176265fb0f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Sep 09 20:54:29.350342 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:29.350232    2581 patch_prober.go:28] interesting pod/e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body=
Sep 09 20:54:29.350342 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:29.350316    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m" podUID="d39d8f5a-1c5c-4b84-a4de-d65511571543" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused"
Sep 09 20:54:30.089702 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:30.089661    2581 patch_prober.go:28] interesting pod/e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Sep 09 20:54:30.089929 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:30.089727    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss" podUID="92d54f63-73ce-4d1f-a0df-c0176265fb0f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Sep 09 20:54:30.350709 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:30.350622    2581 patch_prober.go:28] interesting pod/e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body=
Sep 09 20:54:30.351178 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:30.351149    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m" podUID="d39d8f5a-1c5c-4b84-a4de-d65511571543" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused"
Sep 09 20:54:31.089526 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:31.089492    2581 patch_prober.go:28] interesting pod/e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Sep 09 20:54:31.089705 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:31.089549    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss" podUID="92d54f63-73ce-4d1f-a0df-c0176265fb0f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Sep 09 20:54:31.350497 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:31.350397    2581 patch_prober.go:28] interesting pod/e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body=
Sep 09 20:54:31.351165 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:31.350739    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m" podUID="d39d8f5a-1c5c-4b84-a4de-d65511571543" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused"
Sep 09 20:54:32.089397 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:32.089358    2581 patch_prober.go:28] interesting pod/e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Sep 09 20:54:32.089579 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:32.089427    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss" podUID="92d54f63-73ce-4d1f-a0df-c0176265fb0f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Sep 09 20:54:32.368145 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:32.368035    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m"
Sep 09 20:54:32.368584 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:32.368351    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m"
Sep 09 20:54:32.369191 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:32.369174    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m"
Sep 09 20:54:32.672355 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:32.672257    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss"
Sep 09 20:54:33.093485 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:33.093455    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss"
Sep 09 20:54:33.094367 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:33.094344    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss"
Sep 09 20:54:37.930580 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:37.930547    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-04cfc947-dtt6v"]
Sep 09 20:54:37.934269 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:37.934250    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-04cfc947-dtt6v"
Sep 09 20:54:37.937047 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:37.937025    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-glz44\""
Sep 09 20:54:37.942841 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:37.942817    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-04cfc947-dtt6v"]
Sep 09 20:54:38.049296 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:38.049258    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3f51c04-c060-4bbf-aff6-1d1f61ec2e79-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-04cfc947-dtt6v\" (UID: \"c3f51c04-c060-4bbf-aff6-1d1f61ec2e79\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-04cfc947-dtt6v"
Sep 09 20:54:38.151329 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:38.151282    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3f51c04-c060-4bbf-aff6-1d1f61ec2e79-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-04cfc947-dtt6v\" (UID: \"c3f51c04-c060-4bbf-aff6-1d1f61ec2e79\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-04cfc947-dtt6v"
Sep 09 20:54:38.152266 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:38.152234    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3f51c04-c060-4bbf-aff6-1d1f61ec2e79-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-04cfc947-dtt6v\" (UID: \"c3f51c04-c060-4bbf-aff6-1d1f61ec2e79\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-04cfc947-dtt6v"
Sep 09 20:54:38.246595 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:38.246564    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-04cfc947-dtt6v"
Sep 09 20:54:38.522849 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:38.522817    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-04cfc947-dtt6v"]
Sep 09 20:54:38.524827 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:54:38.524799    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3f51c04_c060_4bbf_aff6_1d1f61ec2e79.slice/crio-d2c515ad8d2183bd215931de7ec6374de3be7cb301c73fe38063d2668869f3d8 WatchSource:0}: Error finding container d2c515ad8d2183bd215931de7ec6374de3be7cb301c73fe38063d2668869f3d8: Status 404 returned error can't find the container with id d2c515ad8d2183bd215931de7ec6374de3be7cb301c73fe38063d2668869f3d8
Sep 09 20:54:38.973128 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:38.973033    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-04cfc947-dtt6v" event={"ID":"c3f51c04-c060-4bbf-aff6-1d1f61ec2e79","Type":"ContainerStarted","Data":"d2c515ad8d2183bd215931de7ec6374de3be7cb301c73fe38063d2668869f3d8"}
Sep 09 20:54:42.995773 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:42.995728    2581 generic.go:356] "Generic (PLEG): container finished" podID="c3f51c04-c060-4bbf-aff6-1d1f61ec2e79" containerID="8d7a1f3a11e913073da54e7a9e9af623746a44689be2946383df04fa3550373c" exitCode=0
Sep 09 20:54:42.996243 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:42.995818    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-04cfc947-dtt6v" event={"ID":"c3f51c04-c060-4bbf-aff6-1d1f61ec2e79","Type":"ContainerDied","Data":"8d7a1f3a11e913073da54e7a9e9af623746a44689be2946383df04fa3550373c"}
Sep 09 20:54:44.131339 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:44.131314    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-04cfc947-dtt6v"
Sep 09 20:54:44.215169 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:44.215135    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3f51c04-c060-4bbf-aff6-1d1f61ec2e79-maas-api-service-ca\") pod \"c3f51c04-c060-4bbf-aff6-1d1f61ec2e79\" (UID: \"c3f51c04-c060-4bbf-aff6-1d1f61ec2e79\") "
Sep 09 20:54:44.215484 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:44.215458    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3f51c04-c060-4bbf-aff6-1d1f61ec2e79-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c3f51c04-c060-4bbf-aff6-1d1f61ec2e79" (UID: "c3f51c04-c060-4bbf-aff6-1d1f61ec2e79"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:54:44.316180 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:44.316067    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3f51c04-c060-4bbf-aff6-1d1f61ec2e79-maas-api-service-ca\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:54:45.007049 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:45.007016    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-04cfc947-dtt6v"
Sep 09 20:54:45.007241 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:45.007054    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-04cfc947-dtt6v" event={"ID":"c3f51c04-c060-4bbf-aff6-1d1f61ec2e79","Type":"ContainerDied","Data":"d2c515ad8d2183bd215931de7ec6374de3be7cb301c73fe38063d2668869f3d8"}
Sep 09 20:54:45.007241 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:45.007130    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d2c515ad8d2183bd215931de7ec6374de3be7cb301c73fe38063d2668869f3d8"
Sep 09 20:54:49.594600 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:49.594567    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-04cfc947-f65f64b46-xlthl"]
Sep 09 20:54:49.595036 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:49.594790    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-04cfc947-f65f64b46-xlthl" podUID="4ca83c6a-7477-4965-84af-aee691cebab4" containerName="payload-pre-processing" containerID="cri-o://12ce9934cc031cfc8c02df44bd8a83ce06f6b2500fa97163fbb3b850708e0d86" gracePeriod=30
Sep 09 20:54:49.889606 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:49.889282    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-04cfc947-f65f64b46-xlthl"
Sep 09 20:54:49.937727 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:49.937698    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-04cfc947-dtt6v"]
Sep 09 20:54:49.941034 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:49.941005    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-04cfc947-dtt6v"]
Sep 09 20:54:49.977045 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:49.977016    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ca83c6a-7477-4965-84af-aee691cebab4-plugins-config-volume\") pod \"4ca83c6a-7477-4965-84af-aee691cebab4\" (UID: \"4ca83c6a-7477-4965-84af-aee691cebab4\") "
Sep 09 20:54:49.977202 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:49.977145    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kfs72\" (UniqueName: \"kubernetes.io/projected/4ca83c6a-7477-4965-84af-aee691cebab4-kube-api-access-kfs72\") pod \"4ca83c6a-7477-4965-84af-aee691cebab4\" (UID: \"4ca83c6a-7477-4965-84af-aee691cebab4\") "
Sep 09 20:54:49.977387 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:49.977364    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ca83c6a-7477-4965-84af-aee691cebab4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4ca83c6a-7477-4965-84af-aee691cebab4" (UID: "4ca83c6a-7477-4965-84af-aee691cebab4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:54:49.979284 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:49.979256    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ca83c6a-7477-4965-84af-aee691cebab4-kube-api-access-kfs72" (OuterVolumeSpecName: "kube-api-access-kfs72") pod "4ca83c6a-7477-4965-84af-aee691cebab4" (UID: "4ca83c6a-7477-4965-84af-aee691cebab4"). InnerVolumeSpecName "kube-api-access-kfs72". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:54:50.032873 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:50.032832    2581 generic.go:356] "Generic (PLEG): container finished" podID="4ca83c6a-7477-4965-84af-aee691cebab4" containerID="12ce9934cc031cfc8c02df44bd8a83ce06f6b2500fa97163fbb3b850708e0d86" exitCode=0
Sep 09 20:54:50.032873 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:50.032879    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-04cfc947-f65f64b46-xlthl"
Sep 09 20:54:50.033078 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:50.032903    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-04cfc947-f65f64b46-xlthl" event={"ID":"4ca83c6a-7477-4965-84af-aee691cebab4","Type":"ContainerDied","Data":"12ce9934cc031cfc8c02df44bd8a83ce06f6b2500fa97163fbb3b850708e0d86"}
Sep 09 20:54:50.033078 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:50.032957    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-04cfc947-f65f64b46-xlthl" event={"ID":"4ca83c6a-7477-4965-84af-aee691cebab4","Type":"ContainerDied","Data":"e3b72e589540333c18251f1d524b8b98302ac735706990815f4cdaafab65c8c1"}
Sep 09 20:54:50.033078 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:50.032991    2581 scope.go:117] "RemoveContainer" containerID="12ce9934cc031cfc8c02df44bd8a83ce06f6b2500fa97163fbb3b850708e0d86"
Sep 09 20:54:50.043650 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:50.043627    2581 scope.go:117] "RemoveContainer" containerID="12ce9934cc031cfc8c02df44bd8a83ce06f6b2500fa97163fbb3b850708e0d86"
Sep 09 20:54:50.044048 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:54:50.044017    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12ce9934cc031cfc8c02df44bd8a83ce06f6b2500fa97163fbb3b850708e0d86\": container with ID starting with 12ce9934cc031cfc8c02df44bd8a83ce06f6b2500fa97163fbb3b850708e0d86 not found: ID does not exist" containerID="12ce9934cc031cfc8c02df44bd8a83ce06f6b2500fa97163fbb3b850708e0d86"
Sep 09 20:54:50.044371 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:50.044262    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12ce9934cc031cfc8c02df44bd8a83ce06f6b2500fa97163fbb3b850708e0d86"} err="failed to get container status \"12ce9934cc031cfc8c02df44bd8a83ce06f6b2500fa97163fbb3b850708e0d86\": rpc error: code = NotFound desc = could not find container \"12ce9934cc031cfc8c02df44bd8a83ce06f6b2500fa97163fbb3b850708e0d86\": container with ID starting with 12ce9934cc031cfc8c02df44bd8a83ce06f6b2500fa97163fbb3b850708e0d86 not found: ID does not exist"
Sep 09 20:54:50.060331 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:50.060303    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-04cfc947-f65f64b46-xlthl"]
Sep 09 20:54:50.064287 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:50.064263    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-04cfc947-f65f64b46-xlthl"]
Sep 09 20:54:50.078741 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:50.078718    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m"]
Sep 09 20:54:50.078920 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:50.078898    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ca83c6a-7477-4965-84af-aee691cebab4-plugins-config-volume\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:54:50.078987 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:50.078925    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kfs72\" (UniqueName: \"kubernetes.io/projected/4ca83c6a-7477-4965-84af-aee691cebab4-kube-api-access-kfs72\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:54:50.078987 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:50.078927    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m" podUID="d39d8f5a-1c5c-4b84-a4de-d65511571543" containerName="istio-proxy" containerID="cri-o://b26d28db32c19ba59481b451abcfa840b9ce1d53b2404675e32c159fc90f4932" gracePeriod=30
Sep 09 20:54:50.820909 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:50.820876    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ca83c6a-7477-4965-84af-aee691cebab4" path="/var/lib/kubelet/pods/4ca83c6a-7477-4965-84af-aee691cebab4/volumes"
Sep 09 20:54:50.821301 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:50.821265    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c3f51c04-c060-4bbf-aff6-1d1f61ec2e79" path="/var/lib/kubelet/pods/c3f51c04-c060-4bbf-aff6-1d1f61ec2e79/volumes"
Sep 09 20:54:51.199872 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:51.199793    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls"]
Sep 09 20:54:51.200370 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:51.200352    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ca83c6a-7477-4965-84af-aee691cebab4" containerName="payload-pre-processing"
Sep 09 20:54:51.200447 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:51.200373    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ca83c6a-7477-4965-84af-aee691cebab4" containerName="payload-pre-processing"
Sep 09 20:54:51.200447 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:51.200385    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3f51c04-c060-4bbf-aff6-1d1f61ec2e79" containerName="revoke-keys"
Sep 09 20:54:51.200447 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:51.200391    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3f51c04-c060-4bbf-aff6-1d1f61ec2e79" containerName="revoke-keys"
Sep 09 20:54:51.200581 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:51.200453    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3f51c04-c060-4bbf-aff6-1d1f61ec2e79" containerName="revoke-keys"
Sep 09 20:54:51.200581 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:51.200463    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ca83c6a-7477-4965-84af-aee691cebab4" containerName="payload-pre-processing"
Sep 09 20:54:51.205415 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:51.205389    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls"
Sep 09 20:54:51.208817 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:51.208778    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-8cfb58ee-gw-openshift-default-dockercfg-vqfs4\""
Sep 09 20:54:51.230324 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:51.230297    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls"]
Sep 09 20:54:51.291466 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:51.291431    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9793617-0c83-4e0b-9caf-07dc35c43f80-credential-socket\") pod \"e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls\" (UID: \"f9793617-0c83-4e0b-9caf-07dc35c43f80\") " pod="openshift-ingress/e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls"
Sep 09 20:54:51.291601 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:51.291477    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9793617-0c83-4e0b-9caf-07dc35c43f80-workload-socket\") pod \"e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls\" (UID: \"f9793617-0c83-4e0b-9caf-07dc35c43f80\") " pod="openshift-ingress/e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls"
Sep 09 20:54:51.291601 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:51.291567    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9793617-0c83-4e0b-9caf-07dc35c43f80-istio-data\") pod \"e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls\" (UID: \"f9793617-0c83-4e0b-9caf-07dc35c43f80\") " pod="openshift-ingress/e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls"
Sep 09 20:54:51.291694 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:51.291608    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9793617-0c83-4e0b-9caf-07dc35c43f80-istio-token\") pod \"e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls\" (UID: \"f9793617-0c83-4e0b-9caf-07dc35c43f80\") " pod="openshift-ingress/e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls"
Sep 09 20:54:51.291694 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:51.291643    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bdlj\" (UniqueName: \"kubernetes.io/projected/f9793617-0c83-4e0b-9caf-07dc35c43f80-kube-api-access-8bdlj\") pod \"e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls\" (UID: \"f9793617-0c83-4e0b-9caf-07dc35c43f80\") " pod="openshift-ingress/e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls"
Sep 09 20:54:51.291694 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:51.291669    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9793617-0c83-4e0b-9caf-07dc35c43f80-istio-envoy\") pod \"e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls\" (UID: \"f9793617-0c83-4e0b-9caf-07dc35c43f80\") " pod="openshift-ingress/e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls"
Sep 09 20:54:51.291806 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:51.291723    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9793617-0c83-4e0b-9caf-07dc35c43f80-istio-podinfo\") pod \"e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls\" (UID: \"f9793617-0c83-4e0b-9caf-07dc35c43f80\") " pod="openshift-ingress/e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls"
Sep 09 20:54:51.291806 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:51.291746    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9793617-0c83-4e0b-9caf-07dc35c43f80-workload-certs\") pod \"e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls\" (UID: \"f9793617-0c83-4e0b-9caf-07dc35c43f80\") " pod="openshift-ingress/e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls"
Sep 09 20:54:51.291806 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:51.291795    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9793617-0c83-4e0b-9caf-07dc35c43f80-istiod-ca-cert\") pod \"e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls\" (UID: \"f9793617-0c83-4e0b-9caf-07dc35c43f80\") " pod="openshift-ingress/e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls"
Sep 09 20:54:51.392636 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:51.392602    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9793617-0c83-4e0b-9caf-07dc35c43f80-istio-data\") pod \"e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls\" (UID: \"f9793617-0c83-4e0b-9caf-07dc35c43f80\") " pod="openshift-ingress/e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls"
Sep 09 20:54:51.392807 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:51.392650    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9793617-0c83-4e0b-9caf-07dc35c43f80-istio-token\") pod \"e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls\" (UID: \"f9793617-0c83-4e0b-9caf-07dc35c43f80\") " pod="openshift-ingress/e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls"
Sep 09 20:54:51.392807 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:51.392675    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bdlj\" (UniqueName: \"kubernetes.io/projected/f9793617-0c83-4e0b-9caf-07dc35c43f80-kube-api-access-8bdlj\") pod \"e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls\" (UID: \"f9793617-0c83-4e0b-9caf-07dc35c43f80\") " pod="openshift-ingress/e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls"
Sep 09 20:54:51.392807 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:51.392697    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9793617-0c83-4e0b-9caf-07dc35c43f80-istio-envoy\") pod \"e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls\" (UID: \"f9793617-0c83-4e0b-9caf-07dc35c43f80\") " pod="openshift-ingress/e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls"
Sep 09 20:54:51.392807 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:51.392734    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9793617-0c83-4e0b-9caf-07dc35c43f80-istio-podinfo\") pod \"e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls\" (UID: \"f9793617-0c83-4e0b-9caf-07dc35c43f80\") " pod="openshift-ingress/e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls"
Sep 09 20:54:51.392807 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:51.392765    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9793617-0c83-4e0b-9caf-07dc35c43f80-workload-certs\") pod \"e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls\" (UID: \"f9793617-0c83-4e0b-9caf-07dc35c43f80\") " pod="openshift-ingress/e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls"
Sep 09 20:54:51.393070 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:51.392811    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9793617-0c83-4e0b-9caf-07dc35c43f80-istiod-ca-cert\") pod \"e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls\" (UID: \"f9793617-0c83-4e0b-9caf-07dc35c43f80\") " pod="openshift-ingress/e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls"
Sep 09 20:54:51.393070 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:51.392881    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9793617-0c83-4e0b-9caf-07dc35c43f80-credential-socket\") pod \"e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls\" (UID: \"f9793617-0c83-4e0b-9caf-07dc35c43f80\") " pod="openshift-ingress/e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls"
Sep 09 20:54:51.393070 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:51.392917    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9793617-0c83-4e0b-9caf-07dc35c43f80-workload-socket\") pod \"e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls\" (UID: \"f9793617-0c83-4e0b-9caf-07dc35c43f80\") " pod="openshift-ingress/e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls"
Sep 09 20:54:51.393259 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:51.393072    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9793617-0c83-4e0b-9caf-07dc35c43f80-istio-data\") pod \"e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls\" (UID: \"f9793617-0c83-4e0b-9caf-07dc35c43f80\") " pod="openshift-ingress/e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls"
Sep 09 20:54:51.393311 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:51.393276    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9793617-0c83-4e0b-9caf-07dc35c43f80-credential-socket\") pod \"e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls\" (UID: \"f9793617-0c83-4e0b-9caf-07dc35c43f80\") " pod="openshift-ingress/e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls"
Sep 09 20:54:51.393362 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:51.393324    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9793617-0c83-4e0b-9caf-07dc35c43f80-workload-certs\") pod \"e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls\" (UID: \"f9793617-0c83-4e0b-9caf-07dc35c43f80\") " pod="openshift-ingress/e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls"
Sep 09 20:54:51.393443 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:51.393418    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9793617-0c83-4e0b-9caf-07dc35c43f80-workload-socket\") pod \"e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls\" (UID: \"f9793617-0c83-4e0b-9caf-07dc35c43f80\") " pod="openshift-ingress/e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls"
Sep 09 20:54:51.393641 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:51.393623    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9793617-0c83-4e0b-9caf-07dc35c43f80-istiod-ca-cert\") pod \"e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls\" (UID: \"f9793617-0c83-4e0b-9caf-07dc35c43f80\") " pod="openshift-ingress/e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls"
Sep 09 20:54:51.395123 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:51.395080    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9793617-0c83-4e0b-9caf-07dc35c43f80-istio-envoy\") pod \"e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls\" (UID: \"f9793617-0c83-4e0b-9caf-07dc35c43f80\") " pod="openshift-ingress/e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls"
Sep 09 20:54:51.395326 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:51.395309    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9793617-0c83-4e0b-9caf-07dc35c43f80-istio-podinfo\") pod \"e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls\" (UID: \"f9793617-0c83-4e0b-9caf-07dc35c43f80\") " pod="openshift-ingress/e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls"
Sep 09 20:54:51.409658 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:51.409630    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9793617-0c83-4e0b-9caf-07dc35c43f80-istio-token\") pod \"e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls\" (UID: \"f9793617-0c83-4e0b-9caf-07dc35c43f80\") " pod="openshift-ingress/e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls"
Sep 09 20:54:51.416375 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:51.416348    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bdlj\" (UniqueName: \"kubernetes.io/projected/f9793617-0c83-4e0b-9caf-07dc35c43f80-kube-api-access-8bdlj\") pod \"e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls\" (UID: \"f9793617-0c83-4e0b-9caf-07dc35c43f80\") " pod="openshift-ingress/e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls"
Sep 09 20:54:51.516604 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:51.516568    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls"
Sep 09 20:54:51.809000 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:51.808973    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls"]
Sep 09 20:54:51.810090 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:54:51.810066    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9793617_0c83_4e0b_9caf_07dc35c43f80.slice/crio-5068e05d734cda41cf2a3e3aa2205d3cdba7ca639daa6d49542b0dbe1680cf8f WatchSource:0}: Error finding container 5068e05d734cda41cf2a3e3aa2205d3cdba7ca639daa6d49542b0dbe1680cf8f: Status 404 returned error can't find the container with id 5068e05d734cda41cf2a3e3aa2205d3cdba7ca639daa6d49542b0dbe1680cf8f
Sep 09 20:54:51.812474 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:51.812436    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:54:51.812561 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:51.812507    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:54:51.812599 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:51.812561    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:54:52.046016 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:52.045971    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls" event={"ID":"f9793617-0c83-4e0b-9caf-07dc35c43f80","Type":"ContainerStarted","Data":"e5a861e0a42587623bc05dbbd040a39f7ca860ec1ec1de196716a5872954ce43"}
Sep 09 20:54:52.046506 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:52.046086    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls" event={"ID":"f9793617-0c83-4e0b-9caf-07dc35c43f80","Type":"ContainerStarted","Data":"5068e05d734cda41cf2a3e3aa2205d3cdba7ca639daa6d49542b0dbe1680cf8f"}
Sep 09 20:54:52.082944 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:52.082835    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls" podStartSLOduration=1.082814285 podStartE2EDuration="1.082814285s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:54:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:54:52.082704664 +0000 UTC m=+853.710332538" watchObservedRunningTime="2026-09-09 20:54:52.082814285 +0000 UTC m=+853.710442160"
Sep 09 20:54:52.516881 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:52.516841    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls"
Sep 09 20:54:52.522094 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:52.522069    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls"
Sep 09 20:54:53.055595 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:53.055556    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls"
Sep 09 20:54:53.057008 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:53.056979    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls"
Sep 09 20:54:54.657728 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:54.657695    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-8cfb58ee-5958d57f66-76h6c"]
Sep 09 20:54:54.661723 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:54.661705    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-8cfb58ee-5958d57f66-76h6c"
Sep 09 20:54:54.665032 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:54.665011    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-8cfb58ee\""
Sep 09 20:54:54.665153 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:54.665080    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-8cfb58ee-dockercfg-z89tz\""
Sep 09 20:54:54.681976 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:54.681940    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-8cfb58ee-5958d57f66-76h6c"]
Sep 09 20:54:54.746071 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:54.745980    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-8cfb58ee-c87845758-wxjcr"]
Sep 09 20:54:54.750385 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:54.750357    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-8cfb58ee-c87845758-wxjcr"
Sep 09 20:54:54.771264 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:54.771015    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-8cfb58ee-c87845758-wxjcr"]
Sep 09 20:54:54.838797 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:54.838760    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9q5nn\" (UniqueName: \"kubernetes.io/projected/cf255fc4-e303-44ad-8b78-a0323af81341-kube-api-access-9q5nn\") pod \"payload-pre-processing-e2e-migrate-8cfb58ee-5958d57f66-76h6c\" (UID: \"cf255fc4-e303-44ad-8b78-a0323af81341\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-8cfb58ee-5958d57f66-76h6c"
Sep 09 20:54:54.838965 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:54.838880    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf255fc4-e303-44ad-8b78-a0323af81341-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-8cfb58ee-5958d57f66-76h6c\" (UID: \"cf255fc4-e303-44ad-8b78-a0323af81341\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-8cfb58ee-5958d57f66-76h6c"
Sep 09 20:54:54.939613 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:54.939522    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9q5nn\" (UniqueName: \"kubernetes.io/projected/cf255fc4-e303-44ad-8b78-a0323af81341-kube-api-access-9q5nn\") pod \"payload-pre-processing-e2e-migrate-8cfb58ee-5958d57f66-76h6c\" (UID: \"cf255fc4-e303-44ad-8b78-a0323af81341\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-8cfb58ee-5958d57f66-76h6c"
Sep 09 20:54:54.939613 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:54.939578    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcvth\" (UniqueName: \"kubernetes.io/projected/4e7acc8f-f1f9-4f22-84b6-d83c98899ec7-kube-api-access-gcvth\") pod \"payload-processing-e2e-migrate-8cfb58ee-c87845758-wxjcr\" (UID: \"4e7acc8f-f1f9-4f22-84b6-d83c98899ec7\") " pod="openshift-ingress/payload-processing-e2e-migrate-8cfb58ee-c87845758-wxjcr"
Sep 09 20:54:54.939792 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:54.939651    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf255fc4-e303-44ad-8b78-a0323af81341-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-8cfb58ee-5958d57f66-76h6c\" (UID: \"cf255fc4-e303-44ad-8b78-a0323af81341\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-8cfb58ee-5958d57f66-76h6c"
Sep 09 20:54:54.939792 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:54.939695    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4e7acc8f-f1f9-4f22-84b6-d83c98899ec7-plugins-config-volume\") pod \"payload-processing-e2e-migrate-8cfb58ee-c87845758-wxjcr\" (UID: \"4e7acc8f-f1f9-4f22-84b6-d83c98899ec7\") " pod="openshift-ingress/payload-processing-e2e-migrate-8cfb58ee-c87845758-wxjcr"
Sep 09 20:54:54.940594 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:54.940571    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf255fc4-e303-44ad-8b78-a0323af81341-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-8cfb58ee-5958d57f66-76h6c\" (UID: \"cf255fc4-e303-44ad-8b78-a0323af81341\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-8cfb58ee-5958d57f66-76h6c"
Sep 09 20:54:54.954205 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:54.954183    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9q5nn\" (UniqueName: \"kubernetes.io/projected/cf255fc4-e303-44ad-8b78-a0323af81341-kube-api-access-9q5nn\") pod \"payload-pre-processing-e2e-migrate-8cfb58ee-5958d57f66-76h6c\" (UID: \"cf255fc4-e303-44ad-8b78-a0323af81341\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-8cfb58ee-5958d57f66-76h6c"
Sep 09 20:54:54.973203 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:54.973169    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-8cfb58ee-5958d57f66-76h6c"
Sep 09 20:54:55.041614 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:55.041570    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4e7acc8f-f1f9-4f22-84b6-d83c98899ec7-plugins-config-volume\") pod \"payload-processing-e2e-migrate-8cfb58ee-c87845758-wxjcr\" (UID: \"4e7acc8f-f1f9-4f22-84b6-d83c98899ec7\") " pod="openshift-ingress/payload-processing-e2e-migrate-8cfb58ee-c87845758-wxjcr"
Sep 09 20:54:55.041783 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:55.041700    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcvth\" (UniqueName: \"kubernetes.io/projected/4e7acc8f-f1f9-4f22-84b6-d83c98899ec7-kube-api-access-gcvth\") pod \"payload-processing-e2e-migrate-8cfb58ee-c87845758-wxjcr\" (UID: \"4e7acc8f-f1f9-4f22-84b6-d83c98899ec7\") " pod="openshift-ingress/payload-processing-e2e-migrate-8cfb58ee-c87845758-wxjcr"
Sep 09 20:54:55.043004 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:55.042953    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4e7acc8f-f1f9-4f22-84b6-d83c98899ec7-plugins-config-volume\") pod \"payload-processing-e2e-migrate-8cfb58ee-c87845758-wxjcr\" (UID: \"4e7acc8f-f1f9-4f22-84b6-d83c98899ec7\") " pod="openshift-ingress/payload-processing-e2e-migrate-8cfb58ee-c87845758-wxjcr"
Sep 09 20:54:55.050591 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:55.050565    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcvth\" (UniqueName: \"kubernetes.io/projected/4e7acc8f-f1f9-4f22-84b6-d83c98899ec7-kube-api-access-gcvth\") pod \"payload-processing-e2e-migrate-8cfb58ee-c87845758-wxjcr\" (UID: \"4e7acc8f-f1f9-4f22-84b6-d83c98899ec7\") " pod="openshift-ingress/payload-processing-e2e-migrate-8cfb58ee-c87845758-wxjcr"
Sep 09 20:54:55.072142 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:55.071935    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-8cfb58ee-c87845758-wxjcr"
Sep 09 20:54:55.270259 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:55.270152    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-8cfb58ee-5958d57f66-76h6c"]
Sep 09 20:54:55.272699 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:54:55.272663    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf255fc4_e303_44ad_8b78_a0323af81341.slice/crio-68b1781621e6915bb36c6499036da50c1cfdaa771541b899ca41aa42954c413c WatchSource:0}: Error finding container 68b1781621e6915bb36c6499036da50c1cfdaa771541b899ca41aa42954c413c: Status 404 returned error can't find the container with id 68b1781621e6915bb36c6499036da50c1cfdaa771541b899ca41aa42954c413c
Sep 09 20:54:55.364930 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:55.364906    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m"
Sep 09 20:54:55.383291 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:55.383255    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-8cfb58ee-c87845758-wxjcr"]
Sep 09 20:54:55.384870 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:54:55.384838    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e7acc8f_f1f9_4f22_84b6_d83c98899ec7.slice/crio-5e1799946afc218043b404eab4e021e192c376b297177915be172fee2eaa8ea3 WatchSource:0}: Error finding container 5e1799946afc218043b404eab4e021e192c376b297177915be172fee2eaa8ea3: Status 404 returned error can't find the container with id 5e1799946afc218043b404eab4e021e192c376b297177915be172fee2eaa8ea3
Sep 09 20:54:55.547566 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:55.547533    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d39d8f5a-1c5c-4b84-a4de-d65511571543-istiod-ca-cert\") pod \"d39d8f5a-1c5c-4b84-a4de-d65511571543\" (UID: \"d39d8f5a-1c5c-4b84-a4de-d65511571543\") "
Sep 09 20:54:55.547754 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:55.547595    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d39d8f5a-1c5c-4b84-a4de-d65511571543-workload-socket\") pod \"d39d8f5a-1c5c-4b84-a4de-d65511571543\" (UID: \"d39d8f5a-1c5c-4b84-a4de-d65511571543\") "
Sep 09 20:54:55.547754 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:55.547615    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d39d8f5a-1c5c-4b84-a4de-d65511571543-istio-podinfo\") pod \"d39d8f5a-1c5c-4b84-a4de-d65511571543\" (UID: \"d39d8f5a-1c5c-4b84-a4de-d65511571543\") "
Sep 09 20:54:55.547754 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:55.547649    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fw452\" (UniqueName: \"kubernetes.io/projected/d39d8f5a-1c5c-4b84-a4de-d65511571543-kube-api-access-fw452\") pod \"d39d8f5a-1c5c-4b84-a4de-d65511571543\" (UID: \"d39d8f5a-1c5c-4b84-a4de-d65511571543\") "
Sep 09 20:54:55.547754 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:55.547669    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d39d8f5a-1c5c-4b84-a4de-d65511571543-istio-token\") pod \"d39d8f5a-1c5c-4b84-a4de-d65511571543\" (UID: \"d39d8f5a-1c5c-4b84-a4de-d65511571543\") "
Sep 09 20:54:55.547754 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:55.547684    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d39d8f5a-1c5c-4b84-a4de-d65511571543-credential-socket\") pod \"d39d8f5a-1c5c-4b84-a4de-d65511571543\" (UID: \"d39d8f5a-1c5c-4b84-a4de-d65511571543\") "
Sep 09 20:54:55.547754 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:55.547745    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d39d8f5a-1c5c-4b84-a4de-d65511571543-workload-certs\") pod \"d39d8f5a-1c5c-4b84-a4de-d65511571543\" (UID: \"d39d8f5a-1c5c-4b84-a4de-d65511571543\") "
Sep 09 20:54:55.548068 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:55.547797    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d39d8f5a-1c5c-4b84-a4de-d65511571543-istio-data\") pod \"d39d8f5a-1c5c-4b84-a4de-d65511571543\" (UID: \"d39d8f5a-1c5c-4b84-a4de-d65511571543\") "
Sep 09 20:54:55.548068 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:55.547880    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d39d8f5a-1c5c-4b84-a4de-d65511571543-istio-envoy\") pod \"d39d8f5a-1c5c-4b84-a4de-d65511571543\" (UID: \"d39d8f5a-1c5c-4b84-a4de-d65511571543\") "
Sep 09 20:54:55.548068 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:55.547949    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d39d8f5a-1c5c-4b84-a4de-d65511571543-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d39d8f5a-1c5c-4b84-a4de-d65511571543" (UID: "d39d8f5a-1c5c-4b84-a4de-d65511571543"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:54:55.548264 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:55.548152    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d39d8f5a-1c5c-4b84-a4de-d65511571543-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d39d8f5a-1c5c-4b84-a4de-d65511571543" (UID: "d39d8f5a-1c5c-4b84-a4de-d65511571543"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:54:55.548264 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:55.548248    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d39d8f5a-1c5c-4b84-a4de-d65511571543-istio-data" (OuterVolumeSpecName: "istio-data") pod "d39d8f5a-1c5c-4b84-a4de-d65511571543" (UID: "d39d8f5a-1c5c-4b84-a4de-d65511571543"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:54:55.548363 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:55.548330    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d39d8f5a-1c5c-4b84-a4de-d65511571543-workload-certs\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:54:55.548363 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:55.548329    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d39d8f5a-1c5c-4b84-a4de-d65511571543-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d39d8f5a-1c5c-4b84-a4de-d65511571543" (UID: "d39d8f5a-1c5c-4b84-a4de-d65511571543"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:54:55.548363 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:55.548348    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d39d8f5a-1c5c-4b84-a4de-d65511571543-istio-data\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:54:55.548363 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:55.548362    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d39d8f5a-1c5c-4b84-a4de-d65511571543-istiod-ca-cert\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:54:55.548558 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:55.548374    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d39d8f5a-1c5c-4b84-a4de-d65511571543-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d39d8f5a-1c5c-4b84-a4de-d65511571543" (UID: "d39d8f5a-1c5c-4b84-a4de-d65511571543"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:54:55.550478 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:55.550430    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d39d8f5a-1c5c-4b84-a4de-d65511571543-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d39d8f5a-1c5c-4b84-a4de-d65511571543" (UID: "d39d8f5a-1c5c-4b84-a4de-d65511571543"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 20:54:55.550620 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:55.550595    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d39d8f5a-1c5c-4b84-a4de-d65511571543-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d39d8f5a-1c5c-4b84-a4de-d65511571543" (UID: "d39d8f5a-1c5c-4b84-a4de-d65511571543"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:54:55.550710 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:55.550687    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d39d8f5a-1c5c-4b84-a4de-d65511571543-kube-api-access-fw452" (OuterVolumeSpecName: "kube-api-access-fw452") pod "d39d8f5a-1c5c-4b84-a4de-d65511571543" (UID: "d39d8f5a-1c5c-4b84-a4de-d65511571543"). InnerVolumeSpecName "kube-api-access-fw452". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:54:55.550785 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:55.550719    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d39d8f5a-1c5c-4b84-a4de-d65511571543-istio-token" (OuterVolumeSpecName: "istio-token") pod "d39d8f5a-1c5c-4b84-a4de-d65511571543" (UID: "d39d8f5a-1c5c-4b84-a4de-d65511571543"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:54:55.650173 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:55.650033    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d39d8f5a-1c5c-4b84-a4de-d65511571543-istio-envoy\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:54:55.650173 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:55.650078    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d39d8f5a-1c5c-4b84-a4de-d65511571543-workload-socket\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:54:55.650173 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:55.650092    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d39d8f5a-1c5c-4b84-a4de-d65511571543-istio-podinfo\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:54:55.650173 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:55.650120    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fw452\" (UniqueName: \"kubernetes.io/projected/d39d8f5a-1c5c-4b84-a4de-d65511571543-kube-api-access-fw452\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:54:55.650173 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:55.650134    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d39d8f5a-1c5c-4b84-a4de-d65511571543-istio-token\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:54:55.650173 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:55.650148    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d39d8f5a-1c5c-4b84-a4de-d65511571543-credential-socket\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:54:56.075522 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:56.075482    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-8cfb58ee-5958d57f66-76h6c" event={"ID":"cf255fc4-e303-44ad-8b78-a0323af81341","Type":"ContainerStarted","Data":"2d6e59fe848405fbeaa3bff0efa5f1456e55e289453343b34facb4118fe8dddb"}
Sep 09 20:54:56.076006 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:56.075535    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-8cfb58ee-5958d57f66-76h6c" event={"ID":"cf255fc4-e303-44ad-8b78-a0323af81341","Type":"ContainerStarted","Data":"68b1781621e6915bb36c6499036da50c1cfdaa771541b899ca41aa42954c413c"}
Sep 09 20:54:56.076006 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:56.075893    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-8cfb58ee-5958d57f66-76h6c"
Sep 09 20:54:56.077697 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:56.077674    2581 generic.go:356] "Generic (PLEG): container finished" podID="d39d8f5a-1c5c-4b84-a4de-d65511571543" containerID="b26d28db32c19ba59481b451abcfa840b9ce1d53b2404675e32c159fc90f4932" exitCode=0
Sep 09 20:54:56.077801 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:56.077712    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m"
Sep 09 20:54:56.077801 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:56.077732    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m" event={"ID":"d39d8f5a-1c5c-4b84-a4de-d65511571543","Type":"ContainerDied","Data":"b26d28db32c19ba59481b451abcfa840b9ce1d53b2404675e32c159fc90f4932"}
Sep 09 20:54:56.077801 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:56.077777    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m" event={"ID":"d39d8f5a-1c5c-4b84-a4de-d65511571543","Type":"ContainerDied","Data":"9214c21c0ff129de76db45ddd2c43668e715be1e2d9ace123d78466d2e199b36"}
Sep 09 20:54:56.077801 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:56.077801    2581 scope.go:117] "RemoveContainer" containerID="b26d28db32c19ba59481b451abcfa840b9ce1d53b2404675e32c159fc90f4932"
Sep 09 20:54:56.079378 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:56.079342    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-8cfb58ee-c87845758-wxjcr" event={"ID":"4e7acc8f-f1f9-4f22-84b6-d83c98899ec7","Type":"ContainerStarted","Data":"81816dddd67a86937b63f07b0c2dec3f6bc20c54fa302f8650af61684134cec2"}
Sep 09 20:54:56.079378 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:56.079377    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-8cfb58ee-c87845758-wxjcr" event={"ID":"4e7acc8f-f1f9-4f22-84b6-d83c98899ec7","Type":"ContainerStarted","Data":"5e1799946afc218043b404eab4e021e192c376b297177915be172fee2eaa8ea3"}
Sep 09 20:54:56.079559 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:56.079496    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-8cfb58ee-c87845758-wxjcr"
Sep 09 20:54:56.087449 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:56.087435    2581 scope.go:117] "RemoveContainer" containerID="cdb0267ebb031e24fc95797a4d62c20b38269e48c451dc33e012bb131160dbb6"
Sep 09 20:54:56.095793 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:56.095612    2581 scope.go:117] "RemoveContainer" containerID="b26d28db32c19ba59481b451abcfa840b9ce1d53b2404675e32c159fc90f4932"
Sep 09 20:54:56.095940 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:54:56.095916    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b26d28db32c19ba59481b451abcfa840b9ce1d53b2404675e32c159fc90f4932\": container with ID starting with b26d28db32c19ba59481b451abcfa840b9ce1d53b2404675e32c159fc90f4932 not found: ID does not exist" containerID="b26d28db32c19ba59481b451abcfa840b9ce1d53b2404675e32c159fc90f4932"
Sep 09 20:54:56.095998 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:56.095974    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b26d28db32c19ba59481b451abcfa840b9ce1d53b2404675e32c159fc90f4932"} err="failed to get container status \"b26d28db32c19ba59481b451abcfa840b9ce1d53b2404675e32c159fc90f4932\": rpc error: code = NotFound desc = could not find container \"b26d28db32c19ba59481b451abcfa840b9ce1d53b2404675e32c159fc90f4932\": container with ID starting with b26d28db32c19ba59481b451abcfa840b9ce1d53b2404675e32c159fc90f4932 not found: ID does not exist"
Sep 09 20:54:56.096051 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:56.096000    2581 scope.go:117] "RemoveContainer" containerID="cdb0267ebb031e24fc95797a4d62c20b38269e48c451dc33e012bb131160dbb6"
Sep 09 20:54:56.096287 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:54:56.096266    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cdb0267ebb031e24fc95797a4d62c20b38269e48c451dc33e012bb131160dbb6\": container with ID starting with cdb0267ebb031e24fc95797a4d62c20b38269e48c451dc33e012bb131160dbb6 not found: ID does not exist" containerID="cdb0267ebb031e24fc95797a4d62c20b38269e48c451dc33e012bb131160dbb6"
Sep 09 20:54:56.096338 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:56.096296    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cdb0267ebb031e24fc95797a4d62c20b38269e48c451dc33e012bb131160dbb6"} err="failed to get container status \"cdb0267ebb031e24fc95797a4d62c20b38269e48c451dc33e012bb131160dbb6\": rpc error: code = NotFound desc = could not find container \"cdb0267ebb031e24fc95797a4d62c20b38269e48c451dc33e012bb131160dbb6\": container with ID starting with cdb0267ebb031e24fc95797a4d62c20b38269e48c451dc33e012bb131160dbb6 not found: ID does not exist"
Sep 09 20:54:56.098234 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:56.098190    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-8cfb58ee-5958d57f66-76h6c" podStartSLOduration=2.09817455 podStartE2EDuration="2.09817455s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:54:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:54:56.094901626 +0000 UTC m=+857.722529499" watchObservedRunningTime="2026-09-09 20:54:56.09817455 +0000 UTC m=+857.725802424"
Sep 09 20:54:56.121978 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:56.118819    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-8cfb58ee-c87845758-wxjcr" podStartSLOduration=2.118798836 podStartE2EDuration="2.118798836s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:54:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:54:56.117711565 +0000 UTC m=+857.745339441" watchObservedRunningTime="2026-09-09 20:54:56.118798836 +0000 UTC m=+857.746426710"
Sep 09 20:54:56.141267 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:56.141230    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m"]
Sep 09 20:54:56.148502 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:56.148469    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-04cfc947-openshift-default-b89b6cb4-l2v9m"]
Sep 09 20:54:56.826080 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:54:56.826036    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d39d8f5a-1c5c-4b84-a4de-d65511571543" path="/var/lib/kubelet/pods/d39d8f5a-1c5c-4b84-a4de-d65511571543/volumes"
Sep 09 20:55:02.113253 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:02.113222    2581 generic.go:356] "Generic (PLEG): container finished" podID="67436bca-5ef3-400f-9892-6d1e3403349b" containerID="81fa1fe91f1684b322720d3bdeed556f730e4e2a53df649adcb43beaa9c74660" exitCode=0
Sep 09 20:55:02.113674 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:02.113302    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx" event={"ID":"67436bca-5ef3-400f-9892-6d1e3403349b","Type":"ContainerDied","Data":"81fa1fe91f1684b322720d3bdeed556f730e4e2a53df649adcb43beaa9c74660"}
Sep 09 20:55:02.113784 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:02.113767    2581 scope.go:117] "RemoveContainer" containerID="81fa1fe91f1684b322720d3bdeed556f730e4e2a53df649adcb43beaa9c74660"
Sep 09 20:55:02.115093 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:02.115068    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:55:02.115197 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:02.115149    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:55:02.115235 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:02.115222    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:55:03.124644 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:03.124602    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx" event={"ID":"67436bca-5ef3-400f-9892-6d1e3403349b","Type":"ContainerStarted","Data":"244a3d0fa614f74a6432e2e15e01f17d36ec41e9e57c66e305ef3b9ed2e5c14e"}
Sep 09 20:55:04.063116 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:04.063058    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx"
Sep 09 20:55:04.063317 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:04.063295    2581 patch_prober.go:28] interesting pod/e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body=
Sep 09 20:55:04.063394 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:04.063354    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx" podUID="67436bca-5ef3-400f-9892-6d1e3403349b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused"
Sep 09 20:55:05.063508 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:05.063477    2581 patch_prober.go:28] interesting pod/e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body=
Sep 09 20:55:05.064017 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:05.063989    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx" podUID="67436bca-5ef3-400f-9892-6d1e3403349b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused"
Sep 09 20:55:06.063655 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:06.063621    2581 patch_prober.go:28] interesting pod/e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body=
Sep 09 20:55:06.064167 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:06.063698    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx" podUID="67436bca-5ef3-400f-9892-6d1e3403349b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused"
Sep 09 20:55:07.063608 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:07.063566    2581 patch_prober.go:28] interesting pod/e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body=
Sep 09 20:55:07.063794 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:07.063629    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx" podUID="67436bca-5ef3-400f-9892-6d1e3403349b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused"
Sep 09 20:55:07.088499 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:07.088471    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-8cfb58ee-5958d57f66-76h6c"
Sep 09 20:55:07.088886 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:07.088867    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-8cfb58ee-c87845758-wxjcr"
Sep 09 20:55:08.063310 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:08.063268    2581 patch_prober.go:28] interesting pod/e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body=
Sep 09 20:55:08.063471 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:08.063330    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx" podUID="67436bca-5ef3-400f-9892-6d1e3403349b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused"
Sep 09 20:55:09.066830 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:09.066801    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx"
Sep 09 20:55:09.067227 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:09.067141    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx"
Sep 09 20:55:09.067796 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:09.067778    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx"
Sep 09 20:55:13.200777 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:13.200743    2581 generic.go:356] "Generic (PLEG): container finished" podID="f9793617-0c83-4e0b-9caf-07dc35c43f80" containerID="e5a861e0a42587623bc05dbbd040a39f7ca860ec1ec1de196716a5872954ce43" exitCode=0
Sep 09 20:55:13.201233 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:13.200813    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls" event={"ID":"f9793617-0c83-4e0b-9caf-07dc35c43f80","Type":"ContainerDied","Data":"e5a861e0a42587623bc05dbbd040a39f7ca860ec1ec1de196716a5872954ce43"}
Sep 09 20:55:13.201233 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:13.201227    2581 scope.go:117] "RemoveContainer" containerID="e5a861e0a42587623bc05dbbd040a39f7ca860ec1ec1de196716a5872954ce43"
Sep 09 20:55:13.202382 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:13.202353    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:55:13.202450 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:13.202422    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:55:13.202486 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:13.202454    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:55:14.212399 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:14.212351    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls" event={"ID":"f9793617-0c83-4e0b-9caf-07dc35c43f80","Type":"ContainerStarted","Data":"99a8bd43927aed48e7df8133f404ffb125fdb79112d3dcb418f21bd10a5a4ce2"}
Sep 09 20:55:14.521125 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:14.517412    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls"
Sep 09 20:55:14.521125 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:14.517577    2581 patch_prober.go:28] interesting pod/e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Sep 09 20:55:14.521125 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:14.517628    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls" podUID="f9793617-0c83-4e0b-9caf-07dc35c43f80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Sep 09 20:55:15.517838 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:15.517788    2581 patch_prober.go:28] interesting pod/e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Sep 09 20:55:15.518388 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:15.517864    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls" podUID="f9793617-0c83-4e0b-9caf-07dc35c43f80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Sep 09 20:55:16.518037 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:16.517996    2581 patch_prober.go:28] interesting pod/e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Sep 09 20:55:16.518669 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:16.518634    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls" podUID="f9793617-0c83-4e0b-9caf-07dc35c43f80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Sep 09 20:55:17.517590 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:17.517541    2581 patch_prober.go:28] interesting pod/e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Sep 09 20:55:17.517797 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:17.517615    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls" podUID="f9793617-0c83-4e0b-9caf-07dc35c43f80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Sep 09 20:55:18.517646 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:18.517604    2581 patch_prober.go:28] interesting pod/e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Sep 09 20:55:18.518266 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:18.518232    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls" podUID="f9793617-0c83-4e0b-9caf-07dc35c43f80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Sep 09 20:55:18.579533 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:18.579498    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8cfb58ee-r66hr"]
Sep 09 20:55:18.580025 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:18.580005    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d39d8f5a-1c5c-4b84-a4de-d65511571543" containerName="istio-proxy"
Sep 09 20:55:18.580140 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:18.580028    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d39d8f5a-1c5c-4b84-a4de-d65511571543" containerName="istio-proxy"
Sep 09 20:55:18.580220 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:18.580205    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d39d8f5a-1c5c-4b84-a4de-d65511571543" containerName="istio-proxy"
Sep 09 20:55:18.580283 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:18.580223    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d39d8f5a-1c5c-4b84-a4de-d65511571543" containerName="istio-proxy"
Sep 09 20:55:18.582925 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:18.582904    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8cfb58ee-r66hr"
Sep 09 20:55:18.585660 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:18.585642    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-glz44\""
Sep 09 20:55:18.591034 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:18.590920    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8cfb58ee-r66hr"]
Sep 09 20:55:18.687596 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:18.687553    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/88447789-c7ad-4235-addd-a0139f8b5028-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8cfb58ee-r66hr\" (UID: \"88447789-c7ad-4235-addd-a0139f8b5028\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8cfb58ee-r66hr"
Sep 09 20:55:18.788671 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:18.788573    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/88447789-c7ad-4235-addd-a0139f8b5028-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8cfb58ee-r66hr\" (UID: \"88447789-c7ad-4235-addd-a0139f8b5028\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8cfb58ee-r66hr"
Sep 09 20:55:18.789273 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:18.789247    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/88447789-c7ad-4235-addd-a0139f8b5028-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8cfb58ee-r66hr\" (UID: \"88447789-c7ad-4235-addd-a0139f8b5028\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8cfb58ee-r66hr"
Sep 09 20:55:18.894461 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:18.894416    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8cfb58ee-r66hr"
Sep 09 20:55:19.184741 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:19.184718    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8cfb58ee-r66hr"]
Sep 09 20:55:19.187008 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:55:19.186977    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88447789_c7ad_4235_addd_a0139f8b5028.slice/crio-92754fbdc0b3c44d7cf98cfa08085d409dea8c086073310101ccd4823fd2388b WatchSource:0}: Error finding container 92754fbdc0b3c44d7cf98cfa08085d409dea8c086073310101ccd4823fd2388b: Status 404 returned error can't find the container with id 92754fbdc0b3c44d7cf98cfa08085d409dea8c086073310101ccd4823fd2388b
Sep 09 20:55:19.244976 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:19.244943    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8cfb58ee-r66hr" event={"ID":"88447789-c7ad-4235-addd-a0139f8b5028","Type":"ContainerStarted","Data":"92754fbdc0b3c44d7cf98cfa08085d409dea8c086073310101ccd4823fd2388b"}
Sep 09 20:55:19.521235 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:19.521211    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls"
Sep 09 20:55:19.521594 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:19.521574    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls"
Sep 09 20:55:19.522355 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:19.522338    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls"
Sep 09 20:55:20.252155 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:20.252125    2581 generic.go:356] "Generic (PLEG): container finished" podID="88447789-c7ad-4235-addd-a0139f8b5028" containerID="393c6d339f5d5fa3bc5594b995f9eee753ae7519dfd71db61ba56015e233d104" exitCode=0
Sep 09 20:55:20.252338 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:20.252179    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8cfb58ee-r66hr" event={"ID":"88447789-c7ad-4235-addd-a0139f8b5028","Type":"ContainerDied","Data":"393c6d339f5d5fa3bc5594b995f9eee753ae7519dfd71db61ba56015e233d104"}
Sep 09 20:55:21.397069 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:21.397045    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8cfb58ee-r66hr"
Sep 09 20:55:21.519134 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:21.519018    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/88447789-c7ad-4235-addd-a0139f8b5028-maas-api-service-ca\") pod \"88447789-c7ad-4235-addd-a0139f8b5028\" (UID: \"88447789-c7ad-4235-addd-a0139f8b5028\") "
Sep 09 20:55:21.519421 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:21.519397    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/88447789-c7ad-4235-addd-a0139f8b5028-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "88447789-c7ad-4235-addd-a0139f8b5028" (UID: "88447789-c7ad-4235-addd-a0139f8b5028"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:55:21.620639 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:21.620602    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/88447789-c7ad-4235-addd-a0139f8b5028-maas-api-service-ca\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:55:22.264253 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:22.264217    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8cfb58ee-r66hr" event={"ID":"88447789-c7ad-4235-addd-a0139f8b5028","Type":"ContainerDied","Data":"92754fbdc0b3c44d7cf98cfa08085d409dea8c086073310101ccd4823fd2388b"}
Sep 09 20:55:22.264253 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:22.264244    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8cfb58ee-r66hr"
Sep 09 20:55:22.264510 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:22.264265    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="92754fbdc0b3c44d7cf98cfa08085d409dea8c086073310101ccd4823fd2388b"
Sep 09 20:55:24.587941 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:24.587909    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-8cfb58ee-c87845758-wxjcr"]
Sep 09 20:55:24.588546 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:24.588130    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-8cfb58ee-c87845758-wxjcr" podUID="4e7acc8f-f1f9-4f22-84b6-d83c98899ec7" containerName="payload-processing" containerID="cri-o://81816dddd67a86937b63f07b0c2dec3f6bc20c54fa302f8650af61684134cec2" gracePeriod=30
Sep 09 20:55:24.599758 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:24.599728    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-8cfb58ee-5958d57f66-76h6c"]
Sep 09 20:55:24.599998 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:24.599955    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-8cfb58ee-5958d57f66-76h6c" podUID="cf255fc4-e303-44ad-8b78-a0323af81341" containerName="payload-pre-processing" containerID="cri-o://2d6e59fe848405fbeaa3bff0efa5f1456e55e289453343b34facb4118fe8dddb" gracePeriod=30
Sep 09 20:55:24.780369 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:24.780338    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8cfb58ee-r66hr"]
Sep 09 20:55:24.783741 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:24.783719    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8cfb58ee-r66hr"]
Sep 09 20:55:24.823446 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:24.823410    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="88447789-c7ad-4235-addd-a0139f8b5028" path="/var/lib/kubelet/pods/88447789-c7ad-4235-addd-a0139f8b5028/volumes"
Sep 09 20:55:24.876138 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:24.876111    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-8cfb58ee-c87845758-wxjcr"
Sep 09 20:55:24.879227 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:24.879207    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-8cfb58ee-5958d57f66-76h6c"
Sep 09 20:55:24.920970 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:24.920255    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls"]
Sep 09 20:55:24.925347 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:24.925311    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls" podUID="f9793617-0c83-4e0b-9caf-07dc35c43f80" containerName="istio-proxy" containerID="cri-o://99a8bd43927aed48e7df8133f404ffb125fdb79112d3dcb418f21bd10a5a4ce2" gracePeriod=30
Sep 09 20:55:24.956416 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:24.956389    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4e7acc8f-f1f9-4f22-84b6-d83c98899ec7-plugins-config-volume\") pod \"4e7acc8f-f1f9-4f22-84b6-d83c98899ec7\" (UID: \"4e7acc8f-f1f9-4f22-84b6-d83c98899ec7\") "
Sep 09 20:55:24.956544 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:24.956437    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9q5nn\" (UniqueName: \"kubernetes.io/projected/cf255fc4-e303-44ad-8b78-a0323af81341-kube-api-access-9q5nn\") pod \"cf255fc4-e303-44ad-8b78-a0323af81341\" (UID: \"cf255fc4-e303-44ad-8b78-a0323af81341\") "
Sep 09 20:55:24.956544 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:24.956457    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gcvth\" (UniqueName: \"kubernetes.io/projected/4e7acc8f-f1f9-4f22-84b6-d83c98899ec7-kube-api-access-gcvth\") pod \"4e7acc8f-f1f9-4f22-84b6-d83c98899ec7\" (UID: \"4e7acc8f-f1f9-4f22-84b6-d83c98899ec7\") "
Sep 09 20:55:24.956614 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:24.956575    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf255fc4-e303-44ad-8b78-a0323af81341-plugins-config-volume\") pod \"cf255fc4-e303-44ad-8b78-a0323af81341\" (UID: \"cf255fc4-e303-44ad-8b78-a0323af81341\") "
Sep 09 20:55:24.956869 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:24.956837    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e7acc8f-f1f9-4f22-84b6-d83c98899ec7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4e7acc8f-f1f9-4f22-84b6-d83c98899ec7" (UID: "4e7acc8f-f1f9-4f22-84b6-d83c98899ec7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:55:24.956995 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:24.956927    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf255fc4-e303-44ad-8b78-a0323af81341-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "cf255fc4-e303-44ad-8b78-a0323af81341" (UID: "cf255fc4-e303-44ad-8b78-a0323af81341"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:55:24.957130 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:24.957097    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf255fc4-e303-44ad-8b78-a0323af81341-plugins-config-volume\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:55:24.957194 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:24.957136    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4e7acc8f-f1f9-4f22-84b6-d83c98899ec7-plugins-config-volume\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:55:24.958632 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:24.958611    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf255fc4-e303-44ad-8b78-a0323af81341-kube-api-access-9q5nn" (OuterVolumeSpecName: "kube-api-access-9q5nn") pod "cf255fc4-e303-44ad-8b78-a0323af81341" (UID: "cf255fc4-e303-44ad-8b78-a0323af81341"). InnerVolumeSpecName "kube-api-access-9q5nn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:55:24.959147 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:24.959130    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e7acc8f-f1f9-4f22-84b6-d83c98899ec7-kube-api-access-gcvth" (OuterVolumeSpecName: "kube-api-access-gcvth") pod "4e7acc8f-f1f9-4f22-84b6-d83c98899ec7" (UID: "4e7acc8f-f1f9-4f22-84b6-d83c98899ec7"). InnerVolumeSpecName "kube-api-access-gcvth". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:55:25.057747 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:25.057713    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9q5nn\" (UniqueName: \"kubernetes.io/projected/cf255fc4-e303-44ad-8b78-a0323af81341-kube-api-access-9q5nn\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:55:25.057747 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:25.057745    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gcvth\" (UniqueName: \"kubernetes.io/projected/4e7acc8f-f1f9-4f22-84b6-d83c98899ec7-kube-api-access-gcvth\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:55:25.282018 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:25.281981    2581 generic.go:356] "Generic (PLEG): container finished" podID="cf255fc4-e303-44ad-8b78-a0323af81341" containerID="2d6e59fe848405fbeaa3bff0efa5f1456e55e289453343b34facb4118fe8dddb" exitCode=0
Sep 09 20:55:25.282224 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:25.282044    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-8cfb58ee-5958d57f66-76h6c"
Sep 09 20:55:25.282224 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:25.282072    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-8cfb58ee-5958d57f66-76h6c" event={"ID":"cf255fc4-e303-44ad-8b78-a0323af81341","Type":"ContainerDied","Data":"2d6e59fe848405fbeaa3bff0efa5f1456e55e289453343b34facb4118fe8dddb"}
Sep 09 20:55:25.282224 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:25.282159    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-8cfb58ee-5958d57f66-76h6c" event={"ID":"cf255fc4-e303-44ad-8b78-a0323af81341","Type":"ContainerDied","Data":"68b1781621e6915bb36c6499036da50c1cfdaa771541b899ca41aa42954c413c"}
Sep 09 20:55:25.282224 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:25.282195    2581 scope.go:117] "RemoveContainer" containerID="2d6e59fe848405fbeaa3bff0efa5f1456e55e289453343b34facb4118fe8dddb"
Sep 09 20:55:25.283608 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:25.283581    2581 generic.go:356] "Generic (PLEG): container finished" podID="4e7acc8f-f1f9-4f22-84b6-d83c98899ec7" containerID="81816dddd67a86937b63f07b0c2dec3f6bc20c54fa302f8650af61684134cec2" exitCode=0
Sep 09 20:55:25.283737 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:25.283631    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-8cfb58ee-c87845758-wxjcr"
Sep 09 20:55:25.283737 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:25.283666    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-8cfb58ee-c87845758-wxjcr" event={"ID":"4e7acc8f-f1f9-4f22-84b6-d83c98899ec7","Type":"ContainerDied","Data":"81816dddd67a86937b63f07b0c2dec3f6bc20c54fa302f8650af61684134cec2"}
Sep 09 20:55:25.283737 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:25.283713    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-8cfb58ee-c87845758-wxjcr" event={"ID":"4e7acc8f-f1f9-4f22-84b6-d83c98899ec7","Type":"ContainerDied","Data":"5e1799946afc218043b404eab4e021e192c376b297177915be172fee2eaa8ea3"}
Sep 09 20:55:25.292238 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:25.292219    2581 scope.go:117] "RemoveContainer" containerID="2d6e59fe848405fbeaa3bff0efa5f1456e55e289453343b34facb4118fe8dddb"
Sep 09 20:55:25.292529 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:55:25.292511    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d6e59fe848405fbeaa3bff0efa5f1456e55e289453343b34facb4118fe8dddb\": container with ID starting with 2d6e59fe848405fbeaa3bff0efa5f1456e55e289453343b34facb4118fe8dddb not found: ID does not exist" containerID="2d6e59fe848405fbeaa3bff0efa5f1456e55e289453343b34facb4118fe8dddb"
Sep 09 20:55:25.292606 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:25.292540    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d6e59fe848405fbeaa3bff0efa5f1456e55e289453343b34facb4118fe8dddb"} err="failed to get container status \"2d6e59fe848405fbeaa3bff0efa5f1456e55e289453343b34facb4118fe8dddb\": rpc error: code = NotFound desc = could not find container \"2d6e59fe848405fbeaa3bff0efa5f1456e55e289453343b34facb4118fe8dddb\": container with ID starting with 2d6e59fe848405fbeaa3bff0efa5f1456e55e289453343b34facb4118fe8dddb not found: ID does not exist"
Sep 09 20:55:25.292606 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:25.292564    2581 scope.go:117] "RemoveContainer" containerID="81816dddd67a86937b63f07b0c2dec3f6bc20c54fa302f8650af61684134cec2"
Sep 09 20:55:25.300414 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:25.300399    2581 scope.go:117] "RemoveContainer" containerID="81816dddd67a86937b63f07b0c2dec3f6bc20c54fa302f8650af61684134cec2"
Sep 09 20:55:25.300669 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:55:25.300652    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81816dddd67a86937b63f07b0c2dec3f6bc20c54fa302f8650af61684134cec2\": container with ID starting with 81816dddd67a86937b63f07b0c2dec3f6bc20c54fa302f8650af61684134cec2 not found: ID does not exist" containerID="81816dddd67a86937b63f07b0c2dec3f6bc20c54fa302f8650af61684134cec2"
Sep 09 20:55:25.300712 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:25.300676    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81816dddd67a86937b63f07b0c2dec3f6bc20c54fa302f8650af61684134cec2"} err="failed to get container status \"81816dddd67a86937b63f07b0c2dec3f6bc20c54fa302f8650af61684134cec2\": rpc error: code = NotFound desc = could not find container \"81816dddd67a86937b63f07b0c2dec3f6bc20c54fa302f8650af61684134cec2\": container with ID starting with 81816dddd67a86937b63f07b0c2dec3f6bc20c54fa302f8650af61684134cec2 not found: ID does not exist"
Sep 09 20:55:25.305512 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:25.305491    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-8cfb58ee-5958d57f66-76h6c"]
Sep 09 20:55:25.309041 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:25.309020    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-8cfb58ee-5958d57f66-76h6c"]
Sep 09 20:55:25.320476 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:25.320447    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-8cfb58ee-c87845758-wxjcr"]
Sep 09 20:55:25.322630 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:25.322608    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-8cfb58ee-c87845758-wxjcr"]
Sep 09 20:55:25.700372 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:25.700335    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-69911538-openshift-default-64b5648d48-l6dmd"]
Sep 09 20:55:25.700853 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:25.700740    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e7acc8f-f1f9-4f22-84b6-d83c98899ec7" containerName="payload-processing"
Sep 09 20:55:25.700853 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:25.700751    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e7acc8f-f1f9-4f22-84b6-d83c98899ec7" containerName="payload-processing"
Sep 09 20:55:25.700853 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:25.700758    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88447789-c7ad-4235-addd-a0139f8b5028" containerName="revoke-keys"
Sep 09 20:55:25.700853 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:25.700763    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="88447789-c7ad-4235-addd-a0139f8b5028" containerName="revoke-keys"
Sep 09 20:55:25.700853 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:25.700783    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf255fc4-e303-44ad-8b78-a0323af81341" containerName="payload-pre-processing"
Sep 09 20:55:25.700853 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:25.700789    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf255fc4-e303-44ad-8b78-a0323af81341" containerName="payload-pre-processing"
Sep 09 20:55:25.700853 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:25.700798    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d39d8f5a-1c5c-4b84-a4de-d65511571543" containerName="istio-proxy"
Sep 09 20:55:25.700853 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:25.700805    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d39d8f5a-1c5c-4b84-a4de-d65511571543" containerName="istio-proxy"
Sep 09 20:55:25.701271 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:25.700862    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="88447789-c7ad-4235-addd-a0139f8b5028" containerName="revoke-keys"
Sep 09 20:55:25.701271 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:25.700870    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf255fc4-e303-44ad-8b78-a0323af81341" containerName="payload-pre-processing"
Sep 09 20:55:25.701271 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:25.700877    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e7acc8f-f1f9-4f22-84b6-d83c98899ec7" containerName="payload-processing"
Sep 09 20:55:25.705504 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:25.705481    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-69911538-openshift-default-64b5648d48-l6dmd"
Sep 09 20:55:25.708288 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:25.708269    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-69911538-openshift-default-dockercfg-9kvjk\""
Sep 09 20:55:25.715408 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:25.715384    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-69911538-openshift-default-64b5648d48-l6dmd"]
Sep 09 20:55:25.867609 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:25.867515    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b140704-2a77-4ca1-8f2a-0aae0d6fe251-istio-envoy\") pod \"e2e-ait-69911538-openshift-default-64b5648d48-l6dmd\" (UID: \"1b140704-2a77-4ca1-8f2a-0aae0d6fe251\") " pod="openshift-ingress/e2e-ait-69911538-openshift-default-64b5648d48-l6dmd"
Sep 09 20:55:25.867609 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:25.867588    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b140704-2a77-4ca1-8f2a-0aae0d6fe251-workload-certs\") pod \"e2e-ait-69911538-openshift-default-64b5648d48-l6dmd\" (UID: \"1b140704-2a77-4ca1-8f2a-0aae0d6fe251\") " pod="openshift-ingress/e2e-ait-69911538-openshift-default-64b5648d48-l6dmd"
Sep 09 20:55:25.867609 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:25.867610    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b140704-2a77-4ca1-8f2a-0aae0d6fe251-istiod-ca-cert\") pod \"e2e-ait-69911538-openshift-default-64b5648d48-l6dmd\" (UID: \"1b140704-2a77-4ca1-8f2a-0aae0d6fe251\") " pod="openshift-ingress/e2e-ait-69911538-openshift-default-64b5648d48-l6dmd"
Sep 09 20:55:25.868178 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:25.867630    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b140704-2a77-4ca1-8f2a-0aae0d6fe251-workload-socket\") pod \"e2e-ait-69911538-openshift-default-64b5648d48-l6dmd\" (UID: \"1b140704-2a77-4ca1-8f2a-0aae0d6fe251\") " pod="openshift-ingress/e2e-ait-69911538-openshift-default-64b5648d48-l6dmd"
Sep 09 20:55:25.868178 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:25.867651    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b140704-2a77-4ca1-8f2a-0aae0d6fe251-istio-token\") pod \"e2e-ait-69911538-openshift-default-64b5648d48-l6dmd\" (UID: \"1b140704-2a77-4ca1-8f2a-0aae0d6fe251\") " pod="openshift-ingress/e2e-ait-69911538-openshift-default-64b5648d48-l6dmd"
Sep 09 20:55:25.868178 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:25.867766    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b140704-2a77-4ca1-8f2a-0aae0d6fe251-istio-data\") pod \"e2e-ait-69911538-openshift-default-64b5648d48-l6dmd\" (UID: \"1b140704-2a77-4ca1-8f2a-0aae0d6fe251\") " pod="openshift-ingress/e2e-ait-69911538-openshift-default-64b5648d48-l6dmd"
Sep 09 20:55:25.868178 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:25.867802    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jf59j\" (UniqueName: \"kubernetes.io/projected/1b140704-2a77-4ca1-8f2a-0aae0d6fe251-kube-api-access-jf59j\") pod \"e2e-ait-69911538-openshift-default-64b5648d48-l6dmd\" (UID: \"1b140704-2a77-4ca1-8f2a-0aae0d6fe251\") " pod="openshift-ingress/e2e-ait-69911538-openshift-default-64b5648d48-l6dmd"
Sep 09 20:55:25.868178 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:25.867831    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b140704-2a77-4ca1-8f2a-0aae0d6fe251-istio-podinfo\") pod \"e2e-ait-69911538-openshift-default-64b5648d48-l6dmd\" (UID: \"1b140704-2a77-4ca1-8f2a-0aae0d6fe251\") " pod="openshift-ingress/e2e-ait-69911538-openshift-default-64b5648d48-l6dmd"
Sep 09 20:55:25.868178 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:25.867883    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b140704-2a77-4ca1-8f2a-0aae0d6fe251-credential-socket\") pod \"e2e-ait-69911538-openshift-default-64b5648d48-l6dmd\" (UID: \"1b140704-2a77-4ca1-8f2a-0aae0d6fe251\") " pod="openshift-ingress/e2e-ait-69911538-openshift-default-64b5648d48-l6dmd"
Sep 09 20:55:25.969123 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:25.969061    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b140704-2a77-4ca1-8f2a-0aae0d6fe251-istio-envoy\") pod \"e2e-ait-69911538-openshift-default-64b5648d48-l6dmd\" (UID: \"1b140704-2a77-4ca1-8f2a-0aae0d6fe251\") " pod="openshift-ingress/e2e-ait-69911538-openshift-default-64b5648d48-l6dmd"
Sep 09 20:55:25.969337 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:25.969315    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b140704-2a77-4ca1-8f2a-0aae0d6fe251-workload-certs\") pod \"e2e-ait-69911538-openshift-default-64b5648d48-l6dmd\" (UID: \"1b140704-2a77-4ca1-8f2a-0aae0d6fe251\") " pod="openshift-ingress/e2e-ait-69911538-openshift-default-64b5648d48-l6dmd"
Sep 09 20:55:25.969462 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:25.969442    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b140704-2a77-4ca1-8f2a-0aae0d6fe251-istiod-ca-cert\") pod \"e2e-ait-69911538-openshift-default-64b5648d48-l6dmd\" (UID: \"1b140704-2a77-4ca1-8f2a-0aae0d6fe251\") " pod="openshift-ingress/e2e-ait-69911538-openshift-default-64b5648d48-l6dmd"
Sep 09 20:55:25.969526 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:25.969477    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b140704-2a77-4ca1-8f2a-0aae0d6fe251-workload-socket\") pod \"e2e-ait-69911538-openshift-default-64b5648d48-l6dmd\" (UID: \"1b140704-2a77-4ca1-8f2a-0aae0d6fe251\") " pod="openshift-ingress/e2e-ait-69911538-openshift-default-64b5648d48-l6dmd"
Sep 09 20:55:25.969526 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:25.969516    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b140704-2a77-4ca1-8f2a-0aae0d6fe251-istio-token\") pod \"e2e-ait-69911538-openshift-default-64b5648d48-l6dmd\" (UID: \"1b140704-2a77-4ca1-8f2a-0aae0d6fe251\") " pod="openshift-ingress/e2e-ait-69911538-openshift-default-64b5648d48-l6dmd"
Sep 09 20:55:25.969640 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:25.969570    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b140704-2a77-4ca1-8f2a-0aae0d6fe251-istio-data\") pod \"e2e-ait-69911538-openshift-default-64b5648d48-l6dmd\" (UID: \"1b140704-2a77-4ca1-8f2a-0aae0d6fe251\") " pod="openshift-ingress/e2e-ait-69911538-openshift-default-64b5648d48-l6dmd"
Sep 09 20:55:25.969640 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:25.969602    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jf59j\" (UniqueName: \"kubernetes.io/projected/1b140704-2a77-4ca1-8f2a-0aae0d6fe251-kube-api-access-jf59j\") pod \"e2e-ait-69911538-openshift-default-64b5648d48-l6dmd\" (UID: \"1b140704-2a77-4ca1-8f2a-0aae0d6fe251\") " pod="openshift-ingress/e2e-ait-69911538-openshift-default-64b5648d48-l6dmd"
Sep 09 20:55:25.969786 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:25.969642    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b140704-2a77-4ca1-8f2a-0aae0d6fe251-istio-podinfo\") pod \"e2e-ait-69911538-openshift-default-64b5648d48-l6dmd\" (UID: \"1b140704-2a77-4ca1-8f2a-0aae0d6fe251\") " pod="openshift-ingress/e2e-ait-69911538-openshift-default-64b5648d48-l6dmd"
Sep 09 20:55:25.969786 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:25.969675    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b140704-2a77-4ca1-8f2a-0aae0d6fe251-workload-certs\") pod \"e2e-ait-69911538-openshift-default-64b5648d48-l6dmd\" (UID: \"1b140704-2a77-4ca1-8f2a-0aae0d6fe251\") " pod="openshift-ingress/e2e-ait-69911538-openshift-default-64b5648d48-l6dmd"
Sep 09 20:55:25.969786 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:25.969700    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b140704-2a77-4ca1-8f2a-0aae0d6fe251-credential-socket\") pod \"e2e-ait-69911538-openshift-default-64b5648d48-l6dmd\" (UID: \"1b140704-2a77-4ca1-8f2a-0aae0d6fe251\") " pod="openshift-ingress/e2e-ait-69911538-openshift-default-64b5648d48-l6dmd"
Sep 09 20:55:25.969952 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:25.969937    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b140704-2a77-4ca1-8f2a-0aae0d6fe251-istio-data\") pod \"e2e-ait-69911538-openshift-default-64b5648d48-l6dmd\" (UID: \"1b140704-2a77-4ca1-8f2a-0aae0d6fe251\") " pod="openshift-ingress/e2e-ait-69911538-openshift-default-64b5648d48-l6dmd"
Sep 09 20:55:25.970145 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:25.970088    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b140704-2a77-4ca1-8f2a-0aae0d6fe251-istiod-ca-cert\") pod \"e2e-ait-69911538-openshift-default-64b5648d48-l6dmd\" (UID: \"1b140704-2a77-4ca1-8f2a-0aae0d6fe251\") " pod="openshift-ingress/e2e-ait-69911538-openshift-default-64b5648d48-l6dmd"
Sep 09 20:55:25.970281 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:25.970178    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b140704-2a77-4ca1-8f2a-0aae0d6fe251-credential-socket\") pod \"e2e-ait-69911538-openshift-default-64b5648d48-l6dmd\" (UID: \"1b140704-2a77-4ca1-8f2a-0aae0d6fe251\") " pod="openshift-ingress/e2e-ait-69911538-openshift-default-64b5648d48-l6dmd"
Sep 09 20:55:25.970352 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:25.970300    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b140704-2a77-4ca1-8f2a-0aae0d6fe251-workload-socket\") pod \"e2e-ait-69911538-openshift-default-64b5648d48-l6dmd\" (UID: \"1b140704-2a77-4ca1-8f2a-0aae0d6fe251\") " pod="openshift-ingress/e2e-ait-69911538-openshift-default-64b5648d48-l6dmd"
Sep 09 20:55:25.971555 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:25.971535    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b140704-2a77-4ca1-8f2a-0aae0d6fe251-istio-envoy\") pod \"e2e-ait-69911538-openshift-default-64b5648d48-l6dmd\" (UID: \"1b140704-2a77-4ca1-8f2a-0aae0d6fe251\") " pod="openshift-ingress/e2e-ait-69911538-openshift-default-64b5648d48-l6dmd"
Sep 09 20:55:25.971911 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:25.971895    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b140704-2a77-4ca1-8f2a-0aae0d6fe251-istio-podinfo\") pod \"e2e-ait-69911538-openshift-default-64b5648d48-l6dmd\" (UID: \"1b140704-2a77-4ca1-8f2a-0aae0d6fe251\") " pod="openshift-ingress/e2e-ait-69911538-openshift-default-64b5648d48-l6dmd"
Sep 09 20:55:25.977968 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:25.977947    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b140704-2a77-4ca1-8f2a-0aae0d6fe251-istio-token\") pod \"e2e-ait-69911538-openshift-default-64b5648d48-l6dmd\" (UID: \"1b140704-2a77-4ca1-8f2a-0aae0d6fe251\") " pod="openshift-ingress/e2e-ait-69911538-openshift-default-64b5648d48-l6dmd"
Sep 09 20:55:25.978156 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:25.978136    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jf59j\" (UniqueName: \"kubernetes.io/projected/1b140704-2a77-4ca1-8f2a-0aae0d6fe251-kube-api-access-jf59j\") pod \"e2e-ait-69911538-openshift-default-64b5648d48-l6dmd\" (UID: \"1b140704-2a77-4ca1-8f2a-0aae0d6fe251\") " pod="openshift-ingress/e2e-ait-69911538-openshift-default-64b5648d48-l6dmd"
Sep 09 20:55:26.017214 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:26.017191    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-69911538-openshift-default-64b5648d48-l6dmd"
Sep 09 20:55:26.294602 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:26.294576    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-69911538-openshift-default-64b5648d48-l6dmd"]
Sep 09 20:55:26.296256 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:55:26.296233    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b140704_2a77_4ca1_8f2a_0aae0d6fe251.slice/crio-c5d8e9f98003d380d3d0484de58b408655aae5a26d8513e76378e7f9e5fe0912 WatchSource:0}: Error finding container c5d8e9f98003d380d3d0484de58b408655aae5a26d8513e76378e7f9e5fe0912: Status 404 returned error can't find the container with id c5d8e9f98003d380d3d0484de58b408655aae5a26d8513e76378e7f9e5fe0912
Sep 09 20:55:26.298540 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:26.298504    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:55:26.298656 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:26.298585    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:55:26.298656 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:26.298630    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:55:26.821056 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:26.821023    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e7acc8f-f1f9-4f22-84b6-d83c98899ec7" path="/var/lib/kubelet/pods/4e7acc8f-f1f9-4f22-84b6-d83c98899ec7/volumes"
Sep 09 20:55:26.821492 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:26.821475    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf255fc4-e303-44ad-8b78-a0323af81341" path="/var/lib/kubelet/pods/cf255fc4-e303-44ad-8b78-a0323af81341/volumes"
Sep 09 20:55:27.299189 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:27.299152    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-69911538-openshift-default-64b5648d48-l6dmd" event={"ID":"1b140704-2a77-4ca1-8f2a-0aae0d6fe251","Type":"ContainerStarted","Data":"8bbddb76f3e6608bb0afc5390ccc86e5e6ca8d657bf9537396b95c7232b1c3b4"}
Sep 09 20:55:27.299360 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:27.299210    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-69911538-openshift-default-64b5648d48-l6dmd" event={"ID":"1b140704-2a77-4ca1-8f2a-0aae0d6fe251","Type":"ContainerStarted","Data":"c5d8e9f98003d380d3d0484de58b408655aae5a26d8513e76378e7f9e5fe0912"}
Sep 09 20:55:27.326167 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:27.326117    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-69911538-openshift-default-64b5648d48-l6dmd" podStartSLOduration=2.326088778 podStartE2EDuration="2.326088778s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:55:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:55:27.324146412 +0000 UTC m=+888.951774285" watchObservedRunningTime="2026-09-09 20:55:27.326088778 +0000 UTC m=+888.953716651"
Sep 09 20:55:28.017608 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:28.017570    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-69911538-openshift-default-64b5648d48-l6dmd"
Sep 09 20:55:28.022733 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:28.022708    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-69911538-openshift-default-64b5648d48-l6dmd"
Sep 09 20:55:28.304807 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:28.304724    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-69911538-openshift-default-64b5648d48-l6dmd"
Sep 09 20:55:28.305798 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:28.305777    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-69911538-openshift-default-64b5648d48-l6dmd"
Sep 09 20:55:30.177084 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.177061    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls"
Sep 09 20:55:30.314412 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.314364    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9793617-0c83-4e0b-9caf-07dc35c43f80-istiod-ca-cert\") pod \"f9793617-0c83-4e0b-9caf-07dc35c43f80\" (UID: \"f9793617-0c83-4e0b-9caf-07dc35c43f80\") "
Sep 09 20:55:30.314593 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.314419    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9793617-0c83-4e0b-9caf-07dc35c43f80-istio-envoy\") pod \"f9793617-0c83-4e0b-9caf-07dc35c43f80\" (UID: \"f9793617-0c83-4e0b-9caf-07dc35c43f80\") "
Sep 09 20:55:30.314593 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.314446    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9793617-0c83-4e0b-9caf-07dc35c43f80-istio-podinfo\") pod \"f9793617-0c83-4e0b-9caf-07dc35c43f80\" (UID: \"f9793617-0c83-4e0b-9caf-07dc35c43f80\") "
Sep 09 20:55:30.314593 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.314520    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9793617-0c83-4e0b-9caf-07dc35c43f80-istio-token\") pod \"f9793617-0c83-4e0b-9caf-07dc35c43f80\" (UID: \"f9793617-0c83-4e0b-9caf-07dc35c43f80\") "
Sep 09 20:55:30.314593 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.314578    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9793617-0c83-4e0b-9caf-07dc35c43f80-workload-socket\") pod \"f9793617-0c83-4e0b-9caf-07dc35c43f80\" (UID: \"f9793617-0c83-4e0b-9caf-07dc35c43f80\") "
Sep 09 20:55:30.314794 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.314641    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9793617-0c83-4e0b-9caf-07dc35c43f80-istio-data\") pod \"f9793617-0c83-4e0b-9caf-07dc35c43f80\" (UID: \"f9793617-0c83-4e0b-9caf-07dc35c43f80\") "
Sep 09 20:55:30.314794 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.314686    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9793617-0c83-4e0b-9caf-07dc35c43f80-credential-socket\") pod \"f9793617-0c83-4e0b-9caf-07dc35c43f80\" (UID: \"f9793617-0c83-4e0b-9caf-07dc35c43f80\") "
Sep 09 20:55:30.314794 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.314734    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f9793617-0c83-4e0b-9caf-07dc35c43f80-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f9793617-0c83-4e0b-9caf-07dc35c43f80" (UID: "f9793617-0c83-4e0b-9caf-07dc35c43f80"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:55:30.314794 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.314748    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9793617-0c83-4e0b-9caf-07dc35c43f80-workload-certs\") pod \"f9793617-0c83-4e0b-9caf-07dc35c43f80\" (UID: \"f9793617-0c83-4e0b-9caf-07dc35c43f80\") "
Sep 09 20:55:30.314794 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.314792    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8bdlj\" (UniqueName: \"kubernetes.io/projected/f9793617-0c83-4e0b-9caf-07dc35c43f80-kube-api-access-8bdlj\") pod \"f9793617-0c83-4e0b-9caf-07dc35c43f80\" (UID: \"f9793617-0c83-4e0b-9caf-07dc35c43f80\") "
Sep 09 20:55:30.315091 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.314901    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9793617-0c83-4e0b-9caf-07dc35c43f80-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f9793617-0c83-4e0b-9caf-07dc35c43f80" (UID: "f9793617-0c83-4e0b-9caf-07dc35c43f80"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:55:30.315091 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.315005    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9793617-0c83-4e0b-9caf-07dc35c43f80-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f9793617-0c83-4e0b-9caf-07dc35c43f80" (UID: "f9793617-0c83-4e0b-9caf-07dc35c43f80"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:55:30.315293 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.315257    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9793617-0c83-4e0b-9caf-07dc35c43f80-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f9793617-0c83-4e0b-9caf-07dc35c43f80" (UID: "f9793617-0c83-4e0b-9caf-07dc35c43f80"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:55:30.315293 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.315277    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9793617-0c83-4e0b-9caf-07dc35c43f80-workload-socket\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:55:30.315433 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.315314    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9793617-0c83-4e0b-9caf-07dc35c43f80-credential-socket\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:55:30.315433 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.315329    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9793617-0c83-4e0b-9caf-07dc35c43f80-istiod-ca-cert\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:55:30.315557 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.315534    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9793617-0c83-4e0b-9caf-07dc35c43f80-istio-data" (OuterVolumeSpecName: "istio-data") pod "f9793617-0c83-4e0b-9caf-07dc35c43f80" (UID: "f9793617-0c83-4e0b-9caf-07dc35c43f80"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:55:30.317044 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.317004    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9793617-0c83-4e0b-9caf-07dc35c43f80-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f9793617-0c83-4e0b-9caf-07dc35c43f80" (UID: "f9793617-0c83-4e0b-9caf-07dc35c43f80"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:55:30.317729 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.317698    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f9793617-0c83-4e0b-9caf-07dc35c43f80-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f9793617-0c83-4e0b-9caf-07dc35c43f80" (UID: "f9793617-0c83-4e0b-9caf-07dc35c43f80"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 20:55:30.317729 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.317709    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9793617-0c83-4e0b-9caf-07dc35c43f80-kube-api-access-8bdlj" (OuterVolumeSpecName: "kube-api-access-8bdlj") pod "f9793617-0c83-4e0b-9caf-07dc35c43f80" (UID: "f9793617-0c83-4e0b-9caf-07dc35c43f80"). InnerVolumeSpecName "kube-api-access-8bdlj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:55:30.318012 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.317990    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9793617-0c83-4e0b-9caf-07dc35c43f80-istio-token" (OuterVolumeSpecName: "istio-token") pod "f9793617-0c83-4e0b-9caf-07dc35c43f80" (UID: "f9793617-0c83-4e0b-9caf-07dc35c43f80"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:55:30.320151 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.320119    2581 generic.go:356] "Generic (PLEG): container finished" podID="f9793617-0c83-4e0b-9caf-07dc35c43f80" containerID="99a8bd43927aed48e7df8133f404ffb125fdb79112d3dcb418f21bd10a5a4ce2" exitCode=0
Sep 09 20:55:30.320255 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.320171    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls"
Sep 09 20:55:30.320255 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.320204    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls" event={"ID":"f9793617-0c83-4e0b-9caf-07dc35c43f80","Type":"ContainerDied","Data":"99a8bd43927aed48e7df8133f404ffb125fdb79112d3dcb418f21bd10a5a4ce2"}
Sep 09 20:55:30.320375 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.320270    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls" event={"ID":"f9793617-0c83-4e0b-9caf-07dc35c43f80","Type":"ContainerDied","Data":"5068e05d734cda41cf2a3e3aa2205d3cdba7ca639daa6d49542b0dbe1680cf8f"}
Sep 09 20:55:30.320375 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.320302    2581 scope.go:117] "RemoveContainer" containerID="99a8bd43927aed48e7df8133f404ffb125fdb79112d3dcb418f21bd10a5a4ce2"
Sep 09 20:55:30.341419 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.341395    2581 scope.go:117] "RemoveContainer" containerID="e5a861e0a42587623bc05dbbd040a39f7ca860ec1ec1de196716a5872954ce43"
Sep 09 20:55:30.350682 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.350663    2581 scope.go:117] "RemoveContainer" containerID="99a8bd43927aed48e7df8133f404ffb125fdb79112d3dcb418f21bd10a5a4ce2"
Sep 09 20:55:30.351002 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:55:30.350982    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99a8bd43927aed48e7df8133f404ffb125fdb79112d3dcb418f21bd10a5a4ce2\": container with ID starting with 99a8bd43927aed48e7df8133f404ffb125fdb79112d3dcb418f21bd10a5a4ce2 not found: ID does not exist" containerID="99a8bd43927aed48e7df8133f404ffb125fdb79112d3dcb418f21bd10a5a4ce2"
Sep 09 20:55:30.351060 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.351013    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99a8bd43927aed48e7df8133f404ffb125fdb79112d3dcb418f21bd10a5a4ce2"} err="failed to get container status \"99a8bd43927aed48e7df8133f404ffb125fdb79112d3dcb418f21bd10a5a4ce2\": rpc error: code = NotFound desc = could not find container \"99a8bd43927aed48e7df8133f404ffb125fdb79112d3dcb418f21bd10a5a4ce2\": container with ID starting with 99a8bd43927aed48e7df8133f404ffb125fdb79112d3dcb418f21bd10a5a4ce2 not found: ID does not exist"
Sep 09 20:55:30.351060 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.351032    2581 scope.go:117] "RemoveContainer" containerID="e5a861e0a42587623bc05dbbd040a39f7ca860ec1ec1de196716a5872954ce43"
Sep 09 20:55:30.351316 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:55:30.351295    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e5a861e0a42587623bc05dbbd040a39f7ca860ec1ec1de196716a5872954ce43\": container with ID starting with e5a861e0a42587623bc05dbbd040a39f7ca860ec1ec1de196716a5872954ce43 not found: ID does not exist" containerID="e5a861e0a42587623bc05dbbd040a39f7ca860ec1ec1de196716a5872954ce43"
Sep 09 20:55:30.351410 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.351321    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e5a861e0a42587623bc05dbbd040a39f7ca860ec1ec1de196716a5872954ce43"} err="failed to get container status \"e5a861e0a42587623bc05dbbd040a39f7ca860ec1ec1de196716a5872954ce43\": rpc error: code = NotFound desc = could not find container \"e5a861e0a42587623bc05dbbd040a39f7ca860ec1ec1de196716a5872954ce43\": container with ID starting with e5a861e0a42587623bc05dbbd040a39f7ca860ec1ec1de196716a5872954ce43 not found: ID does not exist"
Sep 09 20:55:30.356004 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.355983    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls"]
Sep 09 20:55:30.361571 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.361550    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-8cfb58ee-gw-openshift-default-67bf66f45b-fxnls"]
Sep 09 20:55:30.416707 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.416675    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9793617-0c83-4e0b-9caf-07dc35c43f80-istio-data\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:55:30.416707 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.416705    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9793617-0c83-4e0b-9caf-07dc35c43f80-workload-certs\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:55:30.416943 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.416719    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8bdlj\" (UniqueName: \"kubernetes.io/projected/f9793617-0c83-4e0b-9caf-07dc35c43f80-kube-api-access-8bdlj\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:55:30.416943 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.416736    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9793617-0c83-4e0b-9caf-07dc35c43f80-istio-envoy\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:55:30.416943 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.416750    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9793617-0c83-4e0b-9caf-07dc35c43f80-istio-podinfo\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:55:30.416943 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.416762    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9793617-0c83-4e0b-9caf-07dc35c43f80-istio-token\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:55:30.458725 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.458695    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-69911538-7cd58b96c4-g7tfn"]
Sep 09 20:55:30.459135 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.459121    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9793617-0c83-4e0b-9caf-07dc35c43f80" containerName="istio-proxy"
Sep 09 20:55:30.459135 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.459135    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9793617-0c83-4e0b-9caf-07dc35c43f80" containerName="istio-proxy"
Sep 09 20:55:30.459228 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.459216    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9793617-0c83-4e0b-9caf-07dc35c43f80" containerName="istio-proxy"
Sep 09 20:55:30.459228 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.459225    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9793617-0c83-4e0b-9caf-07dc35c43f80" containerName="istio-proxy"
Sep 09 20:55:30.463830 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.463813    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-69911538-7cd58b96c4-g7tfn"
Sep 09 20:55:30.466404 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.466381    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-69911538\""
Sep 09 20:55:30.474573 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.474546    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-69911538-7cd58b96c4-g7tfn"]
Sep 09 20:55:30.480991 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.480971    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-69911538-694ddf7675-hdjh5"]
Sep 09 20:55:30.481414 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.481400    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9793617-0c83-4e0b-9caf-07dc35c43f80" containerName="istio-proxy"
Sep 09 20:55:30.481465 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.481418    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9793617-0c83-4e0b-9caf-07dc35c43f80" containerName="istio-proxy"
Sep 09 20:55:30.484479 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.484455    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-69911538-694ddf7675-hdjh5"
Sep 09 20:55:30.487373 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.487353    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-69911538-dockercfg-ckx7r\""
Sep 09 20:55:30.497615 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.497591    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-69911538-694ddf7675-hdjh5"]
Sep 09 20:55:30.619262 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.619141    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bebcceb2-1d61-4010-aa46-8f3d7ef0bc49-plugins-config-volume\") pod \"payload-processing-e2e-ait-69911538-694ddf7675-hdjh5\" (UID: \"bebcceb2-1d61-4010-aa46-8f3d7ef0bc49\") " pod="openshift-ingress/payload-processing-e2e-ait-69911538-694ddf7675-hdjh5"
Sep 09 20:55:30.619262 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.619217    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9788\" (UniqueName: \"kubernetes.io/projected/a7bfc82f-2271-4cb2-8426-f3da46382656-kube-api-access-n9788\") pod \"payload-pre-processing-e2e-ait-69911538-7cd58b96c4-g7tfn\" (UID: \"a7bfc82f-2271-4cb2-8426-f3da46382656\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-69911538-7cd58b96c4-g7tfn"
Sep 09 20:55:30.619518 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.619332    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qn9zv\" (UniqueName: \"kubernetes.io/projected/bebcceb2-1d61-4010-aa46-8f3d7ef0bc49-kube-api-access-qn9zv\") pod \"payload-processing-e2e-ait-69911538-694ddf7675-hdjh5\" (UID: \"bebcceb2-1d61-4010-aa46-8f3d7ef0bc49\") " pod="openshift-ingress/payload-processing-e2e-ait-69911538-694ddf7675-hdjh5"
Sep 09 20:55:30.619518 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.619372    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7bfc82f-2271-4cb2-8426-f3da46382656-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-69911538-7cd58b96c4-g7tfn\" (UID: \"a7bfc82f-2271-4cb2-8426-f3da46382656\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-69911538-7cd58b96c4-g7tfn"
Sep 09 20:55:30.720510 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.720469    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bebcceb2-1d61-4010-aa46-8f3d7ef0bc49-plugins-config-volume\") pod \"payload-processing-e2e-ait-69911538-694ddf7675-hdjh5\" (UID: \"bebcceb2-1d61-4010-aa46-8f3d7ef0bc49\") " pod="openshift-ingress/payload-processing-e2e-ait-69911538-694ddf7675-hdjh5"
Sep 09 20:55:30.720687 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.720521    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9788\" (UniqueName: \"kubernetes.io/projected/a7bfc82f-2271-4cb2-8426-f3da46382656-kube-api-access-n9788\") pod \"payload-pre-processing-e2e-ait-69911538-7cd58b96c4-g7tfn\" (UID: \"a7bfc82f-2271-4cb2-8426-f3da46382656\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-69911538-7cd58b96c4-g7tfn"
Sep 09 20:55:30.720687 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.720584    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qn9zv\" (UniqueName: \"kubernetes.io/projected/bebcceb2-1d61-4010-aa46-8f3d7ef0bc49-kube-api-access-qn9zv\") pod \"payload-processing-e2e-ait-69911538-694ddf7675-hdjh5\" (UID: \"bebcceb2-1d61-4010-aa46-8f3d7ef0bc49\") " pod="openshift-ingress/payload-processing-e2e-ait-69911538-694ddf7675-hdjh5"
Sep 09 20:55:30.720687 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.720618    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7bfc82f-2271-4cb2-8426-f3da46382656-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-69911538-7cd58b96c4-g7tfn\" (UID: \"a7bfc82f-2271-4cb2-8426-f3da46382656\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-69911538-7cd58b96c4-g7tfn"
Sep 09 20:55:30.721242 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.721222    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bebcceb2-1d61-4010-aa46-8f3d7ef0bc49-plugins-config-volume\") pod \"payload-processing-e2e-ait-69911538-694ddf7675-hdjh5\" (UID: \"bebcceb2-1d61-4010-aa46-8f3d7ef0bc49\") " pod="openshift-ingress/payload-processing-e2e-ait-69911538-694ddf7675-hdjh5"
Sep 09 20:55:30.721438 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.721418    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7bfc82f-2271-4cb2-8426-f3da46382656-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-69911538-7cd58b96c4-g7tfn\" (UID: \"a7bfc82f-2271-4cb2-8426-f3da46382656\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-69911538-7cd58b96c4-g7tfn"
Sep 09 20:55:30.737321 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.737297    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9788\" (UniqueName: \"kubernetes.io/projected/a7bfc82f-2271-4cb2-8426-f3da46382656-kube-api-access-n9788\") pod \"payload-pre-processing-e2e-ait-69911538-7cd58b96c4-g7tfn\" (UID: \"a7bfc82f-2271-4cb2-8426-f3da46382656\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-69911538-7cd58b96c4-g7tfn"
Sep 09 20:55:30.738924 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.738901    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qn9zv\" (UniqueName: \"kubernetes.io/projected/bebcceb2-1d61-4010-aa46-8f3d7ef0bc49-kube-api-access-qn9zv\") pod \"payload-processing-e2e-ait-69911538-694ddf7675-hdjh5\" (UID: \"bebcceb2-1d61-4010-aa46-8f3d7ef0bc49\") " pod="openshift-ingress/payload-processing-e2e-ait-69911538-694ddf7675-hdjh5"
Sep 09 20:55:30.775582 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.775553    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-69911538-7cd58b96c4-g7tfn"
Sep 09 20:55:30.794785 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.794762    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-69911538-694ddf7675-hdjh5"
Sep 09 20:55:30.821294 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:30.821258    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9793617-0c83-4e0b-9caf-07dc35c43f80" path="/var/lib/kubelet/pods/f9793617-0c83-4e0b-9caf-07dc35c43f80/volumes"
Sep 09 20:55:31.052055 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:31.052032    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-69911538-7cd58b96c4-g7tfn"]
Sep 09 20:55:31.053998 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:55:31.053973    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7bfc82f_2271_4cb2_8426_f3da46382656.slice/crio-913824dfe4ad9c51c30b01850ee184442023dfe55409165e2df953fbad604efb WatchSource:0}: Error finding container 913824dfe4ad9c51c30b01850ee184442023dfe55409165e2df953fbad604efb: Status 404 returned error can't find the container with id 913824dfe4ad9c51c30b01850ee184442023dfe55409165e2df953fbad604efb
Sep 09 20:55:31.077282 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:31.077245    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-69911538-694ddf7675-hdjh5"]
Sep 09 20:55:31.079049 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:55:31.079009    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbebcceb2_1d61_4010_aa46_8f3d7ef0bc49.slice/crio-dfcb567fec85dc42d3f3535c7e085ff9968b1fe2365d7311a6e38f7ede8abbc9 WatchSource:0}: Error finding container dfcb567fec85dc42d3f3535c7e085ff9968b1fe2365d7311a6e38f7ede8abbc9: Status 404 returned error can't find the container with id dfcb567fec85dc42d3f3535c7e085ff9968b1fe2365d7311a6e38f7ede8abbc9
Sep 09 20:55:31.328074 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:31.327729    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-69911538-694ddf7675-hdjh5" event={"ID":"bebcceb2-1d61-4010-aa46-8f3d7ef0bc49","Type":"ContainerStarted","Data":"be70f9c2407f47a8002e2070cd75e1d2b573b08c569ffddb21cc94397e461357"}
Sep 09 20:55:31.328074 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:31.327812    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-69911538-694ddf7675-hdjh5" event={"ID":"bebcceb2-1d61-4010-aa46-8f3d7ef0bc49","Type":"ContainerStarted","Data":"dfcb567fec85dc42d3f3535c7e085ff9968b1fe2365d7311a6e38f7ede8abbc9"}
Sep 09 20:55:31.328074 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:31.327849    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-69911538-694ddf7675-hdjh5"
Sep 09 20:55:31.330547 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:31.330518    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-69911538-7cd58b96c4-g7tfn" event={"ID":"a7bfc82f-2271-4cb2-8426-f3da46382656","Type":"ContainerStarted","Data":"e2d1fb162103046456e3e00ec9ecdeff37e2af03a500e5eea97b365b5fbef52d"}
Sep 09 20:55:31.330702 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:31.330599    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-69911538-7cd58b96c4-g7tfn" event={"ID":"a7bfc82f-2271-4cb2-8426-f3da46382656","Type":"ContainerStarted","Data":"913824dfe4ad9c51c30b01850ee184442023dfe55409165e2df953fbad604efb"}
Sep 09 20:55:31.330702 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:31.330640    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-69911538-7cd58b96c4-g7tfn"
Sep 09 20:55:31.347422 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:31.347374    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-69911538-694ddf7675-hdjh5" podStartSLOduration=1.34735847 podStartE2EDuration="1.34735847s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:55:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:55:31.344869272 +0000 UTC m=+892.972497148" watchObservedRunningTime="2026-09-09 20:55:31.34735847 +0000 UTC m=+892.974986378"
Sep 09 20:55:31.365056 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:31.365007    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-69911538-7cd58b96c4-g7tfn" podStartSLOduration=1.364991994 podStartE2EDuration="1.364991994s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:55:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:55:31.362831763 +0000 UTC m=+892.990459638" watchObservedRunningTime="2026-09-09 20:55:31.364991994 +0000 UTC m=+892.992619870"
Sep 09 20:55:38.823365 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:38.823334    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-wsv4v_2b1f9a6e-7a79-4e89-ac3f-4658d6d91550/console-operator/1.log"
Sep 09 20:55:38.823743 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:38.823724    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-wsv4v_2b1f9a6e-7a79-4e89-ac3f-4658d6d91550/console-operator/1.log"
Sep 09 20:55:38.830765 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:38.830741    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-75wj4_55b4fbae-29dc-41bf-a907-b1412b852cef/ovn-acl-logging/0.log"
Sep 09 20:55:38.831309 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:38.831281    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-75wj4_55b4fbae-29dc-41bf-a907-b1412b852cef/ovn-acl-logging/0.log"
Sep 09 20:55:42.337766 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:42.337736    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-69911538-694ddf7675-hdjh5"
Sep 09 20:55:42.338167 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:42.338147    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-69911538-7cd58b96c4-g7tfn"
Sep 09 20:55:53.461431 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:53.461394    2581 generic.go:356] "Generic (PLEG): container finished" podID="1b140704-2a77-4ca1-8f2a-0aae0d6fe251" containerID="8bbddb76f3e6608bb0afc5390ccc86e5e6ca8d657bf9537396b95c7232b1c3b4" exitCode=0
Sep 09 20:55:53.461880 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:53.461465    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-69911538-openshift-default-64b5648d48-l6dmd" event={"ID":"1b140704-2a77-4ca1-8f2a-0aae0d6fe251","Type":"ContainerDied","Data":"8bbddb76f3e6608bb0afc5390ccc86e5e6ca8d657bf9537396b95c7232b1c3b4"}
Sep 09 20:55:53.461925 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:53.461908    2581 scope.go:117] "RemoveContainer" containerID="8bbddb76f3e6608bb0afc5390ccc86e5e6ca8d657bf9537396b95c7232b1c3b4"
Sep 09 20:55:53.463163 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:53.463136    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:55:53.463231 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:53.463206    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:55:53.463270 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:53.463238    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:55:54.468919 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:54.468885    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-69911538-openshift-default-64b5648d48-l6dmd" event={"ID":"1b140704-2a77-4ca1-8f2a-0aae0d6fe251","Type":"ContainerStarted","Data":"29a4d5699f6ad7bc484893a639c26102644b0e02e5491e6cfedc676ed0d3d4b0"}
Sep 09 20:55:54.529059 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:54.529018    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-69911538-ctrfz"]
Sep 09 20:55:54.533227 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:54.533209    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-69911538-ctrfz"
Sep 09 20:55:54.536320 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:54.536296    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-glz44\""
Sep 09 20:55:54.538685 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:54.538655    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-69911538-ctrfz"]
Sep 09 20:55:54.684480 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:54.684442    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca2d0f92-ffdc-4602-bf1c-e360a0bdecf3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-69911538-ctrfz\" (UID: \"ca2d0f92-ffdc-4602-bf1c-e360a0bdecf3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-69911538-ctrfz"
Sep 09 20:55:54.785602 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:54.785567    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca2d0f92-ffdc-4602-bf1c-e360a0bdecf3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-69911538-ctrfz\" (UID: \"ca2d0f92-ffdc-4602-bf1c-e360a0bdecf3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-69911538-ctrfz"
Sep 09 20:55:54.786230 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:54.786206    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca2d0f92-ffdc-4602-bf1c-e360a0bdecf3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-69911538-ctrfz\" (UID: \"ca2d0f92-ffdc-4602-bf1c-e360a0bdecf3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-69911538-ctrfz"
Sep 09 20:55:54.848333 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:54.848297    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-69911538-ctrfz"
Sep 09 20:55:55.023613 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:55.023527    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-69911538-openshift-default-64b5648d48-l6dmd"
Sep 09 20:55:55.025157 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:55.024917    2581 patch_prober.go:28] interesting pod/e2e-ait-69911538-openshift-default-64b5648d48-l6dmd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Sep 09 20:55:55.025157 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:55.024996    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-69911538-openshift-default-64b5648d48-l6dmd" podUID="1b140704-2a77-4ca1-8f2a-0aae0d6fe251" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Sep 09 20:55:55.132696 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:55.132673    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-69911538-ctrfz"]
Sep 09 20:55:55.134636 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:55:55.134611    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca2d0f92_ffdc_4602_bf1c_e360a0bdecf3.slice/crio-3cef485cef1e535c3484e06c33912781a0910ce69bedf9ce8373175f12586e43 WatchSource:0}: Error finding container 3cef485cef1e535c3484e06c33912781a0910ce69bedf9ce8373175f12586e43: Status 404 returned error can't find the container with id 3cef485cef1e535c3484e06c33912781a0910ce69bedf9ce8373175f12586e43
Sep 09 20:55:55.475644 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:55.475554    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-69911538-ctrfz" event={"ID":"ca2d0f92-ffdc-4602-bf1c-e360a0bdecf3","Type":"ContainerStarted","Data":"3cef485cef1e535c3484e06c33912781a0910ce69bedf9ce8373175f12586e43"}
Sep 09 20:55:56.018811 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:56.018297    2581 patch_prober.go:28] interesting pod/e2e-ait-69911538-openshift-default-64b5648d48-l6dmd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Sep 09 20:55:56.018811 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:56.018383    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-69911538-openshift-default-64b5648d48-l6dmd" podUID="1b140704-2a77-4ca1-8f2a-0aae0d6fe251" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Sep 09 20:55:56.481884 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:56.481852    2581 generic.go:356] "Generic (PLEG): container finished" podID="ca2d0f92-ffdc-4602-bf1c-e360a0bdecf3" containerID="3e0a5fdf6784f512a1a43db048b72e287684327b50ccd46867a4069055e1ddc0" exitCode=0
Sep 09 20:55:56.482315 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:56.481930    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-69911538-ctrfz" event={"ID":"ca2d0f92-ffdc-4602-bf1c-e360a0bdecf3","Type":"ContainerDied","Data":"3e0a5fdf6784f512a1a43db048b72e287684327b50ccd46867a4069055e1ddc0"}
Sep 09 20:55:57.018523 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:57.018484    2581 patch_prober.go:28] interesting pod/e2e-ait-69911538-openshift-default-64b5648d48-l6dmd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Sep 09 20:55:57.018802 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:57.018768    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-69911538-openshift-default-64b5648d48-l6dmd" podUID="1b140704-2a77-4ca1-8f2a-0aae0d6fe251" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Sep 09 20:55:57.657960 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:57.657936    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-69911538-ctrfz"
Sep 09 20:55:57.826808 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:57.826710    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca2d0f92-ffdc-4602-bf1c-e360a0bdecf3-maas-api-service-ca\") pod \"ca2d0f92-ffdc-4602-bf1c-e360a0bdecf3\" (UID: \"ca2d0f92-ffdc-4602-bf1c-e360a0bdecf3\") "
Sep 09 20:55:57.827152 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:57.827090    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca2d0f92-ffdc-4602-bf1c-e360a0bdecf3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ca2d0f92-ffdc-4602-bf1c-e360a0bdecf3" (UID: "ca2d0f92-ffdc-4602-bf1c-e360a0bdecf3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:55:57.927797 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:57.927755    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca2d0f92-ffdc-4602-bf1c-e360a0bdecf3-maas-api-service-ca\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:55:58.017615 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:58.017574    2581 patch_prober.go:28] interesting pod/e2e-ait-69911538-openshift-default-64b5648d48-l6dmd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Sep 09 20:55:58.017896 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:58.017866    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-69911538-openshift-default-64b5648d48-l6dmd" podUID="1b140704-2a77-4ca1-8f2a-0aae0d6fe251" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Sep 09 20:55:58.306466 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:58.305580    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-69911538-openshift-default-64b5648d48-l6dmd"
Sep 09 20:55:58.493776 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:58.493751    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-69911538-ctrfz"
Sep 09 20:55:58.493945 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:58.493788    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-69911538-ctrfz" event={"ID":"ca2d0f92-ffdc-4602-bf1c-e360a0bdecf3","Type":"ContainerDied","Data":"3cef485cef1e535c3484e06c33912781a0910ce69bedf9ce8373175f12586e43"}
Sep 09 20:55:58.493945 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:58.493833    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3cef485cef1e535c3484e06c33912781a0910ce69bedf9ce8373175f12586e43"
Sep 09 20:55:59.018387 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:59.018340    2581 patch_prober.go:28] interesting pod/e2e-ait-69911538-openshift-default-64b5648d48-l6dmd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Sep 09 20:55:59.019001 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:55:59.018412    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-69911538-openshift-default-64b5648d48-l6dmd" podUID="1b140704-2a77-4ca1-8f2a-0aae0d6fe251" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Sep 09 20:56:00.022041 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:00.022013    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-69911538-openshift-default-64b5648d48-l6dmd"
Sep 09 20:56:00.023066 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:00.023043    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-69911538-openshift-default-64b5648d48-l6dmd"
Sep 09 20:56:00.420807 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:00.420724    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-69911538-694ddf7675-hdjh5"]
Sep 09 20:56:00.421261 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:00.421227    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-69911538-694ddf7675-hdjh5" podUID="bebcceb2-1d61-4010-aa46-8f3d7ef0bc49" containerName="payload-processing" containerID="cri-o://be70f9c2407f47a8002e2070cd75e1d2b573b08c569ffddb21cc94397e461357" gracePeriod=30
Sep 09 20:56:00.431424 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:00.431128    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-69911538-7cd58b96c4-g7tfn"]
Sep 09 20:56:00.431424 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:00.431365    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-69911538-7cd58b96c4-g7tfn" podUID="a7bfc82f-2271-4cb2-8426-f3da46382656" containerName="payload-pre-processing" containerID="cri-o://e2d1fb162103046456e3e00ec9ecdeff37e2af03a500e5eea97b365b5fbef52d" gracePeriod=30
Sep 09 20:56:00.602851 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:00.602787    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-69911538-ctrfz"]
Sep 09 20:56:00.608008 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:00.607982    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-69911538-ctrfz"]
Sep 09 20:56:00.717236 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:00.717213    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-69911538-694ddf7675-hdjh5"
Sep 09 20:56:00.720417 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:00.720389    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-69911538-7cd58b96c4-g7tfn"
Sep 09 20:56:00.820561 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:00.820526    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca2d0f92-ffdc-4602-bf1c-e360a0bdecf3" path="/var/lib/kubelet/pods/ca2d0f92-ffdc-4602-bf1c-e360a0bdecf3/volumes"
Sep 09 20:56:00.859327 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:00.859282    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n9788\" (UniqueName: \"kubernetes.io/projected/a7bfc82f-2271-4cb2-8426-f3da46382656-kube-api-access-n9788\") pod \"a7bfc82f-2271-4cb2-8426-f3da46382656\" (UID: \"a7bfc82f-2271-4cb2-8426-f3da46382656\") "
Sep 09 20:56:00.859551 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:00.859411    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bebcceb2-1d61-4010-aa46-8f3d7ef0bc49-plugins-config-volume\") pod \"bebcceb2-1d61-4010-aa46-8f3d7ef0bc49\" (UID: \"bebcceb2-1d61-4010-aa46-8f3d7ef0bc49\") "
Sep 09 20:56:00.859551 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:00.859459    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qn9zv\" (UniqueName: \"kubernetes.io/projected/bebcceb2-1d61-4010-aa46-8f3d7ef0bc49-kube-api-access-qn9zv\") pod \"bebcceb2-1d61-4010-aa46-8f3d7ef0bc49\" (UID: \"bebcceb2-1d61-4010-aa46-8f3d7ef0bc49\") "
Sep 09 20:56:00.859551 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:00.859506    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7bfc82f-2271-4cb2-8426-f3da46382656-plugins-config-volume\") pod \"a7bfc82f-2271-4cb2-8426-f3da46382656\" (UID: \"a7bfc82f-2271-4cb2-8426-f3da46382656\") "
Sep 09 20:56:00.859941 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:00.859817    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bebcceb2-1d61-4010-aa46-8f3d7ef0bc49-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bebcceb2-1d61-4010-aa46-8f3d7ef0bc49" (UID: "bebcceb2-1d61-4010-aa46-8f3d7ef0bc49"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:56:00.860063 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:00.860042    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7bfc82f-2271-4cb2-8426-f3da46382656-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a7bfc82f-2271-4cb2-8426-f3da46382656" (UID: "a7bfc82f-2271-4cb2-8426-f3da46382656"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:56:00.861565 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:00.861540    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bebcceb2-1d61-4010-aa46-8f3d7ef0bc49-kube-api-access-qn9zv" (OuterVolumeSpecName: "kube-api-access-qn9zv") pod "bebcceb2-1d61-4010-aa46-8f3d7ef0bc49" (UID: "bebcceb2-1d61-4010-aa46-8f3d7ef0bc49"). InnerVolumeSpecName "kube-api-access-qn9zv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:56:00.861711 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:00.861569    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7bfc82f-2271-4cb2-8426-f3da46382656-kube-api-access-n9788" (OuterVolumeSpecName: "kube-api-access-n9788") pod "a7bfc82f-2271-4cb2-8426-f3da46382656" (UID: "a7bfc82f-2271-4cb2-8426-f3da46382656"). InnerVolumeSpecName "kube-api-access-n9788". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:56:00.961133 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:00.961013    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bebcceb2-1d61-4010-aa46-8f3d7ef0bc49-plugins-config-volume\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:56:00.961133 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:00.961041    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qn9zv\" (UniqueName: \"kubernetes.io/projected/bebcceb2-1d61-4010-aa46-8f3d7ef0bc49-kube-api-access-qn9zv\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:56:00.961133 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:00.961051    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7bfc82f-2271-4cb2-8426-f3da46382656-plugins-config-volume\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:56:00.961133 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:00.961061    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n9788\" (UniqueName: \"kubernetes.io/projected/a7bfc82f-2271-4cb2-8426-f3da46382656-kube-api-access-n9788\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:56:01.511555 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:01.511519    2581 generic.go:356] "Generic (PLEG): container finished" podID="bebcceb2-1d61-4010-aa46-8f3d7ef0bc49" containerID="be70f9c2407f47a8002e2070cd75e1d2b573b08c569ffddb21cc94397e461357" exitCode=0
Sep 09 20:56:01.512011 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:01.511558    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-69911538-694ddf7675-hdjh5"
Sep 09 20:56:01.512011 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:01.511604    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-69911538-694ddf7675-hdjh5" event={"ID":"bebcceb2-1d61-4010-aa46-8f3d7ef0bc49","Type":"ContainerDied","Data":"be70f9c2407f47a8002e2070cd75e1d2b573b08c569ffddb21cc94397e461357"}
Sep 09 20:56:01.512011 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:01.511652    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-69911538-694ddf7675-hdjh5" event={"ID":"bebcceb2-1d61-4010-aa46-8f3d7ef0bc49","Type":"ContainerDied","Data":"dfcb567fec85dc42d3f3535c7e085ff9968b1fe2365d7311a6e38f7ede8abbc9"}
Sep 09 20:56:01.512011 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:01.511691    2581 scope.go:117] "RemoveContainer" containerID="be70f9c2407f47a8002e2070cd75e1d2b573b08c569ffddb21cc94397e461357"
Sep 09 20:56:01.512981 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:01.512949    2581 generic.go:356] "Generic (PLEG): container finished" podID="a7bfc82f-2271-4cb2-8426-f3da46382656" containerID="e2d1fb162103046456e3e00ec9ecdeff37e2af03a500e5eea97b365b5fbef52d" exitCode=0
Sep 09 20:56:01.512981 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:01.512976    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-69911538-7cd58b96c4-g7tfn" event={"ID":"a7bfc82f-2271-4cb2-8426-f3da46382656","Type":"ContainerDied","Data":"e2d1fb162103046456e3e00ec9ecdeff37e2af03a500e5eea97b365b5fbef52d"}
Sep 09 20:56:01.513074 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:01.513006    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-69911538-7cd58b96c4-g7tfn" event={"ID":"a7bfc82f-2271-4cb2-8426-f3da46382656","Type":"ContainerDied","Data":"913824dfe4ad9c51c30b01850ee184442023dfe55409165e2df953fbad604efb"}
Sep 09 20:56:01.513074 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:01.513009    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-69911538-7cd58b96c4-g7tfn"
Sep 09 20:56:01.521932 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:01.521915    2581 scope.go:117] "RemoveContainer" containerID="be70f9c2407f47a8002e2070cd75e1d2b573b08c569ffddb21cc94397e461357"
Sep 09 20:56:01.522235 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:56:01.522212    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be70f9c2407f47a8002e2070cd75e1d2b573b08c569ffddb21cc94397e461357\": container with ID starting with be70f9c2407f47a8002e2070cd75e1d2b573b08c569ffddb21cc94397e461357 not found: ID does not exist" containerID="be70f9c2407f47a8002e2070cd75e1d2b573b08c569ffddb21cc94397e461357"
Sep 09 20:56:01.522308 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:01.522251    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be70f9c2407f47a8002e2070cd75e1d2b573b08c569ffddb21cc94397e461357"} err="failed to get container status \"be70f9c2407f47a8002e2070cd75e1d2b573b08c569ffddb21cc94397e461357\": rpc error: code = NotFound desc = could not find container \"be70f9c2407f47a8002e2070cd75e1d2b573b08c569ffddb21cc94397e461357\": container with ID starting with be70f9c2407f47a8002e2070cd75e1d2b573b08c569ffddb21cc94397e461357 not found: ID does not exist"
Sep 09 20:56:01.522308 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:01.522280    2581 scope.go:117] "RemoveContainer" containerID="e2d1fb162103046456e3e00ec9ecdeff37e2af03a500e5eea97b365b5fbef52d"
Sep 09 20:56:01.530523 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:01.530504    2581 scope.go:117] "RemoveContainer" containerID="e2d1fb162103046456e3e00ec9ecdeff37e2af03a500e5eea97b365b5fbef52d"
Sep 09 20:56:01.530840 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:56:01.530816    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2d1fb162103046456e3e00ec9ecdeff37e2af03a500e5eea97b365b5fbef52d\": container with ID starting with e2d1fb162103046456e3e00ec9ecdeff37e2af03a500e5eea97b365b5fbef52d not found: ID does not exist" containerID="e2d1fb162103046456e3e00ec9ecdeff37e2af03a500e5eea97b365b5fbef52d"
Sep 09 20:56:01.530954 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:01.530844    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2d1fb162103046456e3e00ec9ecdeff37e2af03a500e5eea97b365b5fbef52d"} err="failed to get container status \"e2d1fb162103046456e3e00ec9ecdeff37e2af03a500e5eea97b365b5fbef52d\": rpc error: code = NotFound desc = could not find container \"e2d1fb162103046456e3e00ec9ecdeff37e2af03a500e5eea97b365b5fbef52d\": container with ID starting with e2d1fb162103046456e3e00ec9ecdeff37e2af03a500e5eea97b365b5fbef52d not found: ID does not exist"
Sep 09 20:56:01.538139 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:01.538118    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-69911538-7cd58b96c4-g7tfn"]
Sep 09 20:56:01.547305 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:01.547284    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-69911538-7cd58b96c4-g7tfn"]
Sep 09 20:56:01.561031 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:01.561004    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-69911538-694ddf7675-hdjh5"]
Sep 09 20:56:01.565032 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:01.565012    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-69911538-694ddf7675-hdjh5"]
Sep 09 20:56:01.714763 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:01.714687    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-69911538-openshift-default-64b5648d48-l6dmd"]
Sep 09 20:56:01.714907 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:01.714865    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-69911538-openshift-default-64b5648d48-l6dmd" podUID="1b140704-2a77-4ca1-8f2a-0aae0d6fe251" containerName="istio-proxy" containerID="cri-o://29a4d5699f6ad7bc484893a639c26102644b0e02e5491e6cfedc676ed0d3d4b0" gracePeriod=30
Sep 09 20:56:02.484589 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:02.484557    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5"]
Sep 09 20:56:02.485204 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:02.485172    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bebcceb2-1d61-4010-aa46-8f3d7ef0bc49" containerName="payload-processing"
Sep 09 20:56:02.485204 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:02.485193    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="bebcceb2-1d61-4010-aa46-8f3d7ef0bc49" containerName="payload-processing"
Sep 09 20:56:02.485405 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:02.485227    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7bfc82f-2271-4cb2-8426-f3da46382656" containerName="payload-pre-processing"
Sep 09 20:56:02.485405 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:02.485235    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7bfc82f-2271-4cb2-8426-f3da46382656" containerName="payload-pre-processing"
Sep 09 20:56:02.485405 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:02.485243    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca2d0f92-ffdc-4602-bf1c-e360a0bdecf3" containerName="revoke-keys"
Sep 09 20:56:02.485405 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:02.485252    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca2d0f92-ffdc-4602-bf1c-e360a0bdecf3" containerName="revoke-keys"
Sep 09 20:56:02.485405 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:02.485365    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca2d0f92-ffdc-4602-bf1c-e360a0bdecf3" containerName="revoke-keys"
Sep 09 20:56:02.485405 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:02.485380    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="bebcceb2-1d61-4010-aa46-8f3d7ef0bc49" containerName="payload-processing"
Sep 09 20:56:02.485405 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:02.485389    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7bfc82f-2271-4cb2-8426-f3da46382656" containerName="payload-pre-processing"
Sep 09 20:56:02.490110 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:02.490078    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5"
Sep 09 20:56:02.492964 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:02.492943    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-c4796c6d-openshift-default-dockercfg-hlrvx\""
Sep 09 20:56:02.501347 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:02.501323    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5"]
Sep 09 20:56:02.579803 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:02.579746    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55236624-8426-4187-acfc-ac24200f14ca-istio-podinfo\") pod \"e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5\" (UID: \"55236624-8426-4187-acfc-ac24200f14ca\") " pod="openshift-ingress/e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5"
Sep 09 20:56:02.580349 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:02.579831    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55236624-8426-4187-acfc-ac24200f14ca-istio-envoy\") pod \"e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5\" (UID: \"55236624-8426-4187-acfc-ac24200f14ca\") " pod="openshift-ingress/e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5"
Sep 09 20:56:02.580349 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:02.579888    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55236624-8426-4187-acfc-ac24200f14ca-istio-data\") pod \"e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5\" (UID: \"55236624-8426-4187-acfc-ac24200f14ca\") " pod="openshift-ingress/e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5"
Sep 09 20:56:02.580349 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:02.579929    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55236624-8426-4187-acfc-ac24200f14ca-istio-token\") pod \"e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5\" (UID: \"55236624-8426-4187-acfc-ac24200f14ca\") " pod="openshift-ingress/e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5"
Sep 09 20:56:02.580349 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:02.579958    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55236624-8426-4187-acfc-ac24200f14ca-istiod-ca-cert\") pod \"e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5\" (UID: \"55236624-8426-4187-acfc-ac24200f14ca\") " pod="openshift-ingress/e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5"
Sep 09 20:56:02.580349 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:02.579997    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55236624-8426-4187-acfc-ac24200f14ca-workload-certs\") pod \"e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5\" (UID: \"55236624-8426-4187-acfc-ac24200f14ca\") " pod="openshift-ingress/e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5"
Sep 09 20:56:02.580349 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:02.580045    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55236624-8426-4187-acfc-ac24200f14ca-workload-socket\") pod \"e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5\" (UID: \"55236624-8426-4187-acfc-ac24200f14ca\") " pod="openshift-ingress/e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5"
Sep 09 20:56:02.580349 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:02.580088    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9v4p2\" (UniqueName: \"kubernetes.io/projected/55236624-8426-4187-acfc-ac24200f14ca-kube-api-access-9v4p2\") pod \"e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5\" (UID: \"55236624-8426-4187-acfc-ac24200f14ca\") " pod="openshift-ingress/e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5"
Sep 09 20:56:02.580349 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:02.580134    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55236624-8426-4187-acfc-ac24200f14ca-credential-socket\") pod \"e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5\" (UID: \"55236624-8426-4187-acfc-ac24200f14ca\") " pod="openshift-ingress/e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5"
Sep 09 20:56:02.680917 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:02.680882    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55236624-8426-4187-acfc-ac24200f14ca-istio-data\") pod \"e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5\" (UID: \"55236624-8426-4187-acfc-ac24200f14ca\") " pod="openshift-ingress/e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5"
Sep 09 20:56:02.681130 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:02.680934    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55236624-8426-4187-acfc-ac24200f14ca-istio-token\") pod \"e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5\" (UID: \"55236624-8426-4187-acfc-ac24200f14ca\") " pod="openshift-ingress/e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5"
Sep 09 20:56:02.681130 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:02.680967    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55236624-8426-4187-acfc-ac24200f14ca-istiod-ca-cert\") pod \"e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5\" (UID: \"55236624-8426-4187-acfc-ac24200f14ca\") " pod="openshift-ingress/e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5"
Sep 09 20:56:02.681130 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:02.681001    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55236624-8426-4187-acfc-ac24200f14ca-workload-certs\") pod \"e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5\" (UID: \"55236624-8426-4187-acfc-ac24200f14ca\") " pod="openshift-ingress/e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5"
Sep 09 20:56:02.681130 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:02.681044    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55236624-8426-4187-acfc-ac24200f14ca-workload-socket\") pod \"e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5\" (UID: \"55236624-8426-4187-acfc-ac24200f14ca\") " pod="openshift-ingress/e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5"
Sep 09 20:56:02.681130 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:02.681084    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9v4p2\" (UniqueName: \"kubernetes.io/projected/55236624-8426-4187-acfc-ac24200f14ca-kube-api-access-9v4p2\") pod \"e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5\" (UID: \"55236624-8426-4187-acfc-ac24200f14ca\") " pod="openshift-ingress/e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5"
Sep 09 20:56:02.681130 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:02.681127    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55236624-8426-4187-acfc-ac24200f14ca-credential-socket\") pod \"e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5\" (UID: \"55236624-8426-4187-acfc-ac24200f14ca\") " pod="openshift-ingress/e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5"
Sep 09 20:56:02.681477 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:02.681223    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55236624-8426-4187-acfc-ac24200f14ca-istio-podinfo\") pod \"e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5\" (UID: \"55236624-8426-4187-acfc-ac24200f14ca\") " pod="openshift-ingress/e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5"
Sep 09 20:56:02.681477 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:02.681284    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55236624-8426-4187-acfc-ac24200f14ca-istio-envoy\") pod \"e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5\" (UID: \"55236624-8426-4187-acfc-ac24200f14ca\") " pod="openshift-ingress/e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5"
Sep 09 20:56:02.681477 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:02.681401    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55236624-8426-4187-acfc-ac24200f14ca-istio-data\") pod \"e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5\" (UID: \"55236624-8426-4187-acfc-ac24200f14ca\") " pod="openshift-ingress/e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5"
Sep 09 20:56:02.681635 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:02.681504    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55236624-8426-4187-acfc-ac24200f14ca-workload-socket\") pod \"e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5\" (UID: \"55236624-8426-4187-acfc-ac24200f14ca\") " pod="openshift-ingress/e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5"
Sep 09 20:56:02.681635 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:02.681525    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55236624-8426-4187-acfc-ac24200f14ca-credential-socket\") pod \"e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5\" (UID: \"55236624-8426-4187-acfc-ac24200f14ca\") " pod="openshift-ingress/e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5"
Sep 09 20:56:02.681750 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:02.681709    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55236624-8426-4187-acfc-ac24200f14ca-istiod-ca-cert\") pod \"e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5\" (UID: \"55236624-8426-4187-acfc-ac24200f14ca\") " pod="openshift-ingress/e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5"
Sep 09 20:56:02.681750 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:02.681715    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55236624-8426-4187-acfc-ac24200f14ca-workload-certs\") pod \"e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5\" (UID: \"55236624-8426-4187-acfc-ac24200f14ca\") " pod="openshift-ingress/e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5"
Sep 09 20:56:02.683561 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:02.683536    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55236624-8426-4187-acfc-ac24200f14ca-istio-envoy\") pod \"e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5\" (UID: \"55236624-8426-4187-acfc-ac24200f14ca\") " pod="openshift-ingress/e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5"
Sep 09 20:56:02.683761 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:02.683744    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55236624-8426-4187-acfc-ac24200f14ca-istio-podinfo\") pod \"e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5\" (UID: \"55236624-8426-4187-acfc-ac24200f14ca\") " pod="openshift-ingress/e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5"
Sep 09 20:56:02.698333 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:02.698307    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55236624-8426-4187-acfc-ac24200f14ca-istio-token\") pod \"e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5\" (UID: \"55236624-8426-4187-acfc-ac24200f14ca\") " pod="openshift-ingress/e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5"
Sep 09 20:56:02.699014 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:02.698991    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9v4p2\" (UniqueName: \"kubernetes.io/projected/55236624-8426-4187-acfc-ac24200f14ca-kube-api-access-9v4p2\") pod \"e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5\" (UID: \"55236624-8426-4187-acfc-ac24200f14ca\") " pod="openshift-ingress/e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5"
Sep 09 20:56:02.800959 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:02.800877    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5"
Sep 09 20:56:02.821037 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:02.821010    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7bfc82f-2271-4cb2-8426-f3da46382656" path="/var/lib/kubelet/pods/a7bfc82f-2271-4cb2-8426-f3da46382656/volumes"
Sep 09 20:56:02.821450 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:02.821435    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bebcceb2-1d61-4010-aa46-8f3d7ef0bc49" path="/var/lib/kubelet/pods/bebcceb2-1d61-4010-aa46-8f3d7ef0bc49/volumes"
Sep 09 20:56:03.106801 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:03.106776    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5"]
Sep 09 20:56:03.108653 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:56:03.108626    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55236624_8426_4187_acfc_ac24200f14ca.slice/crio-80c1903192ba22a448714f6b16e6326358259963448dfc3aad52741693d219c2 WatchSource:0}: Error finding container 80c1903192ba22a448714f6b16e6326358259963448dfc3aad52741693d219c2: Status 404 returned error can't find the container with id 80c1903192ba22a448714f6b16e6326358259963448dfc3aad52741693d219c2
Sep 09 20:56:03.110996 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:03.110963    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:56:03.111085 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:03.111030    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:56:03.111085 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:03.111058    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:56:03.526860 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:03.526821    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5" event={"ID":"55236624-8426-4187-acfc-ac24200f14ca","Type":"ContainerStarted","Data":"71ebfcbb62d36726a3927f0bdb5937ace71d84edd1920927eba6a8af8ab0f971"}
Sep 09 20:56:03.527058 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:03.526886    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5" event={"ID":"55236624-8426-4187-acfc-ac24200f14ca","Type":"ContainerStarted","Data":"80c1903192ba22a448714f6b16e6326358259963448dfc3aad52741693d219c2"}
Sep 09 20:56:03.547354 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:03.547304    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5" podStartSLOduration=1.54728468 podStartE2EDuration="1.54728468s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:56:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:56:03.546526803 +0000 UTC m=+925.174154688" watchObservedRunningTime="2026-09-09 20:56:03.54728468 +0000 UTC m=+925.174912553"
Sep 09 20:56:03.801747 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:03.801662    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5"
Sep 09 20:56:04.806780 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:04.806746    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5"
Sep 09 20:56:05.537730 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:05.537703    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5"
Sep 09 20:56:05.538751 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:05.538730    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5"
Sep 09 20:56:06.966091 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:06.966062    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-69911538-openshift-default-64b5648d48-l6dmd"
Sep 09 20:56:07.001799 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.001772    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-c4796c6d-bb9799dbb-xm2m8"]
Sep 09 20:56:07.002212 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.002198    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b140704-2a77-4ca1-8f2a-0aae0d6fe251" containerName="istio-proxy"
Sep 09 20:56:07.002267 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.002213    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b140704-2a77-4ca1-8f2a-0aae0d6fe251" containerName="istio-proxy"
Sep 09 20:56:07.002267 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.002232    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b140704-2a77-4ca1-8f2a-0aae0d6fe251" containerName="istio-proxy"
Sep 09 20:56:07.002267 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.002238    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b140704-2a77-4ca1-8f2a-0aae0d6fe251" containerName="istio-proxy"
Sep 09 20:56:07.002379 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.002295    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b140704-2a77-4ca1-8f2a-0aae0d6fe251" containerName="istio-proxy"
Sep 09 20:56:07.002379 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.002302    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b140704-2a77-4ca1-8f2a-0aae0d6fe251" containerName="istio-proxy"
Sep 09 20:56:07.005208 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.005191    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-c4796c6d-bb9799dbb-xm2m8"
Sep 09 20:56:07.014943 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.014920    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-c4796c6d-dockercfg-7k5tf\""
Sep 09 20:56:07.015064 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.014910    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-c4796c6d\""
Sep 09 20:56:07.026559 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.026530    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jf59j\" (UniqueName: \"kubernetes.io/projected/1b140704-2a77-4ca1-8f2a-0aae0d6fe251-kube-api-access-jf59j\") pod \"1b140704-2a77-4ca1-8f2a-0aae0d6fe251\" (UID: \"1b140704-2a77-4ca1-8f2a-0aae0d6fe251\") "
Sep 09 20:56:07.026816 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.026799    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b140704-2a77-4ca1-8f2a-0aae0d6fe251-workload-socket\") pod \"1b140704-2a77-4ca1-8f2a-0aae0d6fe251\" (UID: \"1b140704-2a77-4ca1-8f2a-0aae0d6fe251\") "
Sep 09 20:56:07.026892 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.026855    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b140704-2a77-4ca1-8f2a-0aae0d6fe251-istio-data\") pod \"1b140704-2a77-4ca1-8f2a-0aae0d6fe251\" (UID: \"1b140704-2a77-4ca1-8f2a-0aae0d6fe251\") "
Sep 09 20:56:07.026951 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.026917    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b140704-2a77-4ca1-8f2a-0aae0d6fe251-istio-podinfo\") pod \"1b140704-2a77-4ca1-8f2a-0aae0d6fe251\" (UID: \"1b140704-2a77-4ca1-8f2a-0aae0d6fe251\") "
Sep 09 20:56:07.027015 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.026953    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b140704-2a77-4ca1-8f2a-0aae0d6fe251-istio-token\") pod \"1b140704-2a77-4ca1-8f2a-0aae0d6fe251\" (UID: \"1b140704-2a77-4ca1-8f2a-0aae0d6fe251\") "
Sep 09 20:56:07.027015 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.026995    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b140704-2a77-4ca1-8f2a-0aae0d6fe251-workload-certs\") pod \"1b140704-2a77-4ca1-8f2a-0aae0d6fe251\" (UID: \"1b140704-2a77-4ca1-8f2a-0aae0d6fe251\") "
Sep 09 20:56:07.027144 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.027054    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b140704-2a77-4ca1-8f2a-0aae0d6fe251-credential-socket\") pod \"1b140704-2a77-4ca1-8f2a-0aae0d6fe251\" (UID: \"1b140704-2a77-4ca1-8f2a-0aae0d6fe251\") "
Sep 09 20:56:07.027144 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.027128    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b140704-2a77-4ca1-8f2a-0aae0d6fe251-istio-envoy\") pod \"1b140704-2a77-4ca1-8f2a-0aae0d6fe251\" (UID: \"1b140704-2a77-4ca1-8f2a-0aae0d6fe251\") "
Sep 09 20:56:07.027253 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.027164    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b140704-2a77-4ca1-8f2a-0aae0d6fe251-istiod-ca-cert\") pod \"1b140704-2a77-4ca1-8f2a-0aae0d6fe251\" (UID: \"1b140704-2a77-4ca1-8f2a-0aae0d6fe251\") "
Sep 09 20:56:07.027253 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.027193    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b140704-2a77-4ca1-8f2a-0aae0d6fe251-istio-data" (OuterVolumeSpecName: "istio-data") pod "1b140704-2a77-4ca1-8f2a-0aae0d6fe251" (UID: "1b140704-2a77-4ca1-8f2a-0aae0d6fe251"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:56:07.027356 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.027303    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7cfc9803-8743-45e0-b8ad-405c60f461f7-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-c4796c6d-bb9799dbb-xm2m8\" (UID: \"7cfc9803-8743-45e0-b8ad-405c60f461f7\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-c4796c6d-bb9799dbb-xm2m8"
Sep 09 20:56:07.027411 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.027389    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b140704-2a77-4ca1-8f2a-0aae0d6fe251-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1b140704-2a77-4ca1-8f2a-0aae0d6fe251" (UID: "1b140704-2a77-4ca1-8f2a-0aae0d6fe251"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:56:07.027466 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.027411    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvkvv\" (UniqueName: \"kubernetes.io/projected/7cfc9803-8743-45e0-b8ad-405c60f461f7-kube-api-access-cvkvv\") pod \"payload-pre-processing-e2e-derive-c4796c6d-bb9799dbb-xm2m8\" (UID: \"7cfc9803-8743-45e0-b8ad-405c60f461f7\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-c4796c6d-bb9799dbb-xm2m8"
Sep 09 20:56:07.027515 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.027475    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b140704-2a77-4ca1-8f2a-0aae0d6fe251-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1b140704-2a77-4ca1-8f2a-0aae0d6fe251" (UID: "1b140704-2a77-4ca1-8f2a-0aae0d6fe251"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:56:07.027629 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.027566    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b140704-2a77-4ca1-8f2a-0aae0d6fe251-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1b140704-2a77-4ca1-8f2a-0aae0d6fe251" (UID: "1b140704-2a77-4ca1-8f2a-0aae0d6fe251"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:56:07.027704 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.027638    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b140704-2a77-4ca1-8f2a-0aae0d6fe251-workload-certs\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:56:07.027704 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.027656    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b140704-2a77-4ca1-8f2a-0aae0d6fe251-credential-socket\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:56:07.027704 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.027670    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b140704-2a77-4ca1-8f2a-0aae0d6fe251-workload-socket\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:56:07.027704 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.027682    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b140704-2a77-4ca1-8f2a-0aae0d6fe251-istio-data\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:56:07.028009 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.027986    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b140704-2a77-4ca1-8f2a-0aae0d6fe251-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1b140704-2a77-4ca1-8f2a-0aae0d6fe251" (UID: "1b140704-2a77-4ca1-8f2a-0aae0d6fe251"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:56:07.029510 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.029477    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b140704-2a77-4ca1-8f2a-0aae0d6fe251-kube-api-access-jf59j" (OuterVolumeSpecName: "kube-api-access-jf59j") pod "1b140704-2a77-4ca1-8f2a-0aae0d6fe251" (UID: "1b140704-2a77-4ca1-8f2a-0aae0d6fe251"). InnerVolumeSpecName "kube-api-access-jf59j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:56:07.030023 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.029974    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b140704-2a77-4ca1-8f2a-0aae0d6fe251-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1b140704-2a77-4ca1-8f2a-0aae0d6fe251" (UID: "1b140704-2a77-4ca1-8f2a-0aae0d6fe251"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:56:07.030137 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.030020    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b140704-2a77-4ca1-8f2a-0aae0d6fe251-istio-token" (OuterVolumeSpecName: "istio-token") pod "1b140704-2a77-4ca1-8f2a-0aae0d6fe251" (UID: "1b140704-2a77-4ca1-8f2a-0aae0d6fe251"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:56:07.030559 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.030541    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1b140704-2a77-4ca1-8f2a-0aae0d6fe251-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1b140704-2a77-4ca1-8f2a-0aae0d6fe251" (UID: "1b140704-2a77-4ca1-8f2a-0aae0d6fe251"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 20:56:07.035822 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.035798    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-c4796c6d-bb9799dbb-xm2m8"]
Sep 09 20:56:07.054947 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.054927    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-c4796c6d-678cf978b9-8xgwn"]
Sep 09 20:56:07.058964 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.058948    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-c4796c6d-678cf978b9-8xgwn"
Sep 09 20:56:07.103870 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.103793    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-c4796c6d-678cf978b9-8xgwn"]
Sep 09 20:56:07.129572 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.128949    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvkvv\" (UniqueName: \"kubernetes.io/projected/7cfc9803-8743-45e0-b8ad-405c60f461f7-kube-api-access-cvkvv\") pod \"payload-pre-processing-e2e-derive-c4796c6d-bb9799dbb-xm2m8\" (UID: \"7cfc9803-8743-45e0-b8ad-405c60f461f7\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-c4796c6d-bb9799dbb-xm2m8"
Sep 09 20:56:07.129572 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.129088    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xt48l\" (UniqueName: \"kubernetes.io/projected/d6e642c9-25d2-4914-bb02-42e3b60127fb-kube-api-access-xt48l\") pod \"payload-processing-e2e-derive-c4796c6d-678cf978b9-8xgwn\" (UID: \"d6e642c9-25d2-4914-bb02-42e3b60127fb\") " pod="openshift-ingress/payload-processing-e2e-derive-c4796c6d-678cf978b9-8xgwn"
Sep 09 20:56:07.129572 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.129174    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7cfc9803-8743-45e0-b8ad-405c60f461f7-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-c4796c6d-bb9799dbb-xm2m8\" (UID: \"7cfc9803-8743-45e0-b8ad-405c60f461f7\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-c4796c6d-bb9799dbb-xm2m8"
Sep 09 20:56:07.129572 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.129199    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d6e642c9-25d2-4914-bb02-42e3b60127fb-plugins-config-volume\") pod \"payload-processing-e2e-derive-c4796c6d-678cf978b9-8xgwn\" (UID: \"d6e642c9-25d2-4914-bb02-42e3b60127fb\") " pod="openshift-ingress/payload-processing-e2e-derive-c4796c6d-678cf978b9-8xgwn"
Sep 09 20:56:07.129572 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.129288    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jf59j\" (UniqueName: \"kubernetes.io/projected/1b140704-2a77-4ca1-8f2a-0aae0d6fe251-kube-api-access-jf59j\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:56:07.129572 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.129304    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b140704-2a77-4ca1-8f2a-0aae0d6fe251-istio-podinfo\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:56:07.129572 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.129318    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b140704-2a77-4ca1-8f2a-0aae0d6fe251-istio-token\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:56:07.129572 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.129330    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b140704-2a77-4ca1-8f2a-0aae0d6fe251-istio-envoy\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:56:07.129572 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.129342    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b140704-2a77-4ca1-8f2a-0aae0d6fe251-istiod-ca-cert\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:56:07.130333 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.130161    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7cfc9803-8743-45e0-b8ad-405c60f461f7-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-c4796c6d-bb9799dbb-xm2m8\" (UID: \"7cfc9803-8743-45e0-b8ad-405c60f461f7\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-c4796c6d-bb9799dbb-xm2m8"
Sep 09 20:56:07.145751 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.145726    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvkvv\" (UniqueName: \"kubernetes.io/projected/7cfc9803-8743-45e0-b8ad-405c60f461f7-kube-api-access-cvkvv\") pod \"payload-pre-processing-e2e-derive-c4796c6d-bb9799dbb-xm2m8\" (UID: \"7cfc9803-8743-45e0-b8ad-405c60f461f7\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-c4796c6d-bb9799dbb-xm2m8"
Sep 09 20:56:07.229834 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.229800    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xt48l\" (UniqueName: \"kubernetes.io/projected/d6e642c9-25d2-4914-bb02-42e3b60127fb-kube-api-access-xt48l\") pod \"payload-processing-e2e-derive-c4796c6d-678cf978b9-8xgwn\" (UID: \"d6e642c9-25d2-4914-bb02-42e3b60127fb\") " pod="openshift-ingress/payload-processing-e2e-derive-c4796c6d-678cf978b9-8xgwn"
Sep 09 20:56:07.230000 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.229849    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d6e642c9-25d2-4914-bb02-42e3b60127fb-plugins-config-volume\") pod \"payload-processing-e2e-derive-c4796c6d-678cf978b9-8xgwn\" (UID: \"d6e642c9-25d2-4914-bb02-42e3b60127fb\") " pod="openshift-ingress/payload-processing-e2e-derive-c4796c6d-678cf978b9-8xgwn"
Sep 09 20:56:07.230552 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.230532    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d6e642c9-25d2-4914-bb02-42e3b60127fb-plugins-config-volume\") pod \"payload-processing-e2e-derive-c4796c6d-678cf978b9-8xgwn\" (UID: \"d6e642c9-25d2-4914-bb02-42e3b60127fb\") " pod="openshift-ingress/payload-processing-e2e-derive-c4796c6d-678cf978b9-8xgwn"
Sep 09 20:56:07.244314 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.244285    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xt48l\" (UniqueName: \"kubernetes.io/projected/d6e642c9-25d2-4914-bb02-42e3b60127fb-kube-api-access-xt48l\") pod \"payload-processing-e2e-derive-c4796c6d-678cf978b9-8xgwn\" (UID: \"d6e642c9-25d2-4914-bb02-42e3b60127fb\") " pod="openshift-ingress/payload-processing-e2e-derive-c4796c6d-678cf978b9-8xgwn"
Sep 09 20:56:07.315694 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.315608    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-c4796c6d-bb9799dbb-xm2m8"
Sep 09 20:56:07.370406 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.370375    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-c4796c6d-678cf978b9-8xgwn"
Sep 09 20:56:07.549691 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.549653    2581 generic.go:356] "Generic (PLEG): container finished" podID="1b140704-2a77-4ca1-8f2a-0aae0d6fe251" containerID="29a4d5699f6ad7bc484893a639c26102644b0e02e5491e6cfedc676ed0d3d4b0" exitCode=0
Sep 09 20:56:07.549835 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.549717    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-69911538-openshift-default-64b5648d48-l6dmd"
Sep 09 20:56:07.549835 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.549745    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-69911538-openshift-default-64b5648d48-l6dmd" event={"ID":"1b140704-2a77-4ca1-8f2a-0aae0d6fe251","Type":"ContainerDied","Data":"29a4d5699f6ad7bc484893a639c26102644b0e02e5491e6cfedc676ed0d3d4b0"}
Sep 09 20:56:07.549835 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.549802    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-69911538-openshift-default-64b5648d48-l6dmd" event={"ID":"1b140704-2a77-4ca1-8f2a-0aae0d6fe251","Type":"ContainerDied","Data":"c5d8e9f98003d380d3d0484de58b408655aae5a26d8513e76378e7f9e5fe0912"}
Sep 09 20:56:07.549835 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.549831    2581 scope.go:117] "RemoveContainer" containerID="29a4d5699f6ad7bc484893a639c26102644b0e02e5491e6cfedc676ed0d3d4b0"
Sep 09 20:56:07.558954 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.558930    2581 scope.go:117] "RemoveContainer" containerID="8bbddb76f3e6608bb0afc5390ccc86e5e6ca8d657bf9537396b95c7232b1c3b4"
Sep 09 20:56:07.567069 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.567051    2581 scope.go:117] "RemoveContainer" containerID="29a4d5699f6ad7bc484893a639c26102644b0e02e5491e6cfedc676ed0d3d4b0"
Sep 09 20:56:07.567392 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:56:07.567372    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29a4d5699f6ad7bc484893a639c26102644b0e02e5491e6cfedc676ed0d3d4b0\": container with ID starting with 29a4d5699f6ad7bc484893a639c26102644b0e02e5491e6cfedc676ed0d3d4b0 not found: ID does not exist" containerID="29a4d5699f6ad7bc484893a639c26102644b0e02e5491e6cfedc676ed0d3d4b0"
Sep 09 20:56:07.567459 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.567406    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29a4d5699f6ad7bc484893a639c26102644b0e02e5491e6cfedc676ed0d3d4b0"} err="failed to get container status \"29a4d5699f6ad7bc484893a639c26102644b0e02e5491e6cfedc676ed0d3d4b0\": rpc error: code = NotFound desc = could not find container \"29a4d5699f6ad7bc484893a639c26102644b0e02e5491e6cfedc676ed0d3d4b0\": container with ID starting with 29a4d5699f6ad7bc484893a639c26102644b0e02e5491e6cfedc676ed0d3d4b0 not found: ID does not exist"
Sep 09 20:56:07.567459 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.567431    2581 scope.go:117] "RemoveContainer" containerID="8bbddb76f3e6608bb0afc5390ccc86e5e6ca8d657bf9537396b95c7232b1c3b4"
Sep 09 20:56:07.567737 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:56:07.567719    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8bbddb76f3e6608bb0afc5390ccc86e5e6ca8d657bf9537396b95c7232b1c3b4\": container with ID starting with 8bbddb76f3e6608bb0afc5390ccc86e5e6ca8d657bf9537396b95c7232b1c3b4 not found: ID does not exist" containerID="8bbddb76f3e6608bb0afc5390ccc86e5e6ca8d657bf9537396b95c7232b1c3b4"
Sep 09 20:56:07.567782 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.567743    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8bbddb76f3e6608bb0afc5390ccc86e5e6ca8d657bf9537396b95c7232b1c3b4"} err="failed to get container status \"8bbddb76f3e6608bb0afc5390ccc86e5e6ca8d657bf9537396b95c7232b1c3b4\": rpc error: code = NotFound desc = could not find container \"8bbddb76f3e6608bb0afc5390ccc86e5e6ca8d657bf9537396b95c7232b1c3b4\": container with ID starting with 8bbddb76f3e6608bb0afc5390ccc86e5e6ca8d657bf9537396b95c7232b1c3b4 not found: ID does not exist"
Sep 09 20:56:07.598162 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.598139    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-69911538-openshift-default-64b5648d48-l6dmd"]
Sep 09 20:56:07.608765 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.608745    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-69911538-openshift-default-64b5648d48-l6dmd"]
Sep 09 20:56:07.627062 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.627038    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-c4796c6d-bb9799dbb-xm2m8"]
Sep 09 20:56:07.628437 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:56:07.628415    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cfc9803_8743_45e0_b8ad_405c60f461f7.slice/crio-b416b3b4209808826cd78d8992466bb3ef51e90667ba1fe4a38afd697bdb6ccb WatchSource:0}: Error finding container b416b3b4209808826cd78d8992466bb3ef51e90667ba1fe4a38afd697bdb6ccb: Status 404 returned error can't find the container with id b416b3b4209808826cd78d8992466bb3ef51e90667ba1fe4a38afd697bdb6ccb
Sep 09 20:56:07.664481 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:07.664461    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-c4796c6d-678cf978b9-8xgwn"]
Sep 09 20:56:07.666594 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:56:07.666572    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6e642c9_25d2_4914_bb02_42e3b60127fb.slice/crio-ace9769b5c0265f38c93c3762a72531a68aee935e7a0226a7391e9fb8978bd01 WatchSource:0}: Error finding container ace9769b5c0265f38c93c3762a72531a68aee935e7a0226a7391e9fb8978bd01: Status 404 returned error can't find the container with id ace9769b5c0265f38c93c3762a72531a68aee935e7a0226a7391e9fb8978bd01
Sep 09 20:56:08.558292 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:08.558252    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-c4796c6d-678cf978b9-8xgwn" event={"ID":"d6e642c9-25d2-4914-bb02-42e3b60127fb","Type":"ContainerStarted","Data":"aec5769411330af812285a7443ce0e248291c27d50ac706f7a1dcb3f23ee5f35"}
Sep 09 20:56:08.558786 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:08.558316    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-c4796c6d-678cf978b9-8xgwn"
Sep 09 20:56:08.558786 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:08.558341    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-c4796c6d-678cf978b9-8xgwn" event={"ID":"d6e642c9-25d2-4914-bb02-42e3b60127fb","Type":"ContainerStarted","Data":"ace9769b5c0265f38c93c3762a72531a68aee935e7a0226a7391e9fb8978bd01"}
Sep 09 20:56:08.560193 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:08.560163    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-c4796c6d-bb9799dbb-xm2m8" event={"ID":"7cfc9803-8743-45e0-b8ad-405c60f461f7","Type":"ContainerStarted","Data":"9e3e5390532af4f3db50cf87582ba8ae4b8598b5f6d741535f7aebd9016031ef"}
Sep 09 20:56:08.560344 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:08.560215    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-c4796c6d-bb9799dbb-xm2m8" event={"ID":"7cfc9803-8743-45e0-b8ad-405c60f461f7","Type":"ContainerStarted","Data":"b416b3b4209808826cd78d8992466bb3ef51e90667ba1fe4a38afd697bdb6ccb"}
Sep 09 20:56:08.560344 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:08.560243    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-c4796c6d-bb9799dbb-xm2m8"
Sep 09 20:56:08.586198 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:08.586135    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-c4796c6d-678cf978b9-8xgwn" podStartSLOduration=1.586113599 podStartE2EDuration="1.586113599s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:56:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:56:08.582091439 +0000 UTC m=+930.209719305" watchObservedRunningTime="2026-09-09 20:56:08.586113599 +0000 UTC m=+930.213741467"
Sep 09 20:56:08.604942 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:08.604891    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-c4796c6d-bb9799dbb-xm2m8" podStartSLOduration=2.604875179 podStartE2EDuration="2.604875179s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:56:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:56:08.602654303 +0000 UTC m=+930.230282181" watchObservedRunningTime="2026-09-09 20:56:08.604875179 +0000 UTC m=+930.232503080"
Sep 09 20:56:08.820511 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:08.820432    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b140704-2a77-4ca1-8f2a-0aae0d6fe251" path="/var/lib/kubelet/pods/1b140704-2a77-4ca1-8f2a-0aae0d6fe251/volumes"
Sep 09 20:56:17.385330 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:17.385292    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-864e4e-jnvj2"]
Sep 09 20:56:17.390332 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:17.390309    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-864e4e-jnvj2"
Sep 09 20:56:17.393178 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:17.393155    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-glz44\""
Sep 09 20:56:17.398807 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:17.398755    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-864e4e-jnvj2"]
Sep 09 20:56:17.429195 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:17.429166    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7535dad-4568-408a-837e-f8e6613c1fc7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-864e4e-jnvj2\" (UID: \"d7535dad-4568-408a-837e-f8e6613c1fc7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-864e4e-jnvj2"
Sep 09 20:56:17.530318 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:17.530284    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7535dad-4568-408a-837e-f8e6613c1fc7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-864e4e-jnvj2\" (UID: \"d7535dad-4568-408a-837e-f8e6613c1fc7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-864e4e-jnvj2"
Sep 09 20:56:17.530943 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:17.530922    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7535dad-4568-408a-837e-f8e6613c1fc7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-864e4e-jnvj2\" (UID: \"d7535dad-4568-408a-837e-f8e6613c1fc7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-864e4e-jnvj2"
Sep 09 20:56:17.703349 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:17.703266    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-864e4e-jnvj2"
Sep 09 20:56:17.982952 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:17.982923    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-864e4e-jnvj2"]
Sep 09 20:56:17.984964 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:56:17.984938    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7535dad_4568_408a_837e_f8e6613c1fc7.slice/crio-b3db0c78069df804378c56693bcbb5efcab67524d0a379d6103d08c30d4deda5 WatchSource:0}: Error finding container b3db0c78069df804378c56693bcbb5efcab67524d0a379d6103d08c30d4deda5: Status 404 returned error can't find the container with id b3db0c78069df804378c56693bcbb5efcab67524d0a379d6103d08c30d4deda5
Sep 09 20:56:18.626545 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:18.626449    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-864e4e-jnvj2" event={"ID":"d7535dad-4568-408a-837e-f8e6613c1fc7","Type":"ContainerStarted","Data":"40b8ebe399738336eeab361350b9c0309da4812e47dc34d76ce4f331db1c0eeb"}
Sep 09 20:56:18.626545 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:18.626530    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-864e4e-jnvj2" event={"ID":"d7535dad-4568-408a-837e-f8e6613c1fc7","Type":"ContainerStarted","Data":"b3db0c78069df804378c56693bcbb5efcab67524d0a379d6103d08c30d4deda5"}
Sep 09 20:56:18.646321 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:18.646273    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-864e4e-jnvj2" podStartSLOduration=1.6462575 podStartE2EDuration="1.6462575s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:56:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:56:18.643714065 +0000 UTC m=+940.271341938" watchObservedRunningTime="2026-09-09 20:56:18.6462575 +0000 UTC m=+940.273885374"
Sep 09 20:56:19.568076 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:19.568043    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-c4796c6d-bb9799dbb-xm2m8"
Sep 09 20:56:19.568272 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:19.568254    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-c4796c6d-678cf978b9-8xgwn"
Sep 09 20:56:19.641761 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:19.641729    2581 generic.go:356] "Generic (PLEG): container finished" podID="d7535dad-4568-408a-837e-f8e6613c1fc7" containerID="40b8ebe399738336eeab361350b9c0309da4812e47dc34d76ce4f331db1c0eeb" exitCode=0
Sep 09 20:56:19.642280 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:19.641771    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-864e4e-jnvj2" event={"ID":"d7535dad-4568-408a-837e-f8e6613c1fc7","Type":"ContainerDied","Data":"40b8ebe399738336eeab361350b9c0309da4812e47dc34d76ce4f331db1c0eeb"}
Sep 09 20:56:20.789397 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:20.789366    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-864e4e-jnvj2"
Sep 09 20:56:20.864025 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:20.863997    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7535dad-4568-408a-837e-f8e6613c1fc7-maas-api-service-ca\") pod \"d7535dad-4568-408a-837e-f8e6613c1fc7\" (UID: \"d7535dad-4568-408a-837e-f8e6613c1fc7\") "
Sep 09 20:56:20.864385 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:20.864360    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7535dad-4568-408a-837e-f8e6613c1fc7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d7535dad-4568-408a-837e-f8e6613c1fc7" (UID: "d7535dad-4568-408a-837e-f8e6613c1fc7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:56:20.965608 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:20.965525    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7535dad-4568-408a-837e-f8e6613c1fc7-maas-api-service-ca\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:56:21.653298 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:21.653268    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-864e4e-jnvj2"
Sep 09 20:56:21.653487 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:21.653297    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-864e4e-jnvj2" event={"ID":"d7535dad-4568-408a-837e-f8e6613c1fc7","Type":"ContainerDied","Data":"b3db0c78069df804378c56693bcbb5efcab67524d0a379d6103d08c30d4deda5"}
Sep 09 20:56:21.653487 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:21.653354    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b3db0c78069df804378c56693bcbb5efcab67524d0a379d6103d08c30d4deda5"
Sep 09 20:56:26.682632 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:26.682603    2581 generic.go:356] "Generic (PLEG): container finished" podID="55236624-8426-4187-acfc-ac24200f14ca" containerID="71ebfcbb62d36726a3927f0bdb5937ace71d84edd1920927eba6a8af8ab0f971" exitCode=0
Sep 09 20:56:26.683035 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:26.682645    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5" event={"ID":"55236624-8426-4187-acfc-ac24200f14ca","Type":"ContainerDied","Data":"71ebfcbb62d36726a3927f0bdb5937ace71d84edd1920927eba6a8af8ab0f971"}
Sep 09 20:56:26.683077 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:26.683036    2581 scope.go:117] "RemoveContainer" containerID="71ebfcbb62d36726a3927f0bdb5937ace71d84edd1920927eba6a8af8ab0f971"
Sep 09 20:56:26.684258 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:26.684228    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:56:26.684333 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:26.684296    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:56:26.684333 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:26.684326    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:56:27.689536 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:27.689500    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5" event={"ID":"55236624-8426-4187-acfc-ac24200f14ca","Type":"ContainerStarted","Data":"11e2bd1a4f9c29382d70e46348bd32c28b32fe88cc7335f53cc11837873a06b4"}
Sep 09 20:56:27.802005 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:27.801961    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5"
Sep 09 20:56:27.802235 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:27.802119    2581 patch_prober.go:28] interesting pod/e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Sep 09 20:56:27.802235 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:27.802170    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5" podUID="55236624-8426-4187-acfc-ac24200f14ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Sep 09 20:56:28.331437 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:28.331405    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-864e4e-jnvj2"]
Sep 09 20:56:28.337707 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:28.337679    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-864e4e-jnvj2"]
Sep 09 20:56:28.801445 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:28.801414    2581 patch_prober.go:28] interesting pod/e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Sep 09 20:56:28.802027 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:28.801989    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5" podUID="55236624-8426-4187-acfc-ac24200f14ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Sep 09 20:56:28.828392 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:28.828354    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7535dad-4568-408a-837e-f8e6613c1fc7" path="/var/lib/kubelet/pods/d7535dad-4568-408a-837e-f8e6613c1fc7/volumes"
Sep 09 20:56:29.801632 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:29.801591    2581 patch_prober.go:28] interesting pod/e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Sep 09 20:56:29.802399 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:29.802363    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5" podUID="55236624-8426-4187-acfc-ac24200f14ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Sep 09 20:56:30.802446 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:30.801949    2581 patch_prober.go:28] interesting pod/e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Sep 09 20:56:30.802446 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:30.802020    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5" podUID="55236624-8426-4187-acfc-ac24200f14ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Sep 09 20:56:31.801696 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:31.801652    2581 patch_prober.go:28] interesting pod/e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Sep 09 20:56:31.801914 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:31.801724    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5" podUID="55236624-8426-4187-acfc-ac24200f14ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Sep 09 20:56:32.801668 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:32.801631    2581 patch_prober.go:28] interesting pod/e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Sep 09 20:56:32.802084 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:32.801689    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5" podUID="55236624-8426-4187-acfc-ac24200f14ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Sep 09 20:56:33.805847 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:33.805808    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5"
Sep 09 20:56:33.806394 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:33.806139    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5"
Sep 09 20:56:33.806928 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:33.806911    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5"
Sep 09 20:56:34.913417 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:34.913383    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c4796c6d-gwcnp"]
Sep 09 20:56:34.913876 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:34.913862    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7535dad-4568-408a-837e-f8e6613c1fc7" containerName="revoke-keys"
Sep 09 20:56:34.913923 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:34.913878    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7535dad-4568-408a-837e-f8e6613c1fc7" containerName="revoke-keys"
Sep 09 20:56:34.913968 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:34.913958    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7535dad-4568-408a-837e-f8e6613c1fc7" containerName="revoke-keys"
Sep 09 20:56:34.917576 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:34.917554    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c4796c6d-gwcnp"
Sep 09 20:56:34.920016 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:34.919997    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-glz44\""
Sep 09 20:56:34.924682 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:34.924655    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c4796c6d-gwcnp"]
Sep 09 20:56:35.020950 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:35.020916    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29df0259-6442-4186-8cda-b3b39fd1eb7b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-c4796c6d-gwcnp\" (UID: \"29df0259-6442-4186-8cda-b3b39fd1eb7b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c4796c6d-gwcnp"
Sep 09 20:56:35.121659 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:35.121617    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29df0259-6442-4186-8cda-b3b39fd1eb7b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-c4796c6d-gwcnp\" (UID: \"29df0259-6442-4186-8cda-b3b39fd1eb7b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c4796c6d-gwcnp"
Sep 09 20:56:35.122275 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:35.122256    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29df0259-6442-4186-8cda-b3b39fd1eb7b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-c4796c6d-gwcnp\" (UID: \"29df0259-6442-4186-8cda-b3b39fd1eb7b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c4796c6d-gwcnp"
Sep 09 20:56:35.230205 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:35.230116    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c4796c6d-gwcnp"
Sep 09 20:56:35.499702 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:35.499678    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c4796c6d-gwcnp"]
Sep 09 20:56:35.502502 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:56:35.502476    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29df0259_6442_4186_8cda_b3b39fd1eb7b.slice/crio-1630a040d9d70388be6118fa9cb1c18f4d1f43962ffa1258bda9b8f5069b984b WatchSource:0}: Error finding container 1630a040d9d70388be6118fa9cb1c18f4d1f43962ffa1258bda9b8f5069b984b: Status 404 returned error can't find the container with id 1630a040d9d70388be6118fa9cb1c18f4d1f43962ffa1258bda9b8f5069b984b
Sep 09 20:56:35.740045 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:35.740005    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c4796c6d-gwcnp" event={"ID":"29df0259-6442-4186-8cda-b3b39fd1eb7b","Type":"ContainerStarted","Data":"1630a040d9d70388be6118fa9cb1c18f4d1f43962ffa1258bda9b8f5069b984b"}
Sep 09 20:56:36.747021 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:36.746985    2581 generic.go:356] "Generic (PLEG): container finished" podID="29df0259-6442-4186-8cda-b3b39fd1eb7b" containerID="2908f3e5f8d3ca8e6a6ccab8c2ec1fe223da36abc4d27796c932b7bc8e3da531" exitCode=0
Sep 09 20:56:36.747433 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:36.747076    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c4796c6d-gwcnp" event={"ID":"29df0259-6442-4186-8cda-b3b39fd1eb7b","Type":"ContainerDied","Data":"2908f3e5f8d3ca8e6a6ccab8c2ec1fe223da36abc4d27796c932b7bc8e3da531"}
Sep 09 20:56:37.890922 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:37.890899    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c4796c6d-gwcnp"
Sep 09 20:56:37.951007 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:37.950977    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29df0259-6442-4186-8cda-b3b39fd1eb7b-maas-api-service-ca\") pod \"29df0259-6442-4186-8cda-b3b39fd1eb7b\" (UID: \"29df0259-6442-4186-8cda-b3b39fd1eb7b\") "
Sep 09 20:56:37.951292 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:37.951270    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29df0259-6442-4186-8cda-b3b39fd1eb7b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "29df0259-6442-4186-8cda-b3b39fd1eb7b" (UID: "29df0259-6442-4186-8cda-b3b39fd1eb7b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:56:37.951650 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:37.951626    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29df0259-6442-4186-8cda-b3b39fd1eb7b-maas-api-service-ca\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:56:38.758823 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:38.758799    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c4796c6d-gwcnp"
Sep 09 20:56:38.758978 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:38.758836    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c4796c6d-gwcnp" event={"ID":"29df0259-6442-4186-8cda-b3b39fd1eb7b","Type":"ContainerDied","Data":"1630a040d9d70388be6118fa9cb1c18f4d1f43962ffa1258bda9b8f5069b984b"}
Sep 09 20:56:38.758978 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:38.758880    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1630a040d9d70388be6118fa9cb1c18f4d1f43962ffa1258bda9b8f5069b984b"
Sep 09 20:56:46.743256 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:46.743225    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-c4796c6d-678cf978b9-8xgwn"]
Sep 09 20:56:46.743672 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:46.743475    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-c4796c6d-678cf978b9-8xgwn" podUID="d6e642c9-25d2-4914-bb02-42e3b60127fb" containerName="payload-processing" containerID="cri-o://aec5769411330af812285a7443ce0e248291c27d50ac706f7a1dcb3f23ee5f35" gracePeriod=30
Sep 09 20:56:46.751429 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:46.751406    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-c4796c6d-bb9799dbb-xm2m8"]
Sep 09 20:56:46.751688 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:46.751643    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-c4796c6d-bb9799dbb-xm2m8" podUID="7cfc9803-8743-45e0-b8ad-405c60f461f7" containerName="payload-pre-processing" containerID="cri-o://9e3e5390532af4f3db50cf87582ba8ae4b8598b5f6d741535f7aebd9016031ef" gracePeriod=30
Sep 09 20:56:46.967793 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:46.966692    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c4796c6d-gwcnp"]
Sep 09 20:56:46.974132 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:46.973846    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c4796c6d-gwcnp"]
Sep 09 20:56:47.063809 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:47.063789    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-c4796c6d-678cf978b9-8xgwn"
Sep 09 20:56:47.066825 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:47.066802    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-c4796c6d-bb9799dbb-xm2m8"
Sep 09 20:56:47.143484 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:47.143452    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5"]
Sep 09 20:56:47.143690 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:47.143669    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5" podUID="55236624-8426-4187-acfc-ac24200f14ca" containerName="istio-proxy" containerID="cri-o://11e2bd1a4f9c29382d70e46348bd32c28b32fe88cc7335f53cc11837873a06b4" gracePeriod=30
Sep 09 20:56:47.156229 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:47.156201    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d6e642c9-25d2-4914-bb02-42e3b60127fb-plugins-config-volume\") pod \"d6e642c9-25d2-4914-bb02-42e3b60127fb\" (UID: \"d6e642c9-25d2-4914-bb02-42e3b60127fb\") "
Sep 09 20:56:47.156364 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:47.156289    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cvkvv\" (UniqueName: \"kubernetes.io/projected/7cfc9803-8743-45e0-b8ad-405c60f461f7-kube-api-access-cvkvv\") pod \"7cfc9803-8743-45e0-b8ad-405c60f461f7\" (UID: \"7cfc9803-8743-45e0-b8ad-405c60f461f7\") "
Sep 09 20:56:47.156419 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:47.156390    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7cfc9803-8743-45e0-b8ad-405c60f461f7-plugins-config-volume\") pod \"7cfc9803-8743-45e0-b8ad-405c60f461f7\" (UID: \"7cfc9803-8743-45e0-b8ad-405c60f461f7\") "
Sep 09 20:56:47.156468 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:47.156448    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xt48l\" (UniqueName: \"kubernetes.io/projected/d6e642c9-25d2-4914-bb02-42e3b60127fb-kube-api-access-xt48l\") pod \"d6e642c9-25d2-4914-bb02-42e3b60127fb\" (UID: \"d6e642c9-25d2-4914-bb02-42e3b60127fb\") "
Sep 09 20:56:47.156678 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:47.156649    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6e642c9-25d2-4914-bb02-42e3b60127fb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d6e642c9-25d2-4914-bb02-42e3b60127fb" (UID: "d6e642c9-25d2-4914-bb02-42e3b60127fb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:56:47.156842 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:47.156822    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7cfc9803-8743-45e0-b8ad-405c60f461f7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7cfc9803-8743-45e0-b8ad-405c60f461f7" (UID: "7cfc9803-8743-45e0-b8ad-405c60f461f7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:56:47.156842 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:47.156838    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d6e642c9-25d2-4914-bb02-42e3b60127fb-plugins-config-volume\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:56:47.158494 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:47.158469    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6e642c9-25d2-4914-bb02-42e3b60127fb-kube-api-access-xt48l" (OuterVolumeSpecName: "kube-api-access-xt48l") pod "d6e642c9-25d2-4914-bb02-42e3b60127fb" (UID: "d6e642c9-25d2-4914-bb02-42e3b60127fb"). InnerVolumeSpecName "kube-api-access-xt48l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:56:47.158834 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:47.158814    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7cfc9803-8743-45e0-b8ad-405c60f461f7-kube-api-access-cvkvv" (OuterVolumeSpecName: "kube-api-access-cvkvv") pod "7cfc9803-8743-45e0-b8ad-405c60f461f7" (UID: "7cfc9803-8743-45e0-b8ad-405c60f461f7"). InnerVolumeSpecName "kube-api-access-cvkvv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:56:47.259120 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:47.258818    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cvkvv\" (UniqueName: \"kubernetes.io/projected/7cfc9803-8743-45e0-b8ad-405c60f461f7-kube-api-access-cvkvv\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:56:47.259120 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:47.258857    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7cfc9803-8743-45e0-b8ad-405c60f461f7-plugins-config-volume\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:56:47.259120 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:47.258871    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xt48l\" (UniqueName: \"kubernetes.io/projected/d6e642c9-25d2-4914-bb02-42e3b60127fb-kube-api-access-xt48l\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:56:47.809918 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:47.809885    2581 generic.go:356] "Generic (PLEG): container finished" podID="d6e642c9-25d2-4914-bb02-42e3b60127fb" containerID="aec5769411330af812285a7443ce0e248291c27d50ac706f7a1dcb3f23ee5f35" exitCode=0
Sep 09 20:56:47.810342 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:47.809929    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-c4796c6d-678cf978b9-8xgwn"
Sep 09 20:56:47.810342 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:47.809966    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-c4796c6d-678cf978b9-8xgwn" event={"ID":"d6e642c9-25d2-4914-bb02-42e3b60127fb","Type":"ContainerDied","Data":"aec5769411330af812285a7443ce0e248291c27d50ac706f7a1dcb3f23ee5f35"}
Sep 09 20:56:47.810342 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:47.810016    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-c4796c6d-678cf978b9-8xgwn" event={"ID":"d6e642c9-25d2-4914-bb02-42e3b60127fb","Type":"ContainerDied","Data":"ace9769b5c0265f38c93c3762a72531a68aee935e7a0226a7391e9fb8978bd01"}
Sep 09 20:56:47.810342 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:47.810041    2581 scope.go:117] "RemoveContainer" containerID="aec5769411330af812285a7443ce0e248291c27d50ac706f7a1dcb3f23ee5f35"
Sep 09 20:56:47.811647 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:47.811616    2581 generic.go:356] "Generic (PLEG): container finished" podID="7cfc9803-8743-45e0-b8ad-405c60f461f7" containerID="9e3e5390532af4f3db50cf87582ba8ae4b8598b5f6d741535f7aebd9016031ef" exitCode=0
Sep 09 20:56:47.811764 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:47.811657    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-c4796c6d-bb9799dbb-xm2m8"
Sep 09 20:56:47.811764 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:47.811657    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-c4796c6d-bb9799dbb-xm2m8" event={"ID":"7cfc9803-8743-45e0-b8ad-405c60f461f7","Type":"ContainerDied","Data":"9e3e5390532af4f3db50cf87582ba8ae4b8598b5f6d741535f7aebd9016031ef"}
Sep 09 20:56:47.811875 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:47.811785    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-c4796c6d-bb9799dbb-xm2m8" event={"ID":"7cfc9803-8743-45e0-b8ad-405c60f461f7","Type":"ContainerDied","Data":"b416b3b4209808826cd78d8992466bb3ef51e90667ba1fe4a38afd697bdb6ccb"}
Sep 09 20:56:47.820912 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:47.820896    2581 scope.go:117] "RemoveContainer" containerID="aec5769411330af812285a7443ce0e248291c27d50ac706f7a1dcb3f23ee5f35"
Sep 09 20:56:47.821195 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:56:47.821175    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aec5769411330af812285a7443ce0e248291c27d50ac706f7a1dcb3f23ee5f35\": container with ID starting with aec5769411330af812285a7443ce0e248291c27d50ac706f7a1dcb3f23ee5f35 not found: ID does not exist" containerID="aec5769411330af812285a7443ce0e248291c27d50ac706f7a1dcb3f23ee5f35"
Sep 09 20:56:47.821290 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:47.821200    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aec5769411330af812285a7443ce0e248291c27d50ac706f7a1dcb3f23ee5f35"} err="failed to get container status \"aec5769411330af812285a7443ce0e248291c27d50ac706f7a1dcb3f23ee5f35\": rpc error: code = NotFound desc = could not find container \"aec5769411330af812285a7443ce0e248291c27d50ac706f7a1dcb3f23ee5f35\": container with ID starting with aec5769411330af812285a7443ce0e248291c27d50ac706f7a1dcb3f23ee5f35 not found: ID does not exist"
Sep 09 20:56:47.821290 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:47.821218    2581 scope.go:117] "RemoveContainer" containerID="9e3e5390532af4f3db50cf87582ba8ae4b8598b5f6d741535f7aebd9016031ef"
Sep 09 20:56:47.831389 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:47.831370    2581 scope.go:117] "RemoveContainer" containerID="9e3e5390532af4f3db50cf87582ba8ae4b8598b5f6d741535f7aebd9016031ef"
Sep 09 20:56:47.831716 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:56:47.831697    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e3e5390532af4f3db50cf87582ba8ae4b8598b5f6d741535f7aebd9016031ef\": container with ID starting with 9e3e5390532af4f3db50cf87582ba8ae4b8598b5f6d741535f7aebd9016031ef not found: ID does not exist" containerID="9e3e5390532af4f3db50cf87582ba8ae4b8598b5f6d741535f7aebd9016031ef"
Sep 09 20:56:47.831779 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:47.831725    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e3e5390532af4f3db50cf87582ba8ae4b8598b5f6d741535f7aebd9016031ef"} err="failed to get container status \"9e3e5390532af4f3db50cf87582ba8ae4b8598b5f6d741535f7aebd9016031ef\": rpc error: code = NotFound desc = could not find container \"9e3e5390532af4f3db50cf87582ba8ae4b8598b5f6d741535f7aebd9016031ef\": container with ID starting with 9e3e5390532af4f3db50cf87582ba8ae4b8598b5f6d741535f7aebd9016031ef not found: ID does not exist"
Sep 09 20:56:47.841534 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:47.841515    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-c4796c6d-678cf978b9-8xgwn"]
Sep 09 20:56:47.847787 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:47.847768    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-c4796c6d-678cf978b9-8xgwn"]
Sep 09 20:56:47.861150 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:47.861130    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-c4796c6d-bb9799dbb-xm2m8"]
Sep 09 20:56:47.868263 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:47.868242    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-c4796c6d-bb9799dbb-xm2m8"]
Sep 09 20:56:48.808457 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:48.808414    2581 patch_prober.go:28] interesting pod/e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 09 20:56:48.808629 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:48.808483    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5" podUID="55236624-8426-4187-acfc-ac24200f14ca" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 20:56:48.820426 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:48.820395    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29df0259-6442-4186-8cda-b3b39fd1eb7b" path="/var/lib/kubelet/pods/29df0259-6442-4186-8cda-b3b39fd1eb7b/volumes"
Sep 09 20:56:48.820751 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:48.820732    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7cfc9803-8743-45e0-b8ad-405c60f461f7" path="/var/lib/kubelet/pods/7cfc9803-8743-45e0-b8ad-405c60f461f7/volumes"
Sep 09 20:56:48.821045 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:48.821032    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6e642c9-25d2-4914-bb02-42e3b60127fb" path="/var/lib/kubelet/pods/d6e642c9-25d2-4914-bb02-42e3b60127fb/volumes"
Sep 09 20:56:52.393336 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:52.393311    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5"
Sep 09 20:56:52.411806 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:52.411740    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55236624-8426-4187-acfc-ac24200f14ca-istio-envoy\") pod \"55236624-8426-4187-acfc-ac24200f14ca\" (UID: \"55236624-8426-4187-acfc-ac24200f14ca\") "
Sep 09 20:56:52.411806 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:52.411781    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55236624-8426-4187-acfc-ac24200f14ca-credential-socket\") pod \"55236624-8426-4187-acfc-ac24200f14ca\" (UID: \"55236624-8426-4187-acfc-ac24200f14ca\") "
Sep 09 20:56:52.412003 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:52.411809    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55236624-8426-4187-acfc-ac24200f14ca-istio-data\") pod \"55236624-8426-4187-acfc-ac24200f14ca\" (UID: \"55236624-8426-4187-acfc-ac24200f14ca\") "
Sep 09 20:56:52.412003 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:52.411862    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55236624-8426-4187-acfc-ac24200f14ca-istio-token\") pod \"55236624-8426-4187-acfc-ac24200f14ca\" (UID: \"55236624-8426-4187-acfc-ac24200f14ca\") "
Sep 09 20:56:52.412003 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:52.411897    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55236624-8426-4187-acfc-ac24200f14ca-workload-certs\") pod \"55236624-8426-4187-acfc-ac24200f14ca\" (UID: \"55236624-8426-4187-acfc-ac24200f14ca\") "
Sep 09 20:56:52.412003 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:52.411926    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55236624-8426-4187-acfc-ac24200f14ca-istiod-ca-cert\") pod \"55236624-8426-4187-acfc-ac24200f14ca\" (UID: \"55236624-8426-4187-acfc-ac24200f14ca\") "
Sep 09 20:56:52.412003 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:52.411960    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55236624-8426-4187-acfc-ac24200f14ca-istio-podinfo\") pod \"55236624-8426-4187-acfc-ac24200f14ca\" (UID: \"55236624-8426-4187-acfc-ac24200f14ca\") "
Sep 09 20:56:52.412284 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:52.412016    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9v4p2\" (UniqueName: \"kubernetes.io/projected/55236624-8426-4187-acfc-ac24200f14ca-kube-api-access-9v4p2\") pod \"55236624-8426-4187-acfc-ac24200f14ca\" (UID: \"55236624-8426-4187-acfc-ac24200f14ca\") "
Sep 09 20:56:52.412284 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:52.412049    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55236624-8426-4187-acfc-ac24200f14ca-workload-socket\") pod \"55236624-8426-4187-acfc-ac24200f14ca\" (UID: \"55236624-8426-4187-acfc-ac24200f14ca\") "
Sep 09 20:56:52.412284 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:52.412158    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55236624-8426-4187-acfc-ac24200f14ca-istio-data" (OuterVolumeSpecName: "istio-data") pod "55236624-8426-4187-acfc-ac24200f14ca" (UID: "55236624-8426-4187-acfc-ac24200f14ca"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:56:52.412284 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:52.412201    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55236624-8426-4187-acfc-ac24200f14ca-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "55236624-8426-4187-acfc-ac24200f14ca" (UID: "55236624-8426-4187-acfc-ac24200f14ca"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:56:52.412284 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:52.412163    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55236624-8426-4187-acfc-ac24200f14ca-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "55236624-8426-4187-acfc-ac24200f14ca" (UID: "55236624-8426-4187-acfc-ac24200f14ca"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:56:52.412538 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:52.412399    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55236624-8426-4187-acfc-ac24200f14ca-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "55236624-8426-4187-acfc-ac24200f14ca" (UID: "55236624-8426-4187-acfc-ac24200f14ca"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:56:52.412596 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:52.412532    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55236624-8426-4187-acfc-ac24200f14ca-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "55236624-8426-4187-acfc-ac24200f14ca" (UID: "55236624-8426-4187-acfc-ac24200f14ca"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:56:52.412596 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:52.412539    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55236624-8426-4187-acfc-ac24200f14ca-credential-socket\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:56:52.412596 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:52.412584    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55236624-8426-4187-acfc-ac24200f14ca-istio-data\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:56:52.412753 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:52.412602    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55236624-8426-4187-acfc-ac24200f14ca-workload-certs\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:56:52.412753 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:52.412616    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55236624-8426-4187-acfc-ac24200f14ca-workload-socket\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:56:52.414263 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:52.414233    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55236624-8426-4187-acfc-ac24200f14ca-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "55236624-8426-4187-acfc-ac24200f14ca" (UID: "55236624-8426-4187-acfc-ac24200f14ca"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:56:52.414427 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:52.414285    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55236624-8426-4187-acfc-ac24200f14ca-kube-api-access-9v4p2" (OuterVolumeSpecName: "kube-api-access-9v4p2") pod "55236624-8426-4187-acfc-ac24200f14ca" (UID: "55236624-8426-4187-acfc-ac24200f14ca"). InnerVolumeSpecName "kube-api-access-9v4p2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:56:52.414535 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:52.414513    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/55236624-8426-4187-acfc-ac24200f14ca-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "55236624-8426-4187-acfc-ac24200f14ca" (UID: "55236624-8426-4187-acfc-ac24200f14ca"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 20:56:52.414634 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:52.414562    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55236624-8426-4187-acfc-ac24200f14ca-istio-token" (OuterVolumeSpecName: "istio-token") pod "55236624-8426-4187-acfc-ac24200f14ca" (UID: "55236624-8426-4187-acfc-ac24200f14ca"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:56:52.513413 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:52.513384    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55236624-8426-4187-acfc-ac24200f14ca-istio-token\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:56:52.513413 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:52.513407    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55236624-8426-4187-acfc-ac24200f14ca-istiod-ca-cert\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:56:52.513413 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:52.513416    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55236624-8426-4187-acfc-ac24200f14ca-istio-podinfo\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:56:52.513613 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:52.513425    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9v4p2\" (UniqueName: \"kubernetes.io/projected/55236624-8426-4187-acfc-ac24200f14ca-kube-api-access-9v4p2\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:56:52.513613 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:52.513433    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55236624-8426-4187-acfc-ac24200f14ca-istio-envoy\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:56:52.847930 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:52.847898    2581 generic.go:356] "Generic (PLEG): container finished" podID="55236624-8426-4187-acfc-ac24200f14ca" containerID="11e2bd1a4f9c29382d70e46348bd32c28b32fe88cc7335f53cc11837873a06b4" exitCode=0
Sep 09 20:56:52.848076 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:52.847932    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5"
Sep 09 20:56:52.848076 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:52.847973    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5" event={"ID":"55236624-8426-4187-acfc-ac24200f14ca","Type":"ContainerDied","Data":"11e2bd1a4f9c29382d70e46348bd32c28b32fe88cc7335f53cc11837873a06b4"}
Sep 09 20:56:52.848076 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:52.848014    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5" event={"ID":"55236624-8426-4187-acfc-ac24200f14ca","Type":"ContainerDied","Data":"80c1903192ba22a448714f6b16e6326358259963448dfc3aad52741693d219c2"}
Sep 09 20:56:52.848076 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:52.848042    2581 scope.go:117] "RemoveContainer" containerID="11e2bd1a4f9c29382d70e46348bd32c28b32fe88cc7335f53cc11837873a06b4"
Sep 09 20:56:52.857077 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:52.857054    2581 scope.go:117] "RemoveContainer" containerID="71ebfcbb62d36726a3927f0bdb5937ace71d84edd1920927eba6a8af8ab0f971"
Sep 09 20:56:52.865160 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:52.865141    2581 scope.go:117] "RemoveContainer" containerID="11e2bd1a4f9c29382d70e46348bd32c28b32fe88cc7335f53cc11837873a06b4"
Sep 09 20:56:52.865404 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:56:52.865386    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11e2bd1a4f9c29382d70e46348bd32c28b32fe88cc7335f53cc11837873a06b4\": container with ID starting with 11e2bd1a4f9c29382d70e46348bd32c28b32fe88cc7335f53cc11837873a06b4 not found: ID does not exist" containerID="11e2bd1a4f9c29382d70e46348bd32c28b32fe88cc7335f53cc11837873a06b4"
Sep 09 20:56:52.865468 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:52.865413    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11e2bd1a4f9c29382d70e46348bd32c28b32fe88cc7335f53cc11837873a06b4"} err="failed to get container status \"11e2bd1a4f9c29382d70e46348bd32c28b32fe88cc7335f53cc11837873a06b4\": rpc error: code = NotFound desc = could not find container \"11e2bd1a4f9c29382d70e46348bd32c28b32fe88cc7335f53cc11837873a06b4\": container with ID starting with 11e2bd1a4f9c29382d70e46348bd32c28b32fe88cc7335f53cc11837873a06b4 not found: ID does not exist"
Sep 09 20:56:52.865468 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:52.865435    2581 scope.go:117] "RemoveContainer" containerID="71ebfcbb62d36726a3927f0bdb5937ace71d84edd1920927eba6a8af8ab0f971"
Sep 09 20:56:52.865697 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:56:52.865683    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71ebfcbb62d36726a3927f0bdb5937ace71d84edd1920927eba6a8af8ab0f971\": container with ID starting with 71ebfcbb62d36726a3927f0bdb5937ace71d84edd1920927eba6a8af8ab0f971 not found: ID does not exist" containerID="71ebfcbb62d36726a3927f0bdb5937ace71d84edd1920927eba6a8af8ab0f971"
Sep 09 20:56:52.865768 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:52.865699    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71ebfcbb62d36726a3927f0bdb5937ace71d84edd1920927eba6a8af8ab0f971"} err="failed to get container status \"71ebfcbb62d36726a3927f0bdb5937ace71d84edd1920927eba6a8af8ab0f971\": rpc error: code = NotFound desc = could not find container \"71ebfcbb62d36726a3927f0bdb5937ace71d84edd1920927eba6a8af8ab0f971\": container with ID starting with 71ebfcbb62d36726a3927f0bdb5937ace71d84edd1920927eba6a8af8ab0f971 not found: ID does not exist"
Sep 09 20:56:52.871352 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:52.871331    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5"]
Sep 09 20:56:52.876928 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:52.876905    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-c4796c6d-openshift-default-6d6dfd5c56-9zss5"]
Sep 09 20:56:54.820507 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:54.820472    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55236624-8426-4187-acfc-ac24200f14ca" path="/var/lib/kubelet/pods/55236624-8426-4187-acfc-ac24200f14ca/volumes"
Sep 09 20:56:57.073978 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:57.073947    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-47e41c/premium-w4-47e41c-kserve-6b47f4769b-lrvrt"]
Sep 09 20:56:57.074402 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:57.074387    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55236624-8426-4187-acfc-ac24200f14ca" containerName="istio-proxy"
Sep 09 20:56:57.074460 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:57.074403    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="55236624-8426-4187-acfc-ac24200f14ca" containerName="istio-proxy"
Sep 09 20:56:57.074460 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:57.074412    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55236624-8426-4187-acfc-ac24200f14ca" containerName="istio-proxy"
Sep 09 20:56:57.074460 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:57.074418    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="55236624-8426-4187-acfc-ac24200f14ca" containerName="istio-proxy"
Sep 09 20:56:57.074460 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:57.074428    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7cfc9803-8743-45e0-b8ad-405c60f461f7" containerName="payload-pre-processing"
Sep 09 20:56:57.074460 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:57.074434    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="7cfc9803-8743-45e0-b8ad-405c60f461f7" containerName="payload-pre-processing"
Sep 09 20:56:57.074460 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:57.074449    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6e642c9-25d2-4914-bb02-42e3b60127fb" containerName="payload-processing"
Sep 09 20:56:57.074460 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:57.074455    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6e642c9-25d2-4914-bb02-42e3b60127fb" containerName="payload-processing"
Sep 09 20:56:57.074668 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:57.074466    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29df0259-6442-4186-8cda-b3b39fd1eb7b" containerName="revoke-keys"
Sep 09 20:56:57.074668 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:57.074471    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="29df0259-6442-4186-8cda-b3b39fd1eb7b" containerName="revoke-keys"
Sep 09 20:56:57.074668 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:57.074523    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="29df0259-6442-4186-8cda-b3b39fd1eb7b" containerName="revoke-keys"
Sep 09 20:56:57.074668 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:57.074532    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="55236624-8426-4187-acfc-ac24200f14ca" containerName="istio-proxy"
Sep 09 20:56:57.074668 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:57.074538    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="7cfc9803-8743-45e0-b8ad-405c60f461f7" containerName="payload-pre-processing"
Sep 09 20:56:57.074668 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:57.074543    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="55236624-8426-4187-acfc-ac24200f14ca" containerName="istio-proxy"
Sep 09 20:56:57.074668 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:57.074550    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6e642c9-25d2-4914-bb02-42e3b60127fb" containerName="payload-processing"
Sep 09 20:56:57.078778 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:57.078761    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-47e41c/premium-w4-47e41c-kserve-6b47f4769b-lrvrt"
Sep 09 20:56:57.086090 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:57.086068    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-47e41c\"/\"openshift-service-ca.crt\""
Sep 09 20:56:57.086504 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:57.086487    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-47e41c\"/\"default-dockercfg-cbzth\""
Sep 09 20:56:57.088875 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:57.088856    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-47e41c\"/\"kube-root-ca.crt\""
Sep 09 20:56:57.091885 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:57.091869    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-47e41c\"/\"premium-w4-47e41c-kserve-self-signed-certs\""
Sep 09 20:56:57.103701 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:57.103669    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-47e41c/premium-w4-47e41c-kserve-6b47f4769b-lrvrt"]
Sep 09 20:56:57.156761 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:57.156729    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6a2c1169-5aab-447b-90ff-1fbc19876d31-home\") pod \"premium-w4-47e41c-kserve-6b47f4769b-lrvrt\" (UID: \"6a2c1169-5aab-447b-90ff-1fbc19876d31\") " pod="e2e-models-e2e-worker-w4-47e41c/premium-w4-47e41c-kserve-6b47f4769b-lrvrt"
Sep 09 20:56:57.156934 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:57.156777    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hqc2\" (UniqueName: \"kubernetes.io/projected/6a2c1169-5aab-447b-90ff-1fbc19876d31-kube-api-access-7hqc2\") pod \"premium-w4-47e41c-kserve-6b47f4769b-lrvrt\" (UID: \"6a2c1169-5aab-447b-90ff-1fbc19876d31\") " pod="e2e-models-e2e-worker-w4-47e41c/premium-w4-47e41c-kserve-6b47f4769b-lrvrt"
Sep 09 20:56:57.156934 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:57.156844    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a2c1169-5aab-447b-90ff-1fbc19876d31-tls-certs\") pod \"premium-w4-47e41c-kserve-6b47f4769b-lrvrt\" (UID: \"6a2c1169-5aab-447b-90ff-1fbc19876d31\") " pod="e2e-models-e2e-worker-w4-47e41c/premium-w4-47e41c-kserve-6b47f4769b-lrvrt"
Sep 09 20:56:57.156934 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:57.156925    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6a2c1169-5aab-447b-90ff-1fbc19876d31-model-cache\") pod \"premium-w4-47e41c-kserve-6b47f4769b-lrvrt\" (UID: \"6a2c1169-5aab-447b-90ff-1fbc19876d31\") " pod="e2e-models-e2e-worker-w4-47e41c/premium-w4-47e41c-kserve-6b47f4769b-lrvrt"
Sep 09 20:56:57.157039 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:57.156954    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a2c1169-5aab-447b-90ff-1fbc19876d31-tmp-dir\") pod \"premium-w4-47e41c-kserve-6b47f4769b-lrvrt\" (UID: \"6a2c1169-5aab-447b-90ff-1fbc19876d31\") " pod="e2e-models-e2e-worker-w4-47e41c/premium-w4-47e41c-kserve-6b47f4769b-lrvrt"
Sep 09 20:56:57.157039 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:57.156981    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6a2c1169-5aab-447b-90ff-1fbc19876d31-dshm\") pod \"premium-w4-47e41c-kserve-6b47f4769b-lrvrt\" (UID: \"6a2c1169-5aab-447b-90ff-1fbc19876d31\") " pod="e2e-models-e2e-worker-w4-47e41c/premium-w4-47e41c-kserve-6b47f4769b-lrvrt"
Sep 09 20:56:57.257547 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:57.257515    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6a2c1169-5aab-447b-90ff-1fbc19876d31-dshm\") pod \"premium-w4-47e41c-kserve-6b47f4769b-lrvrt\" (UID: \"6a2c1169-5aab-447b-90ff-1fbc19876d31\") " pod="e2e-models-e2e-worker-w4-47e41c/premium-w4-47e41c-kserve-6b47f4769b-lrvrt"
Sep 09 20:56:57.257712 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:57.257597    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6a2c1169-5aab-447b-90ff-1fbc19876d31-home\") pod \"premium-w4-47e41c-kserve-6b47f4769b-lrvrt\" (UID: \"6a2c1169-5aab-447b-90ff-1fbc19876d31\") " pod="e2e-models-e2e-worker-w4-47e41c/premium-w4-47e41c-kserve-6b47f4769b-lrvrt"
Sep 09 20:56:57.257712 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:57.257640    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hqc2\" (UniqueName: \"kubernetes.io/projected/6a2c1169-5aab-447b-90ff-1fbc19876d31-kube-api-access-7hqc2\") pod \"premium-w4-47e41c-kserve-6b47f4769b-lrvrt\" (UID: \"6a2c1169-5aab-447b-90ff-1fbc19876d31\") " pod="e2e-models-e2e-worker-w4-47e41c/premium-w4-47e41c-kserve-6b47f4769b-lrvrt"
Sep 09 20:56:57.257712 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:57.257675    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a2c1169-5aab-447b-90ff-1fbc19876d31-tls-certs\") pod \"premium-w4-47e41c-kserve-6b47f4769b-lrvrt\" (UID: \"6a2c1169-5aab-447b-90ff-1fbc19876d31\") " pod="e2e-models-e2e-worker-w4-47e41c/premium-w4-47e41c-kserve-6b47f4769b-lrvrt"
Sep 09 20:56:57.257850 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:57.257724    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6a2c1169-5aab-447b-90ff-1fbc19876d31-model-cache\") pod \"premium-w4-47e41c-kserve-6b47f4769b-lrvrt\" (UID: \"6a2c1169-5aab-447b-90ff-1fbc19876d31\") " pod="e2e-models-e2e-worker-w4-47e41c/premium-w4-47e41c-kserve-6b47f4769b-lrvrt"
Sep 09 20:56:57.257901 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:57.257860    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a2c1169-5aab-447b-90ff-1fbc19876d31-tmp-dir\") pod \"premium-w4-47e41c-kserve-6b47f4769b-lrvrt\" (UID: \"6a2c1169-5aab-447b-90ff-1fbc19876d31\") " pod="e2e-models-e2e-worker-w4-47e41c/premium-w4-47e41c-kserve-6b47f4769b-lrvrt"
Sep 09 20:56:57.258038 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:57.258013    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6a2c1169-5aab-447b-90ff-1fbc19876d31-home\") pod \"premium-w4-47e41c-kserve-6b47f4769b-lrvrt\" (UID: \"6a2c1169-5aab-447b-90ff-1fbc19876d31\") " pod="e2e-models-e2e-worker-w4-47e41c/premium-w4-47e41c-kserve-6b47f4769b-lrvrt"
Sep 09 20:56:57.258204 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:57.258071    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6a2c1169-5aab-447b-90ff-1fbc19876d31-model-cache\") pod \"premium-w4-47e41c-kserve-6b47f4769b-lrvrt\" (UID: \"6a2c1169-5aab-447b-90ff-1fbc19876d31\") " pod="e2e-models-e2e-worker-w4-47e41c/premium-w4-47e41c-kserve-6b47f4769b-lrvrt"
Sep 09 20:56:57.258346 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:57.258326    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a2c1169-5aab-447b-90ff-1fbc19876d31-tmp-dir\") pod \"premium-w4-47e41c-kserve-6b47f4769b-lrvrt\" (UID: \"6a2c1169-5aab-447b-90ff-1fbc19876d31\") " pod="e2e-models-e2e-worker-w4-47e41c/premium-w4-47e41c-kserve-6b47f4769b-lrvrt"
Sep 09 20:56:57.259830 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:57.259809    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6a2c1169-5aab-447b-90ff-1fbc19876d31-dshm\") pod \"premium-w4-47e41c-kserve-6b47f4769b-lrvrt\" (UID: \"6a2c1169-5aab-447b-90ff-1fbc19876d31\") " pod="e2e-models-e2e-worker-w4-47e41c/premium-w4-47e41c-kserve-6b47f4769b-lrvrt"
Sep 09 20:56:57.260082 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:57.260064    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a2c1169-5aab-447b-90ff-1fbc19876d31-tls-certs\") pod \"premium-w4-47e41c-kserve-6b47f4769b-lrvrt\" (UID: \"6a2c1169-5aab-447b-90ff-1fbc19876d31\") " pod="e2e-models-e2e-worker-w4-47e41c/premium-w4-47e41c-kserve-6b47f4769b-lrvrt"
Sep 09 20:56:57.270038 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:57.270019    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hqc2\" (UniqueName: \"kubernetes.io/projected/6a2c1169-5aab-447b-90ff-1fbc19876d31-kube-api-access-7hqc2\") pod \"premium-w4-47e41c-kserve-6b47f4769b-lrvrt\" (UID: \"6a2c1169-5aab-447b-90ff-1fbc19876d31\") " pod="e2e-models-e2e-worker-w4-47e41c/premium-w4-47e41c-kserve-6b47f4769b-lrvrt"
Sep 09 20:56:57.396387 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:57.396304    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-47e41c/premium-w4-47e41c-kserve-6b47f4769b-lrvrt"
Sep 09 20:56:57.691045 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:57.691023    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-47e41c/premium-w4-47e41c-kserve-6b47f4769b-lrvrt"]
Sep 09 20:56:57.692688 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:56:57.692665    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6a2c1169_5aab_447b_90ff_1fbc19876d31.slice/crio-ffcc97bfa6b8fdf8d4332ce7c2e83b0e168ee7f8fa29f3bd5229b1bbf0c80c5c WatchSource:0}: Error finding container ffcc97bfa6b8fdf8d4332ce7c2e83b0e168ee7f8fa29f3bd5229b1bbf0c80c5c: Status 404 returned error can't find the container with id ffcc97bfa6b8fdf8d4332ce7c2e83b0e168ee7f8fa29f3bd5229b1bbf0c80c5c
Sep 09 20:56:57.878242 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:57.878207    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-47e41c/premium-w4-47e41c-kserve-6b47f4769b-lrvrt" event={"ID":"6a2c1169-5aab-447b-90ff-1fbc19876d31","Type":"ContainerStarted","Data":"3e2e5f33f279ef8a60d96a8ddfb159d17f0b0a0abc20889262fd84e2150efd54"}
Sep 09 20:56:57.878393 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:57.878249    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-47e41c/premium-w4-47e41c-kserve-6b47f4769b-lrvrt" event={"ID":"6a2c1169-5aab-447b-90ff-1fbc19876d31","Type":"ContainerStarted","Data":"ffcc97bfa6b8fdf8d4332ce7c2e83b0e168ee7f8fa29f3bd5229b1bbf0c80c5c"}
Sep 09 20:56:57.908890 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:57.908840    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-47e41c/premium-w4-47e41c-kserve-6b47f4769b-lrvrt" podStartSLOduration=0.90882691 podStartE2EDuration="908.82691ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:56:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:56:57.907287993 +0000 UTC m=+979.534915866" watchObservedRunningTime="2026-09-09 20:56:57.90882691 +0000 UTC m=+979.536454782"
Sep 09 20:56:59.030775 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:59.030743    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss"]
Sep 09 20:56:59.031193 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:59.030924    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss" podUID="92d54f63-73ce-4d1f-a0df-c0176265fb0f" containerName="istio-proxy" containerID="cri-o://ac347d12a46d5e9eaa809a19beafaf25bc19c0c03309650ab1ad5573efbfe027" gracePeriod=30
Sep 09 20:56:59.838700 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:59.838664    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4"]
Sep 09 20:56:59.842356 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:59.842336    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4"
Sep 09 20:56:59.847993 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:59.847967    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-explicit-2d4edf-openshift-default-dockercfg-c24dk\""
Sep 09 20:56:59.875006 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:59.874971    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4"]
Sep 09 20:56:59.886237 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:59.886206    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8184bd0-d46c-41e0-9e1d-11ae37508fcc-credential-socket\") pod \"e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4\" (UID: \"d8184bd0-d46c-41e0-9e1d-11ae37508fcc\") " pod="openshift-ingress/e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4"
Sep 09 20:56:59.886384 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:59.886277    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d8184bd0-d46c-41e0-9e1d-11ae37508fcc-istio-envoy\") pod \"e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4\" (UID: \"d8184bd0-d46c-41e0-9e1d-11ae37508fcc\") " pod="openshift-ingress/e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4"
Sep 09 20:56:59.886384 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:59.886307    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d8184bd0-d46c-41e0-9e1d-11ae37508fcc-istiod-ca-cert\") pod \"e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4\" (UID: \"d8184bd0-d46c-41e0-9e1d-11ae37508fcc\") " pod="openshift-ingress/e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4"
Sep 09 20:56:59.886503 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:59.886383    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8184bd0-d46c-41e0-9e1d-11ae37508fcc-workload-socket\") pod \"e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4\" (UID: \"d8184bd0-d46c-41e0-9e1d-11ae37508fcc\") " pod="openshift-ingress/e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4"
Sep 09 20:56:59.886503 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:59.886432    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d8184bd0-d46c-41e0-9e1d-11ae37508fcc-istio-data\") pod \"e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4\" (UID: \"d8184bd0-d46c-41e0-9e1d-11ae37508fcc\") " pod="openshift-ingress/e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4"
Sep 09 20:56:59.886634 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:59.886605    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d8184bd0-d46c-41e0-9e1d-11ae37508fcc-istio-podinfo\") pod \"e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4\" (UID: \"d8184bd0-d46c-41e0-9e1d-11ae37508fcc\") " pod="openshift-ingress/e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4"
Sep 09 20:56:59.886984 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:59.886958    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkfqr\" (UniqueName: \"kubernetes.io/projected/d8184bd0-d46c-41e0-9e1d-11ae37508fcc-kube-api-access-jkfqr\") pod \"e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4\" (UID: \"d8184bd0-d46c-41e0-9e1d-11ae37508fcc\") " pod="openshift-ingress/e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4"
Sep 09 20:56:59.887068 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:59.887043    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d8184bd0-d46c-41e0-9e1d-11ae37508fcc-workload-certs\") pod \"e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4\" (UID: \"d8184bd0-d46c-41e0-9e1d-11ae37508fcc\") " pod="openshift-ingress/e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4"
Sep 09 20:56:59.887387 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:59.887353    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d8184bd0-d46c-41e0-9e1d-11ae37508fcc-istio-token\") pod \"e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4\" (UID: \"d8184bd0-d46c-41e0-9e1d-11ae37508fcc\") " pod="openshift-ingress/e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4"
Sep 09 20:56:59.988555 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:59.988522    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkfqr\" (UniqueName: \"kubernetes.io/projected/d8184bd0-d46c-41e0-9e1d-11ae37508fcc-kube-api-access-jkfqr\") pod \"e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4\" (UID: \"d8184bd0-d46c-41e0-9e1d-11ae37508fcc\") " pod="openshift-ingress/e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4"
Sep 09 20:56:59.988725 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:59.988569    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d8184bd0-d46c-41e0-9e1d-11ae37508fcc-workload-certs\") pod \"e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4\" (UID: \"d8184bd0-d46c-41e0-9e1d-11ae37508fcc\") " pod="openshift-ingress/e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4"
Sep 09 20:56:59.988725 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:59.988629    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d8184bd0-d46c-41e0-9e1d-11ae37508fcc-istio-token\") pod \"e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4\" (UID: \"d8184bd0-d46c-41e0-9e1d-11ae37508fcc\") " pod="openshift-ingress/e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4"
Sep 09 20:56:59.988725 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:59.988681    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8184bd0-d46c-41e0-9e1d-11ae37508fcc-credential-socket\") pod \"e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4\" (UID: \"d8184bd0-d46c-41e0-9e1d-11ae37508fcc\") " pod="openshift-ingress/e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4"
Sep 09 20:56:59.988725 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:59.988716    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d8184bd0-d46c-41e0-9e1d-11ae37508fcc-istio-envoy\") pod \"e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4\" (UID: \"d8184bd0-d46c-41e0-9e1d-11ae37508fcc\") " pod="openshift-ingress/e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4"
Sep 09 20:56:59.988943 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:59.988742    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d8184bd0-d46c-41e0-9e1d-11ae37508fcc-istiod-ca-cert\") pod \"e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4\" (UID: \"d8184bd0-d46c-41e0-9e1d-11ae37508fcc\") " pod="openshift-ingress/e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4"
Sep 09 20:56:59.988943 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:59.988781    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8184bd0-d46c-41e0-9e1d-11ae37508fcc-workload-socket\") pod \"e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4\" (UID: \"d8184bd0-d46c-41e0-9e1d-11ae37508fcc\") " pod="openshift-ingress/e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4"
Sep 09 20:56:59.988943 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:59.988827    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d8184bd0-d46c-41e0-9e1d-11ae37508fcc-istio-data\") pod \"e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4\" (UID: \"d8184bd0-d46c-41e0-9e1d-11ae37508fcc\") " pod="openshift-ingress/e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4"
Sep 09 20:56:59.988943 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:59.988909    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d8184bd0-d46c-41e0-9e1d-11ae37508fcc-istio-podinfo\") pod \"e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4\" (UID: \"d8184bd0-d46c-41e0-9e1d-11ae37508fcc\") " pod="openshift-ingress/e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4"
Sep 09 20:56:59.989221 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:59.989013    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d8184bd0-d46c-41e0-9e1d-11ae37508fcc-workload-certs\") pod \"e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4\" (UID: \"d8184bd0-d46c-41e0-9e1d-11ae37508fcc\") " pod="openshift-ingress/e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4"
Sep 09 20:56:59.989221 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:59.989080    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8184bd0-d46c-41e0-9e1d-11ae37508fcc-credential-socket\") pod \"e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4\" (UID: \"d8184bd0-d46c-41e0-9e1d-11ae37508fcc\") " pod="openshift-ingress/e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4"
Sep 09 20:56:59.989334 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:59.989243    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8184bd0-d46c-41e0-9e1d-11ae37508fcc-workload-socket\") pod \"e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4\" (UID: \"d8184bd0-d46c-41e0-9e1d-11ae37508fcc\") " pod="openshift-ingress/e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4"
Sep 09 20:56:59.989334 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:59.989314    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d8184bd0-d46c-41e0-9e1d-11ae37508fcc-istio-data\") pod \"e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4\" (UID: \"d8184bd0-d46c-41e0-9e1d-11ae37508fcc\") " pod="openshift-ingress/e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4"
Sep 09 20:56:59.989470 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:59.989447    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d8184bd0-d46c-41e0-9e1d-11ae37508fcc-istiod-ca-cert\") pod \"e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4\" (UID: \"d8184bd0-d46c-41e0-9e1d-11ae37508fcc\") " pod="openshift-ingress/e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4"
Sep 09 20:56:59.991076 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:59.991052    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d8184bd0-d46c-41e0-9e1d-11ae37508fcc-istio-envoy\") pod \"e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4\" (UID: \"d8184bd0-d46c-41e0-9e1d-11ae37508fcc\") " pod="openshift-ingress/e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4"
Sep 09 20:56:59.991393 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:56:59.991373    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d8184bd0-d46c-41e0-9e1d-11ae37508fcc-istio-podinfo\") pod \"e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4\" (UID: \"d8184bd0-d46c-41e0-9e1d-11ae37508fcc\") " pod="openshift-ingress/e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4"
Sep 09 20:57:00.001633 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:00.001610    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkfqr\" (UniqueName: \"kubernetes.io/projected/d8184bd0-d46c-41e0-9e1d-11ae37508fcc-kube-api-access-jkfqr\") pod \"e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4\" (UID: \"d8184bd0-d46c-41e0-9e1d-11ae37508fcc\") " pod="openshift-ingress/e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4"
Sep 09 20:57:00.024962 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:00.024939    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d8184bd0-d46c-41e0-9e1d-11ae37508fcc-istio-token\") pod \"e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4\" (UID: \"d8184bd0-d46c-41e0-9e1d-11ae37508fcc\") " pod="openshift-ingress/e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4"
Sep 09 20:57:00.152888 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:00.152802    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4"
Sep 09 20:57:00.437645 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:00.437622    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4"]
Sep 09 20:57:00.440092 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:57:00.440065    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8184bd0_d46c_41e0_9e1d_11ae37508fcc.slice/crio-83cce5558d40c908c04127122993c28f4f31a7d0883118f00891d6d3ade383b7 WatchSource:0}: Error finding container 83cce5558d40c908c04127122993c28f4f31a7d0883118f00891d6d3ade383b7: Status 404 returned error can't find the container with id 83cce5558d40c908c04127122993c28f4f31a7d0883118f00891d6d3ade383b7
Sep 09 20:57:00.442364 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:00.442330    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:57:00.442490 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:00.442395    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:57:00.442490 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:00.442427    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:57:00.898862 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:00.898828    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4" event={"ID":"d8184bd0-d46c-41e0-9e1d-11ae37508fcc","Type":"ContainerStarted","Data":"e88f4843a1aaf0e61ac2131e4fcaf8d0df367b7ebaca415297c47280ccfb0e07"}
Sep 09 20:57:00.899044 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:00.898878    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4" event={"ID":"d8184bd0-d46c-41e0-9e1d-11ae37508fcc","Type":"ContainerStarted","Data":"83cce5558d40c908c04127122993c28f4f31a7d0883118f00891d6d3ade383b7"}
Sep 09 20:57:00.929694 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:00.929599    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4" podStartSLOduration=1.929581208 podStartE2EDuration="1.929581208s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:56:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:57:00.927780753 +0000 UTC m=+982.555408626" watchObservedRunningTime="2026-09-09 20:57:00.929581208 +0000 UTC m=+982.557209080"
Sep 09 20:57:01.152997 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:01.152896    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4"
Sep 09 20:57:01.158839 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:01.158811    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4"
Sep 09 20:57:01.905561 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:01.905529    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4"
Sep 09 20:57:01.906492 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:01.906470    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4"
Sep 09 20:57:03.097227 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:03.097197    2581 patch_prober.go:28] interesting pod/e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 09 20:57:03.097601 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:03.097250    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss" podUID="92d54f63-73ce-4d1f-a0df-c0176265fb0f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 20:57:04.284146 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:04.284118    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss"
Sep 09 20:57:04.334979 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:04.334890    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92d54f63-73ce-4d1f-a0df-c0176265fb0f-istio-token\") pod \"92d54f63-73ce-4d1f-a0df-c0176265fb0f\" (UID: \"92d54f63-73ce-4d1f-a0df-c0176265fb0f\") "
Sep 09 20:57:04.334979 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:04.334955    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-26vmz\" (UniqueName: \"kubernetes.io/projected/92d54f63-73ce-4d1f-a0df-c0176265fb0f-kube-api-access-26vmz\") pod \"92d54f63-73ce-4d1f-a0df-c0176265fb0f\" (UID: \"92d54f63-73ce-4d1f-a0df-c0176265fb0f\") "
Sep 09 20:57:04.335245 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:04.335018    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92d54f63-73ce-4d1f-a0df-c0176265fb0f-istio-podinfo\") pod \"92d54f63-73ce-4d1f-a0df-c0176265fb0f\" (UID: \"92d54f63-73ce-4d1f-a0df-c0176265fb0f\") "
Sep 09 20:57:04.335245 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:04.335053    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92d54f63-73ce-4d1f-a0df-c0176265fb0f-istio-envoy\") pod \"92d54f63-73ce-4d1f-a0df-c0176265fb0f\" (UID: \"92d54f63-73ce-4d1f-a0df-c0176265fb0f\") "
Sep 09 20:57:04.336384 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:04.335079    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92d54f63-73ce-4d1f-a0df-c0176265fb0f-credential-socket\") pod \"92d54f63-73ce-4d1f-a0df-c0176265fb0f\" (UID: \"92d54f63-73ce-4d1f-a0df-c0176265fb0f\") "
Sep 09 20:57:04.336384 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:04.335482    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92d54f63-73ce-4d1f-a0df-c0176265fb0f-workload-certs\") pod \"92d54f63-73ce-4d1f-a0df-c0176265fb0f\" (UID: \"92d54f63-73ce-4d1f-a0df-c0176265fb0f\") "
Sep 09 20:57:04.336384 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:04.335526    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92d54f63-73ce-4d1f-a0df-c0176265fb0f-workload-socket\") pod \"92d54f63-73ce-4d1f-a0df-c0176265fb0f\" (UID: \"92d54f63-73ce-4d1f-a0df-c0176265fb0f\") "
Sep 09 20:57:04.336384 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:04.335581    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92d54f63-73ce-4d1f-a0df-c0176265fb0f-istio-data\") pod \"92d54f63-73ce-4d1f-a0df-c0176265fb0f\" (UID: \"92d54f63-73ce-4d1f-a0df-c0176265fb0f\") "
Sep 09 20:57:04.336384 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:04.335630    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92d54f63-73ce-4d1f-a0df-c0176265fb0f-istiod-ca-cert\") pod \"92d54f63-73ce-4d1f-a0df-c0176265fb0f\" (UID: \"92d54f63-73ce-4d1f-a0df-c0176265fb0f\") "
Sep 09 20:57:04.336384 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:04.335843    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92d54f63-73ce-4d1f-a0df-c0176265fb0f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "92d54f63-73ce-4d1f-a0df-c0176265fb0f" (UID: "92d54f63-73ce-4d1f-a0df-c0176265fb0f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:57:04.336384 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:04.336065    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92d54f63-73ce-4d1f-a0df-c0176265fb0f-istio-data" (OuterVolumeSpecName: "istio-data") pod "92d54f63-73ce-4d1f-a0df-c0176265fb0f" (UID: "92d54f63-73ce-4d1f-a0df-c0176265fb0f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:57:04.336384 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:04.336087    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92d54f63-73ce-4d1f-a0df-c0176265fb0f-credential-socket\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:57:04.336384 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:04.336086    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92d54f63-73ce-4d1f-a0df-c0176265fb0f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "92d54f63-73ce-4d1f-a0df-c0176265fb0f" (UID: "92d54f63-73ce-4d1f-a0df-c0176265fb0f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:57:04.336384 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:04.336165    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92d54f63-73ce-4d1f-a0df-c0176265fb0f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "92d54f63-73ce-4d1f-a0df-c0176265fb0f" (UID: "92d54f63-73ce-4d1f-a0df-c0176265fb0f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:57:04.336384 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:04.336367    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92d54f63-73ce-4d1f-a0df-c0176265fb0f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "92d54f63-73ce-4d1f-a0df-c0176265fb0f" (UID: "92d54f63-73ce-4d1f-a0df-c0176265fb0f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:57:04.337798 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:04.337772    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/92d54f63-73ce-4d1f-a0df-c0176265fb0f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "92d54f63-73ce-4d1f-a0df-c0176265fb0f" (UID: "92d54f63-73ce-4d1f-a0df-c0176265fb0f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 20:57:04.337914 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:04.337801    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92d54f63-73ce-4d1f-a0df-c0176265fb0f-kube-api-access-26vmz" (OuterVolumeSpecName: "kube-api-access-26vmz") pod "92d54f63-73ce-4d1f-a0df-c0176265fb0f" (UID: "92d54f63-73ce-4d1f-a0df-c0176265fb0f"). InnerVolumeSpecName "kube-api-access-26vmz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:57:04.338018 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:04.338000    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92d54f63-73ce-4d1f-a0df-c0176265fb0f-istio-token" (OuterVolumeSpecName: "istio-token") pod "92d54f63-73ce-4d1f-a0df-c0176265fb0f" (UID: "92d54f63-73ce-4d1f-a0df-c0176265fb0f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:57:04.338337 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:04.338312    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92d54f63-73ce-4d1f-a0df-c0176265fb0f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "92d54f63-73ce-4d1f-a0df-c0176265fb0f" (UID: "92d54f63-73ce-4d1f-a0df-c0176265fb0f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:57:04.437012 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:04.436983    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92d54f63-73ce-4d1f-a0df-c0176265fb0f-istio-token\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:57:04.437012 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:04.437005    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-26vmz\" (UniqueName: \"kubernetes.io/projected/92d54f63-73ce-4d1f-a0df-c0176265fb0f-kube-api-access-26vmz\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:57:04.437012 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:04.437018    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92d54f63-73ce-4d1f-a0df-c0176265fb0f-istio-podinfo\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:57:04.437247 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:04.437026    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92d54f63-73ce-4d1f-a0df-c0176265fb0f-istio-envoy\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:57:04.437247 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:04.437034    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92d54f63-73ce-4d1f-a0df-c0176265fb0f-workload-certs\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:57:04.437247 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:04.437041    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92d54f63-73ce-4d1f-a0df-c0176265fb0f-workload-socket\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:57:04.437247 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:04.437048    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92d54f63-73ce-4d1f-a0df-c0176265fb0f-istio-data\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:57:04.437247 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:04.437058    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92d54f63-73ce-4d1f-a0df-c0176265fb0f-istiod-ca-cert\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:57:04.909231 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:04.909192    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-2d4edf-756bd96bbb-29lsh"]
Sep 09 20:57:04.909616 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:04.909601    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92d54f63-73ce-4d1f-a0df-c0176265fb0f" containerName="istio-proxy"
Sep 09 20:57:04.909616 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:04.909616    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="92d54f63-73ce-4d1f-a0df-c0176265fb0f" containerName="istio-proxy"
Sep 09 20:57:04.909745 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:04.909624    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92d54f63-73ce-4d1f-a0df-c0176265fb0f" containerName="istio-proxy"
Sep 09 20:57:04.909745 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:04.909629    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="92d54f63-73ce-4d1f-a0df-c0176265fb0f" containerName="istio-proxy"
Sep 09 20:57:04.909745 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:04.909709    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="92d54f63-73ce-4d1f-a0df-c0176265fb0f" containerName="istio-proxy"
Sep 09 20:57:04.909745 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:04.909717    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="92d54f63-73ce-4d1f-a0df-c0176265fb0f" containerName="istio-proxy"
Sep 09 20:57:04.912927 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:04.912905    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-2d4edf-595d4b596f-swj8r"]
Sep 09 20:57:04.913056 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:04.913034    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-2d4edf-756bd96bbb-29lsh"
Sep 09 20:57:04.916195 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:04.916177    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-2d4edf-595d4b596f-swj8r"
Sep 09 20:57:04.917718 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:04.917700    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-2d4edf\""
Sep 09 20:57:04.923888 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:04.923862    2581 generic.go:356] "Generic (PLEG): container finished" podID="92d54f63-73ce-4d1f-a0df-c0176265fb0f" containerID="ac347d12a46d5e9eaa809a19beafaf25bc19c0c03309650ab1ad5573efbfe027" exitCode=0
Sep 09 20:57:04.923990 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:04.923902    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss"
Sep 09 20:57:04.923990 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:04.923934    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss" event={"ID":"92d54f63-73ce-4d1f-a0df-c0176265fb0f","Type":"ContainerDied","Data":"ac347d12a46d5e9eaa809a19beafaf25bc19c0c03309650ab1ad5573efbfe027"}
Sep 09 20:57:04.923990 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:04.923976    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss" event={"ID":"92d54f63-73ce-4d1f-a0df-c0176265fb0f","Type":"ContainerDied","Data":"5c0e1d20867b5c3cb14a9c3523df2eb52b487ff7f149afb35b792a27c5331acf"}
Sep 09 20:57:04.924156 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:04.923997    2581 scope.go:117] "RemoveContainer" containerID="ac347d12a46d5e9eaa809a19beafaf25bc19c0c03309650ab1ad5573efbfe027"
Sep 09 20:57:04.930299 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:04.930249    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-2d4edf-756bd96bbb-29lsh"]
Sep 09 20:57:04.935889 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:04.935866    2581 scope.go:117] "RemoveContainer" containerID="161ac86d91fcab7e0c69dbe55bbfc7fe88717654d867b3e6c97699cd72955648"
Sep 09 20:57:04.937481 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:04.937139    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-2d4edf-595d4b596f-swj8r"]
Sep 09 20:57:04.951481 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:04.951460    2581 scope.go:117] "RemoveContainer" containerID="ac347d12a46d5e9eaa809a19beafaf25bc19c0c03309650ab1ad5573efbfe027"
Sep 09 20:57:04.951777 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:57:04.951757    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac347d12a46d5e9eaa809a19beafaf25bc19c0c03309650ab1ad5573efbfe027\": container with ID starting with ac347d12a46d5e9eaa809a19beafaf25bc19c0c03309650ab1ad5573efbfe027 not found: ID does not exist" containerID="ac347d12a46d5e9eaa809a19beafaf25bc19c0c03309650ab1ad5573efbfe027"
Sep 09 20:57:04.951855 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:04.951785    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac347d12a46d5e9eaa809a19beafaf25bc19c0c03309650ab1ad5573efbfe027"} err="failed to get container status \"ac347d12a46d5e9eaa809a19beafaf25bc19c0c03309650ab1ad5573efbfe027\": rpc error: code = NotFound desc = could not find container \"ac347d12a46d5e9eaa809a19beafaf25bc19c0c03309650ab1ad5573efbfe027\": container with ID starting with ac347d12a46d5e9eaa809a19beafaf25bc19c0c03309650ab1ad5573efbfe027 not found: ID does not exist"
Sep 09 20:57:04.951855 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:04.951802    2581 scope.go:117] "RemoveContainer" containerID="161ac86d91fcab7e0c69dbe55bbfc7fe88717654d867b3e6c97699cd72955648"
Sep 09 20:57:04.952223 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:57:04.952202    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"161ac86d91fcab7e0c69dbe55bbfc7fe88717654d867b3e6c97699cd72955648\": container with ID starting with 161ac86d91fcab7e0c69dbe55bbfc7fe88717654d867b3e6c97699cd72955648 not found: ID does not exist" containerID="161ac86d91fcab7e0c69dbe55bbfc7fe88717654d867b3e6c97699cd72955648"
Sep 09 20:57:04.952286 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:04.952230    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"161ac86d91fcab7e0c69dbe55bbfc7fe88717654d867b3e6c97699cd72955648"} err="failed to get container status \"161ac86d91fcab7e0c69dbe55bbfc7fe88717654d867b3e6c97699cd72955648\": rpc error: code = NotFound desc = could not find container \"161ac86d91fcab7e0c69dbe55bbfc7fe88717654d867b3e6c97699cd72955648\": container with ID starting with 161ac86d91fcab7e0c69dbe55bbfc7fe88717654d867b3e6c97699cd72955648 not found: ID does not exist"
Sep 09 20:57:04.967487 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:04.967460    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss"]
Sep 09 20:57:04.970604 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:04.970578    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-autores-864e4e-openshift-default-6c8d96c55d-vkmss"]
Sep 09 20:57:05.043559 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:05.043511    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lctq\" (UniqueName: \"kubernetes.io/projected/ed2eebef-e282-407b-8c6b-60277cca322a-kube-api-access-9lctq\") pod \"payload-processing-e2e-explicit-2d4edf-595d4b596f-swj8r\" (UID: \"ed2eebef-e282-407b-8c6b-60277cca322a\") " pod="openshift-ingress/payload-processing-e2e-explicit-2d4edf-595d4b596f-swj8r"
Sep 09 20:57:05.043750 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:05.043566    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed2eebef-e282-407b-8c6b-60277cca322a-plugins-config-volume\") pod \"payload-processing-e2e-explicit-2d4edf-595d4b596f-swj8r\" (UID: \"ed2eebef-e282-407b-8c6b-60277cca322a\") " pod="openshift-ingress/payload-processing-e2e-explicit-2d4edf-595d4b596f-swj8r"
Sep 09 20:57:05.043815 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:05.043740    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f0278e8-613e-43c3-ba18-733ba28577db-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-2d4edf-756bd96bbb-29lsh\" (UID: \"4f0278e8-613e-43c3-ba18-733ba28577db\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-2d4edf-756bd96bbb-29lsh"
Sep 09 20:57:05.043877 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:05.043810    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkvzh\" (UniqueName: \"kubernetes.io/projected/4f0278e8-613e-43c3-ba18-733ba28577db-kube-api-access-dkvzh\") pod \"payload-pre-processing-e2e-explicit-2d4edf-756bd96bbb-29lsh\" (UID: \"4f0278e8-613e-43c3-ba18-733ba28577db\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-2d4edf-756bd96bbb-29lsh"
Sep 09 20:57:05.144634 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:05.144544    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lctq\" (UniqueName: \"kubernetes.io/projected/ed2eebef-e282-407b-8c6b-60277cca322a-kube-api-access-9lctq\") pod \"payload-processing-e2e-explicit-2d4edf-595d4b596f-swj8r\" (UID: \"ed2eebef-e282-407b-8c6b-60277cca322a\") " pod="openshift-ingress/payload-processing-e2e-explicit-2d4edf-595d4b596f-swj8r"
Sep 09 20:57:05.144634 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:05.144591    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed2eebef-e282-407b-8c6b-60277cca322a-plugins-config-volume\") pod \"payload-processing-e2e-explicit-2d4edf-595d4b596f-swj8r\" (UID: \"ed2eebef-e282-407b-8c6b-60277cca322a\") " pod="openshift-ingress/payload-processing-e2e-explicit-2d4edf-595d4b596f-swj8r"
Sep 09 20:57:05.144867 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:05.144653    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f0278e8-613e-43c3-ba18-733ba28577db-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-2d4edf-756bd96bbb-29lsh\" (UID: \"4f0278e8-613e-43c3-ba18-733ba28577db\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-2d4edf-756bd96bbb-29lsh"
Sep 09 20:57:05.144867 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:05.144688    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkvzh\" (UniqueName: \"kubernetes.io/projected/4f0278e8-613e-43c3-ba18-733ba28577db-kube-api-access-dkvzh\") pod \"payload-pre-processing-e2e-explicit-2d4edf-756bd96bbb-29lsh\" (UID: \"4f0278e8-613e-43c3-ba18-733ba28577db\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-2d4edf-756bd96bbb-29lsh"
Sep 09 20:57:05.145388 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:05.145364    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f0278e8-613e-43c3-ba18-733ba28577db-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-2d4edf-756bd96bbb-29lsh\" (UID: \"4f0278e8-613e-43c3-ba18-733ba28577db\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-2d4edf-756bd96bbb-29lsh"
Sep 09 20:57:05.145505 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:05.145445    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed2eebef-e282-407b-8c6b-60277cca322a-plugins-config-volume\") pod \"payload-processing-e2e-explicit-2d4edf-595d4b596f-swj8r\" (UID: \"ed2eebef-e282-407b-8c6b-60277cca322a\") " pod="openshift-ingress/payload-processing-e2e-explicit-2d4edf-595d4b596f-swj8r"
Sep 09 20:57:05.153423 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:05.153397    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lctq\" (UniqueName: \"kubernetes.io/projected/ed2eebef-e282-407b-8c6b-60277cca322a-kube-api-access-9lctq\") pod \"payload-processing-e2e-explicit-2d4edf-595d4b596f-swj8r\" (UID: \"ed2eebef-e282-407b-8c6b-60277cca322a\") " pod="openshift-ingress/payload-processing-e2e-explicit-2d4edf-595d4b596f-swj8r"
Sep 09 20:57:05.153845 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:05.153827    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkvzh\" (UniqueName: \"kubernetes.io/projected/4f0278e8-613e-43c3-ba18-733ba28577db-kube-api-access-dkvzh\") pod \"payload-pre-processing-e2e-explicit-2d4edf-756bd96bbb-29lsh\" (UID: \"4f0278e8-613e-43c3-ba18-733ba28577db\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-2d4edf-756bd96bbb-29lsh"
Sep 09 20:57:05.225259 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:05.225225    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-2d4edf-756bd96bbb-29lsh"
Sep 09 20:57:05.244680 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:05.244658    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-2d4edf-595d4b596f-swj8r"
Sep 09 20:57:05.502279 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:05.502251    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-2d4edf-756bd96bbb-29lsh"]
Sep 09 20:57:05.504246 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:57:05.504218    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f0278e8_613e_43c3_ba18_733ba28577db.slice/crio-7ebc0618d578384a53887ecfeaa85e8b10065f782d199a70299e69ce6ee5b06e WatchSource:0}: Error finding container 7ebc0618d578384a53887ecfeaa85e8b10065f782d199a70299e69ce6ee5b06e: Status 404 returned error can't find the container with id 7ebc0618d578384a53887ecfeaa85e8b10065f782d199a70299e69ce6ee5b06e
Sep 09 20:57:05.532736 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:05.532715    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-2d4edf-595d4b596f-swj8r"]
Sep 09 20:57:05.534470 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:57:05.534444    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded2eebef_e282_407b_8c6b_60277cca322a.slice/crio-89d6d826f3e12b147970ebb34f6994addc3dd320adc4a0b24bfa63a0aa0ea13d WatchSource:0}: Error finding container 89d6d826f3e12b147970ebb34f6994addc3dd320adc4a0b24bfa63a0aa0ea13d: Status 404 returned error can't find the container with id 89d6d826f3e12b147970ebb34f6994addc3dd320adc4a0b24bfa63a0aa0ea13d
Sep 09 20:57:05.937556 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:05.937453    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-2d4edf-756bd96bbb-29lsh" event={"ID":"4f0278e8-613e-43c3-ba18-733ba28577db","Type":"ContainerStarted","Data":"325ce7b78a2c408aa6c2a864d9464e6c3d6bd342931d8817e66819a5130be9cb"}
Sep 09 20:57:05.937556 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:05.937520    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-2d4edf-756bd96bbb-29lsh" event={"ID":"4f0278e8-613e-43c3-ba18-733ba28577db","Type":"ContainerStarted","Data":"7ebc0618d578384a53887ecfeaa85e8b10065f782d199a70299e69ce6ee5b06e"}
Sep 09 20:57:05.937556 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:05.937550    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-2d4edf-756bd96bbb-29lsh"
Sep 09 20:57:05.939273 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:05.939245    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-2d4edf-595d4b596f-swj8r" event={"ID":"ed2eebef-e282-407b-8c6b-60277cca322a","Type":"ContainerStarted","Data":"71bbdca5fc8a8609f05c4d3c2282d878563408385cd092fa55009f8309ee57ad"}
Sep 09 20:57:05.939407 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:05.939294    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-2d4edf-595d4b596f-swj8r" event={"ID":"ed2eebef-e282-407b-8c6b-60277cca322a","Type":"ContainerStarted","Data":"89d6d826f3e12b147970ebb34f6994addc3dd320adc4a0b24bfa63a0aa0ea13d"}
Sep 09 20:57:05.939407 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:05.939385    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-explicit-2d4edf-595d4b596f-swj8r"
Sep 09 20:57:05.973056 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:05.973006    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-explicit-2d4edf-756bd96bbb-29lsh" podStartSLOduration=1.972990435 podStartE2EDuration="1.972990435s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:57:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:57:05.972046408 +0000 UTC m=+987.599674281" watchObservedRunningTime="2026-09-09 20:57:05.972990435 +0000 UTC m=+987.600618345"
Sep 09 20:57:06.010634 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:06.010585    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-explicit-2d4edf-595d4b596f-swj8r" podStartSLOduration=2.010570575 podStartE2EDuration="2.010570575s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:57:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:57:06.007358332 +0000 UTC m=+987.634986205" watchObservedRunningTime="2026-09-09 20:57:06.010570575 +0000 UTC m=+987.638198449"
Sep 09 20:57:06.820501 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:06.820466    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="92d54f63-73ce-4d1f-a0df-c0176265fb0f" path="/var/lib/kubelet/pods/92d54f63-73ce-4d1f-a0df-c0176265fb0f/volumes"
Sep 09 20:57:07.396918 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:07.396881    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-47e41c/premium-w4-47e41c-kserve-6b47f4769b-lrvrt"
Sep 09 20:57:07.397078 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:07.396936    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-47e41c/premium-w4-47e41c-kserve-6b47f4769b-lrvrt"
Sep 09 20:57:07.409510 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:07.409489    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-47e41c/premium-w4-47e41c-kserve-6b47f4769b-lrvrt"
Sep 09 20:57:07.964950 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:07.964925    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-47e41c/premium-w4-47e41c-kserve-6b47f4769b-lrvrt"
Sep 09 20:57:16.950715 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:16.950679    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-explicit-2d4edf-595d4b596f-swj8r"
Sep 09 20:57:16.951198 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:16.950886    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-2d4edf-756bd96bbb-29lsh"
Sep 09 20:57:31.093796 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:31.093763    2581 generic.go:356] "Generic (PLEG): container finished" podID="d8184bd0-d46c-41e0-9e1d-11ae37508fcc" containerID="e88f4843a1aaf0e61ac2131e4fcaf8d0df367b7ebaca415297c47280ccfb0e07" exitCode=0
Sep 09 20:57:31.094233 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:31.093833    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4" event={"ID":"d8184bd0-d46c-41e0-9e1d-11ae37508fcc","Type":"ContainerDied","Data":"e88f4843a1aaf0e61ac2131e4fcaf8d0df367b7ebaca415297c47280ccfb0e07"}
Sep 09 20:57:31.094293 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:31.094262    2581 scope.go:117] "RemoveContainer" containerID="e88f4843a1aaf0e61ac2131e4fcaf8d0df367b7ebaca415297c47280ccfb0e07"
Sep 09 20:57:31.095439 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:31.095413    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:57:31.095512 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:31.095480    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:57:31.095550 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:31.095529    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:57:31.905434 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:31.905402    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4"
Sep 09 20:57:32.102126 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:32.102079    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4" event={"ID":"d8184bd0-d46c-41e0-9e1d-11ae37508fcc","Type":"ContainerStarted","Data":"16ae552310fd1d4b83501d2eeaa0bcba3fb38203fb8a131aa3d84ec520226a24"}
Sep 09 20:57:32.153208 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:32.153155    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4"
Sep 09 20:57:32.154476 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:32.154444    2581 patch_prober.go:28] interesting pod/e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Sep 09 20:57:32.154614 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:32.154507    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4" podUID="d8184bd0-d46c-41e0-9e1d-11ae37508fcc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Sep 09 20:57:33.153493 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:33.153455    2581 patch_prober.go:28] interesting pod/e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Sep 09 20:57:33.153885 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:33.153526    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4" podUID="d8184bd0-d46c-41e0-9e1d-11ae37508fcc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Sep 09 20:57:34.153949 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:34.153913    2581 patch_prober.go:28] interesting pod/e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Sep 09 20:57:34.154369 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:34.153969    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4" podUID="d8184bd0-d46c-41e0-9e1d-11ae37508fcc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Sep 09 20:57:35.153803 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:35.153771    2581 patch_prober.go:28] interesting pod/e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Sep 09 20:57:35.153990 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:35.153825    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4" podUID="d8184bd0-d46c-41e0-9e1d-11ae37508fcc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Sep 09 20:57:36.153629 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:36.153591    2581 patch_prober.go:28] interesting pod/e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Sep 09 20:57:36.153814 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:36.153650    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4" podUID="d8184bd0-d46c-41e0-9e1d-11ae37508fcc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Sep 09 20:57:37.153682 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:37.153615    2581 patch_prober.go:28] interesting pod/e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Sep 09 20:57:37.153682 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:37.153670    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4" podUID="d8184bd0-d46c-41e0-9e1d-11ae37508fcc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Sep 09 20:57:38.157665 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:38.157637    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4"
Sep 09 20:57:38.158150 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:38.157898    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4"
Sep 09 20:57:38.158574 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:57:38.158559    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4"
Sep 09 20:58:01.428799 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:01.428759    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-explicit-2d4edf/explicit-model-2d4edf-kserve-665bb8cb7-wf7sb"]
Sep 09 20:58:01.433986 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:01.433966    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-2d4edf/explicit-model-2d4edf-kserve-665bb8cb7-wf7sb"
Sep 09 20:58:01.436665 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:01.436644    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-2d4edf\"/\"explicit-model-2d4edf-kserve-self-signed-certs\""
Sep 09 20:58:01.437698 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:01.437681    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-2d4edf\"/\"default-dockercfg-8bnkf\""
Sep 09 20:58:01.437754 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:01.437703    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-2d4edf\"/\"kube-root-ca.crt\""
Sep 09 20:58:01.437810 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:01.437686    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-2d4edf\"/\"openshift-service-ca.crt\""
Sep 09 20:58:01.446081 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:01.446046    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-2d4edf/explicit-model-2d4edf-kserve-665bb8cb7-wf7sb"]
Sep 09 20:58:01.532084 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:01.532049    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/027d2a28-eb79-405e-bef2-986ff736a808-home\") pod \"explicit-model-2d4edf-kserve-665bb8cb7-wf7sb\" (UID: \"027d2a28-eb79-405e-bef2-986ff736a808\") " pod="ai-tenant-e2e-explicit-2d4edf/explicit-model-2d4edf-kserve-665bb8cb7-wf7sb"
Sep 09 20:58:01.532265 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:01.532095    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vphl\" (UniqueName: \"kubernetes.io/projected/027d2a28-eb79-405e-bef2-986ff736a808-kube-api-access-8vphl\") pod \"explicit-model-2d4edf-kserve-665bb8cb7-wf7sb\" (UID: \"027d2a28-eb79-405e-bef2-986ff736a808\") " pod="ai-tenant-e2e-explicit-2d4edf/explicit-model-2d4edf-kserve-665bb8cb7-wf7sb"
Sep 09 20:58:01.532265 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:01.532171    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/027d2a28-eb79-405e-bef2-986ff736a808-dshm\") pod \"explicit-model-2d4edf-kserve-665bb8cb7-wf7sb\" (UID: \"027d2a28-eb79-405e-bef2-986ff736a808\") " pod="ai-tenant-e2e-explicit-2d4edf/explicit-model-2d4edf-kserve-665bb8cb7-wf7sb"
Sep 09 20:58:01.532265 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:01.532209    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/027d2a28-eb79-405e-bef2-986ff736a808-tmp-dir\") pod \"explicit-model-2d4edf-kserve-665bb8cb7-wf7sb\" (UID: \"027d2a28-eb79-405e-bef2-986ff736a808\") " pod="ai-tenant-e2e-explicit-2d4edf/explicit-model-2d4edf-kserve-665bb8cb7-wf7sb"
Sep 09 20:58:01.532265 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:01.532251    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/027d2a28-eb79-405e-bef2-986ff736a808-model-cache\") pod \"explicit-model-2d4edf-kserve-665bb8cb7-wf7sb\" (UID: \"027d2a28-eb79-405e-bef2-986ff736a808\") " pod="ai-tenant-e2e-explicit-2d4edf/explicit-model-2d4edf-kserve-665bb8cb7-wf7sb"
Sep 09 20:58:01.532407 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:01.532310    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/027d2a28-eb79-405e-bef2-986ff736a808-tls-certs\") pod \"explicit-model-2d4edf-kserve-665bb8cb7-wf7sb\" (UID: \"027d2a28-eb79-405e-bef2-986ff736a808\") " pod="ai-tenant-e2e-explicit-2d4edf/explicit-model-2d4edf-kserve-665bb8cb7-wf7sb"
Sep 09 20:58:01.633573 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:01.633541    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/027d2a28-eb79-405e-bef2-986ff736a808-dshm\") pod \"explicit-model-2d4edf-kserve-665bb8cb7-wf7sb\" (UID: \"027d2a28-eb79-405e-bef2-986ff736a808\") " pod="ai-tenant-e2e-explicit-2d4edf/explicit-model-2d4edf-kserve-665bb8cb7-wf7sb"
Sep 09 20:58:01.633766 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:01.633580    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/027d2a28-eb79-405e-bef2-986ff736a808-tmp-dir\") pod \"explicit-model-2d4edf-kserve-665bb8cb7-wf7sb\" (UID: \"027d2a28-eb79-405e-bef2-986ff736a808\") " pod="ai-tenant-e2e-explicit-2d4edf/explicit-model-2d4edf-kserve-665bb8cb7-wf7sb"
Sep 09 20:58:01.633766 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:01.633628    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/027d2a28-eb79-405e-bef2-986ff736a808-model-cache\") pod \"explicit-model-2d4edf-kserve-665bb8cb7-wf7sb\" (UID: \"027d2a28-eb79-405e-bef2-986ff736a808\") " pod="ai-tenant-e2e-explicit-2d4edf/explicit-model-2d4edf-kserve-665bb8cb7-wf7sb"
Sep 09 20:58:01.633766 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:01.633685    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/027d2a28-eb79-405e-bef2-986ff736a808-tls-certs\") pod \"explicit-model-2d4edf-kserve-665bb8cb7-wf7sb\" (UID: \"027d2a28-eb79-405e-bef2-986ff736a808\") " pod="ai-tenant-e2e-explicit-2d4edf/explicit-model-2d4edf-kserve-665bb8cb7-wf7sb"
Sep 09 20:58:01.633934 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:01.633798    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/027d2a28-eb79-405e-bef2-986ff736a808-home\") pod \"explicit-model-2d4edf-kserve-665bb8cb7-wf7sb\" (UID: \"027d2a28-eb79-405e-bef2-986ff736a808\") " pod="ai-tenant-e2e-explicit-2d4edf/explicit-model-2d4edf-kserve-665bb8cb7-wf7sb"
Sep 09 20:58:01.633934 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:01.633848    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vphl\" (UniqueName: \"kubernetes.io/projected/027d2a28-eb79-405e-bef2-986ff736a808-kube-api-access-8vphl\") pod \"explicit-model-2d4edf-kserve-665bb8cb7-wf7sb\" (UID: \"027d2a28-eb79-405e-bef2-986ff736a808\") " pod="ai-tenant-e2e-explicit-2d4edf/explicit-model-2d4edf-kserve-665bb8cb7-wf7sb"
Sep 09 20:58:01.634065 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:01.634022    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/027d2a28-eb79-405e-bef2-986ff736a808-tmp-dir\") pod \"explicit-model-2d4edf-kserve-665bb8cb7-wf7sb\" (UID: \"027d2a28-eb79-405e-bef2-986ff736a808\") " pod="ai-tenant-e2e-explicit-2d4edf/explicit-model-2d4edf-kserve-665bb8cb7-wf7sb"
Sep 09 20:58:01.634065 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:01.634049    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/027d2a28-eb79-405e-bef2-986ff736a808-model-cache\") pod \"explicit-model-2d4edf-kserve-665bb8cb7-wf7sb\" (UID: \"027d2a28-eb79-405e-bef2-986ff736a808\") " pod="ai-tenant-e2e-explicit-2d4edf/explicit-model-2d4edf-kserve-665bb8cb7-wf7sb"
Sep 09 20:58:01.634198 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:01.634161    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/027d2a28-eb79-405e-bef2-986ff736a808-home\") pod \"explicit-model-2d4edf-kserve-665bb8cb7-wf7sb\" (UID: \"027d2a28-eb79-405e-bef2-986ff736a808\") " pod="ai-tenant-e2e-explicit-2d4edf/explicit-model-2d4edf-kserve-665bb8cb7-wf7sb"
Sep 09 20:58:01.635742 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:01.635722    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/027d2a28-eb79-405e-bef2-986ff736a808-dshm\") pod \"explicit-model-2d4edf-kserve-665bb8cb7-wf7sb\" (UID: \"027d2a28-eb79-405e-bef2-986ff736a808\") " pod="ai-tenant-e2e-explicit-2d4edf/explicit-model-2d4edf-kserve-665bb8cb7-wf7sb"
Sep 09 20:58:01.636151 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:01.636131    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/027d2a28-eb79-405e-bef2-986ff736a808-tls-certs\") pod \"explicit-model-2d4edf-kserve-665bb8cb7-wf7sb\" (UID: \"027d2a28-eb79-405e-bef2-986ff736a808\") " pod="ai-tenant-e2e-explicit-2d4edf/explicit-model-2d4edf-kserve-665bb8cb7-wf7sb"
Sep 09 20:58:01.646430 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:01.646402    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vphl\" (UniqueName: \"kubernetes.io/projected/027d2a28-eb79-405e-bef2-986ff736a808-kube-api-access-8vphl\") pod \"explicit-model-2d4edf-kserve-665bb8cb7-wf7sb\" (UID: \"027d2a28-eb79-405e-bef2-986ff736a808\") " pod="ai-tenant-e2e-explicit-2d4edf/explicit-model-2d4edf-kserve-665bb8cb7-wf7sb"
Sep 09 20:58:01.745196 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:01.745174    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-2d4edf/explicit-model-2d4edf-kserve-665bb8cb7-wf7sb"
Sep 09 20:58:02.023757 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:02.023679    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-2d4edf/explicit-model-2d4edf-kserve-665bb8cb7-wf7sb"]
Sep 09 20:58:02.027028 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:58:02.026996    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod027d2a28_eb79_405e_bef2_986ff736a808.slice/crio-2190ed85bf2fcfe5be6d4847e1b504878c8fb750173d980663d5f69fe34666dc WatchSource:0}: Error finding container 2190ed85bf2fcfe5be6d4847e1b504878c8fb750173d980663d5f69fe34666dc: Status 404 returned error can't find the container with id 2190ed85bf2fcfe5be6d4847e1b504878c8fb750173d980663d5f69fe34666dc
Sep 09 20:58:02.279926 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:02.279829    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-2d4edf/explicit-model-2d4edf-kserve-665bb8cb7-wf7sb" event={"ID":"027d2a28-eb79-405e-bef2-986ff736a808","Type":"ContainerStarted","Data":"19a1dc62a84ec50b9b5555287675123eafc3f4aeac19f33b869f2523d31dbc47"}
Sep 09 20:58:02.279926 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:02.279874    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-2d4edf/explicit-model-2d4edf-kserve-665bb8cb7-wf7sb" event={"ID":"027d2a28-eb79-405e-bef2-986ff736a808","Type":"ContainerStarted","Data":"2190ed85bf2fcfe5be6d4847e1b504878c8fb750173d980663d5f69fe34666dc"}
Sep 09 20:58:02.322139 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:02.322076    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-explicit-2d4edf/explicit-model-2d4edf-kserve-665bb8cb7-wf7sb" podStartSLOduration=1.32205458 podStartE2EDuration="1.32205458s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:58:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:58:02.313022637 +0000 UTC m=+1043.940650509" watchObservedRunningTime="2026-09-09 20:58:02.32205458 +0000 UTC m=+1043.949682453"
Sep 09 20:58:11.745560 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:11.745514    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-explicit-2d4edf/explicit-model-2d4edf-kserve-665bb8cb7-wf7sb"
Sep 09 20:58:11.745560 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:11.745573    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-explicit-2d4edf/explicit-model-2d4edf-kserve-665bb8cb7-wf7sb"
Sep 09 20:58:11.758486 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:11.758462    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-explicit-2d4edf/explicit-model-2d4edf-kserve-665bb8cb7-wf7sb"
Sep 09 20:58:12.347575 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:12.347543    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-explicit-2d4edf/explicit-model-2d4edf-kserve-665bb8cb7-wf7sb"
Sep 09 20:58:24.642244 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:24.642212    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h"]
Sep 09 20:58:24.646053 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:24.646031    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h"
Sep 09 20:58:24.648436 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:24.648420    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-e2f4649f-openshift-default-dockercfg-pvp79\""
Sep 09 20:58:24.663754 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:24.663732    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h"]
Sep 09 20:58:24.772893 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:24.772856    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4l68\" (UniqueName: \"kubernetes.io/projected/2d1fc6e2-9473-40ed-b93a-dc1a3722d473-kube-api-access-l4l68\") pod \"e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h\" (UID: \"2d1fc6e2-9473-40ed-b93a-dc1a3722d473\") " pod="openshift-ingress/e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h"
Sep 09 20:58:24.773092 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:24.772900    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d1fc6e2-9473-40ed-b93a-dc1a3722d473-istio-podinfo\") pod \"e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h\" (UID: \"2d1fc6e2-9473-40ed-b93a-dc1a3722d473\") " pod="openshift-ingress/e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h"
Sep 09 20:58:24.773092 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:24.773019    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d1fc6e2-9473-40ed-b93a-dc1a3722d473-istio-envoy\") pod \"e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h\" (UID: \"2d1fc6e2-9473-40ed-b93a-dc1a3722d473\") " pod="openshift-ingress/e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h"
Sep 09 20:58:24.773092 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:24.773073    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d1fc6e2-9473-40ed-b93a-dc1a3722d473-credential-socket\") pod \"e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h\" (UID: \"2d1fc6e2-9473-40ed-b93a-dc1a3722d473\") " pod="openshift-ingress/e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h"
Sep 09 20:58:24.773343 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:24.773129    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d1fc6e2-9473-40ed-b93a-dc1a3722d473-istio-data\") pod \"e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h\" (UID: \"2d1fc6e2-9473-40ed-b93a-dc1a3722d473\") " pod="openshift-ingress/e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h"
Sep 09 20:58:24.773343 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:24.773198    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d1fc6e2-9473-40ed-b93a-dc1a3722d473-workload-certs\") pod \"e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h\" (UID: \"2d1fc6e2-9473-40ed-b93a-dc1a3722d473\") " pod="openshift-ingress/e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h"
Sep 09 20:58:24.773343 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:24.773247    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d1fc6e2-9473-40ed-b93a-dc1a3722d473-workload-socket\") pod \"e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h\" (UID: \"2d1fc6e2-9473-40ed-b93a-dc1a3722d473\") " pod="openshift-ingress/e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h"
Sep 09 20:58:24.773530 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:24.773349    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d1fc6e2-9473-40ed-b93a-dc1a3722d473-istio-token\") pod \"e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h\" (UID: \"2d1fc6e2-9473-40ed-b93a-dc1a3722d473\") " pod="openshift-ingress/e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h"
Sep 09 20:58:24.773530 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:24.773410    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d1fc6e2-9473-40ed-b93a-dc1a3722d473-istiod-ca-cert\") pod \"e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h\" (UID: \"2d1fc6e2-9473-40ed-b93a-dc1a3722d473\") " pod="openshift-ingress/e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h"
Sep 09 20:58:24.874944 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:24.874913    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d1fc6e2-9473-40ed-b93a-dc1a3722d473-istiod-ca-cert\") pod \"e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h\" (UID: \"2d1fc6e2-9473-40ed-b93a-dc1a3722d473\") " pod="openshift-ingress/e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h"
Sep 09 20:58:24.875136 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:24.874959    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4l68\" (UniqueName: \"kubernetes.io/projected/2d1fc6e2-9473-40ed-b93a-dc1a3722d473-kube-api-access-l4l68\") pod \"e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h\" (UID: \"2d1fc6e2-9473-40ed-b93a-dc1a3722d473\") " pod="openshift-ingress/e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h"
Sep 09 20:58:24.875136 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:24.874986    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d1fc6e2-9473-40ed-b93a-dc1a3722d473-istio-podinfo\") pod \"e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h\" (UID: \"2d1fc6e2-9473-40ed-b93a-dc1a3722d473\") " pod="openshift-ingress/e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h"
Sep 09 20:58:24.875136 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:24.875027    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d1fc6e2-9473-40ed-b93a-dc1a3722d473-istio-envoy\") pod \"e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h\" (UID: \"2d1fc6e2-9473-40ed-b93a-dc1a3722d473\") " pod="openshift-ingress/e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h"
Sep 09 20:58:24.875136 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:24.875058    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d1fc6e2-9473-40ed-b93a-dc1a3722d473-credential-socket\") pod \"e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h\" (UID: \"2d1fc6e2-9473-40ed-b93a-dc1a3722d473\") " pod="openshift-ingress/e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h"
Sep 09 20:58:24.875136 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:24.875128    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d1fc6e2-9473-40ed-b93a-dc1a3722d473-istio-data\") pod \"e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h\" (UID: \"2d1fc6e2-9473-40ed-b93a-dc1a3722d473\") " pod="openshift-ingress/e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h"
Sep 09 20:58:24.875428 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:24.875209    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d1fc6e2-9473-40ed-b93a-dc1a3722d473-workload-certs\") pod \"e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h\" (UID: \"2d1fc6e2-9473-40ed-b93a-dc1a3722d473\") " pod="openshift-ingress/e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h"
Sep 09 20:58:24.875428 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:24.875274    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d1fc6e2-9473-40ed-b93a-dc1a3722d473-workload-socket\") pod \"e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h\" (UID: \"2d1fc6e2-9473-40ed-b93a-dc1a3722d473\") " pod="openshift-ingress/e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h"
Sep 09 20:58:24.875428 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:24.875365    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d1fc6e2-9473-40ed-b93a-dc1a3722d473-istio-token\") pod \"e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h\" (UID: \"2d1fc6e2-9473-40ed-b93a-dc1a3722d473\") " pod="openshift-ingress/e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h"
Sep 09 20:58:24.875591 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:24.875463    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d1fc6e2-9473-40ed-b93a-dc1a3722d473-istio-data\") pod \"e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h\" (UID: \"2d1fc6e2-9473-40ed-b93a-dc1a3722d473\") " pod="openshift-ingress/e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h"
Sep 09 20:58:24.875591 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:24.875556    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d1fc6e2-9473-40ed-b93a-dc1a3722d473-workload-certs\") pod \"e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h\" (UID: \"2d1fc6e2-9473-40ed-b93a-dc1a3722d473\") " pod="openshift-ingress/e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h"
Sep 09 20:58:24.875739 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:24.875641    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d1fc6e2-9473-40ed-b93a-dc1a3722d473-workload-socket\") pod \"e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h\" (UID: \"2d1fc6e2-9473-40ed-b93a-dc1a3722d473\") " pod="openshift-ingress/e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h"
Sep 09 20:58:24.875739 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:24.875702    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d1fc6e2-9473-40ed-b93a-dc1a3722d473-credential-socket\") pod \"e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h\" (UID: \"2d1fc6e2-9473-40ed-b93a-dc1a3722d473\") " pod="openshift-ingress/e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h"
Sep 09 20:58:24.875815 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:24.875767    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d1fc6e2-9473-40ed-b93a-dc1a3722d473-istiod-ca-cert\") pod \"e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h\" (UID: \"2d1fc6e2-9473-40ed-b93a-dc1a3722d473\") " pod="openshift-ingress/e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h"
Sep 09 20:58:24.877364 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:24.877343    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d1fc6e2-9473-40ed-b93a-dc1a3722d473-istio-envoy\") pod \"e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h\" (UID: \"2d1fc6e2-9473-40ed-b93a-dc1a3722d473\") " pod="openshift-ingress/e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h"
Sep 09 20:58:24.877605 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:24.877587    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d1fc6e2-9473-40ed-b93a-dc1a3722d473-istio-podinfo\") pod \"e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h\" (UID: \"2d1fc6e2-9473-40ed-b93a-dc1a3722d473\") " pod="openshift-ingress/e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h"
Sep 09 20:58:24.887745 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:24.887723    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d1fc6e2-9473-40ed-b93a-dc1a3722d473-istio-token\") pod \"e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h\" (UID: \"2d1fc6e2-9473-40ed-b93a-dc1a3722d473\") " pod="openshift-ingress/e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h"
Sep 09 20:58:24.892582 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:24.892525    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4l68\" (UniqueName: \"kubernetes.io/projected/2d1fc6e2-9473-40ed-b93a-dc1a3722d473-kube-api-access-l4l68\") pod \"e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h\" (UID: \"2d1fc6e2-9473-40ed-b93a-dc1a3722d473\") " pod="openshift-ingress/e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h"
Sep 09 20:58:24.956579 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:24.956536    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h"
Sep 09 20:58:25.270602 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:25.270565    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h"]
Sep 09 20:58:25.273023 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:58:25.272997    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d1fc6e2_9473_40ed_b93a_dc1a3722d473.slice/crio-a833d347c012648ff1888058a90fd950666d256ca6246dcebc89ddaefb1f1560 WatchSource:0}: Error finding container a833d347c012648ff1888058a90fd950666d256ca6246dcebc89ddaefb1f1560: Status 404 returned error can't find the container with id a833d347c012648ff1888058a90fd950666d256ca6246dcebc89ddaefb1f1560
Sep 09 20:58:25.275268 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:25.275233    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:58:25.275359 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:25.275300    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:58:25.275359 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:25.275332    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:58:25.418818 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:25.418787    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h" event={"ID":"2d1fc6e2-9473-40ed-b93a-dc1a3722d473","Type":"ContainerStarted","Data":"6d095799028f6da4228e0ddd522626f1941dea4f73570f2cd6fb2e198d0e5e69"}
Sep 09 20:58:25.419033 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:25.418838    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h" event={"ID":"2d1fc6e2-9473-40ed-b93a-dc1a3722d473","Type":"ContainerStarted","Data":"a833d347c012648ff1888058a90fd950666d256ca6246dcebc89ddaefb1f1560"}
Sep 09 20:58:25.441197 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:25.441134    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h" podStartSLOduration=1.441117822 podStartE2EDuration="1.441117822s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:58:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:58:25.439057633 +0000 UTC m=+1067.066685507" watchObservedRunningTime="2026-09-09 20:58:25.441117822 +0000 UTC m=+1067.068745686"
Sep 09 20:58:25.957210 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:25.957170    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h"
Sep 09 20:58:26.961739 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:26.961707    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h"
Sep 09 20:58:27.429777 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:27.429742    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h"
Sep 09 20:58:27.430710 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:27.430688    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h"
Sep 09 20:58:33.379963 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:33.379885    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e2f4649f-657d677566-bq9lx"]
Sep 09 20:58:33.384839 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:33.384819    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-e2f4649f-657d677566-bq9lx"
Sep 09 20:58:33.387665 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:33.387644    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-e2f4649f\""
Sep 09 20:58:33.387771 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:33.387758    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-e2f4649f-dockercfg-crrd5\""
Sep 09 20:58:33.396628 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:33.396600    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e2f4649f-657d677566-bq9lx"]
Sep 09 20:58:33.419561 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:33.419538    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e2f4649f-6c8555c6f7-ns4bw"]
Sep 09 20:58:33.423050 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:33.423035    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e2f4649f-6c8555c6f7-ns4bw"
Sep 09 20:58:33.433741 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:33.433717    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e2f4649f-6c8555c6f7-ns4bw"]
Sep 09 20:58:33.465260 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:33.465228    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ba3fb41e-2c50-4146-aeaf-063a3be8eeb1-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-e2f4649f-657d677566-bq9lx\" (UID: \"ba3fb41e-2c50-4146-aeaf-063a3be8eeb1\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e2f4649f-657d677566-bq9lx"
Sep 09 20:58:33.465411 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:33.465268    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a602ad2d-daf0-418d-a4e8-841ff4a82438-plugins-config-volume\") pod \"payload-processing-e2e-mt-e2f4649f-6c8555c6f7-ns4bw\" (UID: \"a602ad2d-daf0-418d-a4e8-841ff4a82438\") " pod="openshift-ingress/payload-processing-e2e-mt-e2f4649f-6c8555c6f7-ns4bw"
Sep 09 20:58:33.465411 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:33.465321    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvrpj\" (UniqueName: \"kubernetes.io/projected/a602ad2d-daf0-418d-a4e8-841ff4a82438-kube-api-access-dvrpj\") pod \"payload-processing-e2e-mt-e2f4649f-6c8555c6f7-ns4bw\" (UID: \"a602ad2d-daf0-418d-a4e8-841ff4a82438\") " pod="openshift-ingress/payload-processing-e2e-mt-e2f4649f-6c8555c6f7-ns4bw"
Sep 09 20:58:33.465411 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:33.465379    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tx7d4\" (UniqueName: \"kubernetes.io/projected/ba3fb41e-2c50-4146-aeaf-063a3be8eeb1-kube-api-access-tx7d4\") pod \"payload-pre-processing-e2e-mt-e2f4649f-657d677566-bq9lx\" (UID: \"ba3fb41e-2c50-4146-aeaf-063a3be8eeb1\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e2f4649f-657d677566-bq9lx"
Sep 09 20:58:33.566091 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:33.566045    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ba3fb41e-2c50-4146-aeaf-063a3be8eeb1-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-e2f4649f-657d677566-bq9lx\" (UID: \"ba3fb41e-2c50-4146-aeaf-063a3be8eeb1\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e2f4649f-657d677566-bq9lx"
Sep 09 20:58:33.566091 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:33.566129    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a602ad2d-daf0-418d-a4e8-841ff4a82438-plugins-config-volume\") pod \"payload-processing-e2e-mt-e2f4649f-6c8555c6f7-ns4bw\" (UID: \"a602ad2d-daf0-418d-a4e8-841ff4a82438\") " pod="openshift-ingress/payload-processing-e2e-mt-e2f4649f-6c8555c6f7-ns4bw"
Sep 09 20:58:33.566358 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:33.566184    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvrpj\" (UniqueName: \"kubernetes.io/projected/a602ad2d-daf0-418d-a4e8-841ff4a82438-kube-api-access-dvrpj\") pod \"payload-processing-e2e-mt-e2f4649f-6c8555c6f7-ns4bw\" (UID: \"a602ad2d-daf0-418d-a4e8-841ff4a82438\") " pod="openshift-ingress/payload-processing-e2e-mt-e2f4649f-6c8555c6f7-ns4bw"
Sep 09 20:58:33.566358 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:33.566289    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tx7d4\" (UniqueName: \"kubernetes.io/projected/ba3fb41e-2c50-4146-aeaf-063a3be8eeb1-kube-api-access-tx7d4\") pod \"payload-pre-processing-e2e-mt-e2f4649f-657d677566-bq9lx\" (UID: \"ba3fb41e-2c50-4146-aeaf-063a3be8eeb1\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e2f4649f-657d677566-bq9lx"
Sep 09 20:58:33.566858 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:33.566838    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ba3fb41e-2c50-4146-aeaf-063a3be8eeb1-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-e2f4649f-657d677566-bq9lx\" (UID: \"ba3fb41e-2c50-4146-aeaf-063a3be8eeb1\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e2f4649f-657d677566-bq9lx"
Sep 09 20:58:33.566898 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:33.566869    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a602ad2d-daf0-418d-a4e8-841ff4a82438-plugins-config-volume\") pod \"payload-processing-e2e-mt-e2f4649f-6c8555c6f7-ns4bw\" (UID: \"a602ad2d-daf0-418d-a4e8-841ff4a82438\") " pod="openshift-ingress/payload-processing-e2e-mt-e2f4649f-6c8555c6f7-ns4bw"
Sep 09 20:58:33.574683 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:33.574661    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvrpj\" (UniqueName: \"kubernetes.io/projected/a602ad2d-daf0-418d-a4e8-841ff4a82438-kube-api-access-dvrpj\") pod \"payload-processing-e2e-mt-e2f4649f-6c8555c6f7-ns4bw\" (UID: \"a602ad2d-daf0-418d-a4e8-841ff4a82438\") " pod="openshift-ingress/payload-processing-e2e-mt-e2f4649f-6c8555c6f7-ns4bw"
Sep 09 20:58:33.574991 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:33.574972    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tx7d4\" (UniqueName: \"kubernetes.io/projected/ba3fb41e-2c50-4146-aeaf-063a3be8eeb1-kube-api-access-tx7d4\") pod \"payload-pre-processing-e2e-mt-e2f4649f-657d677566-bq9lx\" (UID: \"ba3fb41e-2c50-4146-aeaf-063a3be8eeb1\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e2f4649f-657d677566-bq9lx"
Sep 09 20:58:33.697162 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:33.697062    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-e2f4649f-657d677566-bq9lx"
Sep 09 20:58:33.734516 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:33.734479    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e2f4649f-6c8555c6f7-ns4bw"
Sep 09 20:58:33.975197 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:33.975167    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e2f4649f-657d677566-bq9lx"]
Sep 09 20:58:33.976477 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:58:33.976452    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba3fb41e_2c50_4146_aeaf_063a3be8eeb1.slice/crio-56d331b56cebbba3b4a57017a942421b743147fb6562f6987f522959109c58f8 WatchSource:0}: Error finding container 56d331b56cebbba3b4a57017a942421b743147fb6562f6987f522959109c58f8: Status 404 returned error can't find the container with id 56d331b56cebbba3b4a57017a942421b743147fb6562f6987f522959109c58f8
Sep 09 20:58:34.022754 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:34.022729    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e2f4649f-6c8555c6f7-ns4bw"]
Sep 09 20:58:34.025357 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:58:34.025334    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda602ad2d_daf0_418d_a4e8_841ff4a82438.slice/crio-0cce3cdf93ffac1f752e07c63b7639cb951116a4123970f524286dd2f9daac99 WatchSource:0}: Error finding container 0cce3cdf93ffac1f752e07c63b7639cb951116a4123970f524286dd2f9daac99: Status 404 returned error can't find the container with id 0cce3cdf93ffac1f752e07c63b7639cb951116a4123970f524286dd2f9daac99
Sep 09 20:58:34.466470 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:34.466428    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e2f4649f-6c8555c6f7-ns4bw" event={"ID":"a602ad2d-daf0-418d-a4e8-841ff4a82438","Type":"ContainerStarted","Data":"0974c8621941b948c53d838529093887af850548c79de42ac518abfb7a3b578f"}
Sep 09 20:58:34.466960 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:34.466503    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-e2f4649f-6c8555c6f7-ns4bw"
Sep 09 20:58:34.466960 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:34.466532    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e2f4649f-6c8555c6f7-ns4bw" event={"ID":"a602ad2d-daf0-418d-a4e8-841ff4a82438","Type":"ContainerStarted","Data":"0cce3cdf93ffac1f752e07c63b7639cb951116a4123970f524286dd2f9daac99"}
Sep 09 20:58:34.468033 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:34.468007    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-e2f4649f-657d677566-bq9lx" event={"ID":"ba3fb41e-2c50-4146-aeaf-063a3be8eeb1","Type":"ContainerStarted","Data":"26e9ef9b79ae370f0ebd34ab6168c12e30ec673529dba6d3ece1586223e1fd35"}
Sep 09 20:58:34.468168 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:34.468073    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-e2f4649f-657d677566-bq9lx"
Sep 09 20:58:34.468168 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:34.468125    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-e2f4649f-657d677566-bq9lx" event={"ID":"ba3fb41e-2c50-4146-aeaf-063a3be8eeb1","Type":"ContainerStarted","Data":"56d331b56cebbba3b4a57017a942421b743147fb6562f6987f522959109c58f8"}
Sep 09 20:58:34.488808 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:34.488753    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-e2f4649f-6c8555c6f7-ns4bw" podStartSLOduration=1.488731832 podStartE2EDuration="1.488731832s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:58:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:58:34.486699773 +0000 UTC m=+1076.114327648" watchObservedRunningTime="2026-09-09 20:58:34.488731832 +0000 UTC m=+1076.116359708"
Sep 09 20:58:34.509312 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:34.509254    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-e2f4649f-657d677566-bq9lx" podStartSLOduration=1.5092347510000002 podStartE2EDuration="1.509234751s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:58:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:58:34.508251582 +0000 UTC m=+1076.135879457" watchObservedRunningTime="2026-09-09 20:58:34.509234751 +0000 UTC m=+1076.136862627"
Sep 09 20:58:34.823330 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:34.823293    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-2d4edf-l2lhw"]
Sep 09 20:58:34.827534 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:34.827515    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-2d4edf-l2lhw"
Sep 09 20:58:34.830816 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:34.830789    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-glz44\""
Sep 09 20:58:34.832950 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:34.832924    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-2d4edf-l2lhw"]
Sep 09 20:58:34.877267 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:34.877231    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8fa2269-0606-42c2-8803-feab16d87374-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-2d4edf-l2lhw\" (UID: \"f8fa2269-0606-42c2-8803-feab16d87374\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-2d4edf-l2lhw"
Sep 09 20:58:34.978594 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:34.978556    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8fa2269-0606-42c2-8803-feab16d87374-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-2d4edf-l2lhw\" (UID: \"f8fa2269-0606-42c2-8803-feab16d87374\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-2d4edf-l2lhw"
Sep 09 20:58:34.979206 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:34.979186    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8fa2269-0606-42c2-8803-feab16d87374-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-2d4edf-l2lhw\" (UID: \"f8fa2269-0606-42c2-8803-feab16d87374\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-2d4edf-l2lhw"
Sep 09 20:58:35.143153 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:35.143044    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-2d4edf-l2lhw"
Sep 09 20:58:35.415885 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:35.415858    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-2d4edf-l2lhw"]
Sep 09 20:58:35.418326 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:58:35.418296    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8fa2269_0606_42c2_8803_feab16d87374.slice/crio-284bcd4b3fb05cfadf9a97b737597fe375cd45b7cf0df7b54031c77a45b7e7ae WatchSource:0}: Error finding container 284bcd4b3fb05cfadf9a97b737597fe375cd45b7cf0df7b54031c77a45b7e7ae: Status 404 returned error can't find the container with id 284bcd4b3fb05cfadf9a97b737597fe375cd45b7cf0df7b54031c77a45b7e7ae
Sep 09 20:58:35.477590 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:35.477557    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-2d4edf-l2lhw" event={"ID":"f8fa2269-0606-42c2-8803-feab16d87374","Type":"ContainerStarted","Data":"284bcd4b3fb05cfadf9a97b737597fe375cd45b7cf0df7b54031c77a45b7e7ae"}
Sep 09 20:58:36.484751 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:36.484718    2581 generic.go:356] "Generic (PLEG): container finished" podID="f8fa2269-0606-42c2-8803-feab16d87374" containerID="e00aff975e91660a54658df827bfac09babb55cd4052aa302607638554bae993" exitCode=0
Sep 09 20:58:36.485158 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:36.484800    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-2d4edf-l2lhw" event={"ID":"f8fa2269-0606-42c2-8803-feab16d87374","Type":"ContainerDied","Data":"e00aff975e91660a54658df827bfac09babb55cd4052aa302607638554bae993"}
Sep 09 20:58:37.624085 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:37.624063    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-2d4edf-l2lhw"
Sep 09 20:58:37.708836 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:37.708801    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8fa2269-0606-42c2-8803-feab16d87374-maas-api-service-ca\") pod \"f8fa2269-0606-42c2-8803-feab16d87374\" (UID: \"f8fa2269-0606-42c2-8803-feab16d87374\") "
Sep 09 20:58:37.709200 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:37.709176    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8fa2269-0606-42c2-8803-feab16d87374-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f8fa2269-0606-42c2-8803-feab16d87374" (UID: "f8fa2269-0606-42c2-8803-feab16d87374"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:58:37.810705 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:37.810619    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8fa2269-0606-42c2-8803-feab16d87374-maas-api-service-ca\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:58:38.498012 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:38.497977    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-2d4edf-l2lhw" event={"ID":"f8fa2269-0606-42c2-8803-feab16d87374","Type":"ContainerDied","Data":"284bcd4b3fb05cfadf9a97b737597fe375cd45b7cf0df7b54031c77a45b7e7ae"}
Sep 09 20:58:38.498252 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:38.498019    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-2d4edf-l2lhw"
Sep 09 20:58:38.498252 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:38.498023    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="284bcd4b3fb05cfadf9a97b737597fe375cd45b7cf0df7b54031c77a45b7e7ae"
Sep 09 20:58:45.336566 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:45.336462    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-2d4edf-595d4b596f-swj8r"]
Sep 09 20:58:45.337043 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:45.336677    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-explicit-2d4edf-595d4b596f-swj8r" podUID="ed2eebef-e282-407b-8c6b-60277cca322a" containerName="payload-processing" containerID="cri-o://71bbdca5fc8a8609f05c4d3c2282d878563408385cd092fa55009f8309ee57ad" gracePeriod=30
Sep 09 20:58:45.345617 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:45.345439    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-2d4edf-756bd96bbb-29lsh"]
Sep 09 20:58:45.345702 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:45.345613    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-explicit-2d4edf-756bd96bbb-29lsh" podUID="4f0278e8-613e-43c3-ba18-733ba28577db" containerName="payload-pre-processing" containerID="cri-o://325ce7b78a2c408aa6c2a864d9464e6c3d6bd342931d8817e66819a5130be9cb" gracePeriod=30
Sep 09 20:58:45.480444 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:45.480410    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-e2f4649f-6c8555c6f7-ns4bw"
Sep 09 20:58:45.480626 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:45.480496    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-e2f4649f-657d677566-bq9lx"
Sep 09 20:58:45.550633 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:45.550590    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-2d4edf-l2lhw"]
Sep 09 20:58:45.556854 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:45.556822    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-2d4edf-l2lhw"]
Sep 09 20:58:45.565140 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:45.565082    2581 generic.go:356] "Generic (PLEG): container finished" podID="ed2eebef-e282-407b-8c6b-60277cca322a" containerID="71bbdca5fc8a8609f05c4d3c2282d878563408385cd092fa55009f8309ee57ad" exitCode=0
Sep 09 20:58:45.565299 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:45.565141    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-2d4edf-595d4b596f-swj8r" event={"ID":"ed2eebef-e282-407b-8c6b-60277cca322a","Type":"ContainerDied","Data":"71bbdca5fc8a8609f05c4d3c2282d878563408385cd092fa55009f8309ee57ad"}
Sep 09 20:58:45.567889 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:45.567866    2581 generic.go:356] "Generic (PLEG): container finished" podID="4f0278e8-613e-43c3-ba18-733ba28577db" containerID="325ce7b78a2c408aa6c2a864d9464e6c3d6bd342931d8817e66819a5130be9cb" exitCode=0
Sep 09 20:58:45.568073 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:45.567904    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-2d4edf-756bd96bbb-29lsh" event={"ID":"4f0278e8-613e-43c3-ba18-733ba28577db","Type":"ContainerDied","Data":"325ce7b78a2c408aa6c2a864d9464e6c3d6bd342931d8817e66819a5130be9cb"}
Sep 09 20:58:45.763792 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:45.763759    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-2d4edf-595d4b596f-swj8r"
Sep 09 20:58:45.766723 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:45.766702    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-2d4edf-756bd96bbb-29lsh"
Sep 09 20:58:45.901208 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:45.900258    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dkvzh\" (UniqueName: \"kubernetes.io/projected/4f0278e8-613e-43c3-ba18-733ba28577db-kube-api-access-dkvzh\") pod \"4f0278e8-613e-43c3-ba18-733ba28577db\" (UID: \"4f0278e8-613e-43c3-ba18-733ba28577db\") "
Sep 09 20:58:45.901208 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:45.900321    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed2eebef-e282-407b-8c6b-60277cca322a-plugins-config-volume\") pod \"ed2eebef-e282-407b-8c6b-60277cca322a\" (UID: \"ed2eebef-e282-407b-8c6b-60277cca322a\") "
Sep 09 20:58:45.901208 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:45.900412    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9lctq\" (UniqueName: \"kubernetes.io/projected/ed2eebef-e282-407b-8c6b-60277cca322a-kube-api-access-9lctq\") pod \"ed2eebef-e282-407b-8c6b-60277cca322a\" (UID: \"ed2eebef-e282-407b-8c6b-60277cca322a\") "
Sep 09 20:58:45.901208 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:45.900556    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f0278e8-613e-43c3-ba18-733ba28577db-plugins-config-volume\") pod \"4f0278e8-613e-43c3-ba18-733ba28577db\" (UID: \"4f0278e8-613e-43c3-ba18-733ba28577db\") "
Sep 09 20:58:45.920123 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:45.919796    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed2eebef-e282-407b-8c6b-60277cca322a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ed2eebef-e282-407b-8c6b-60277cca322a" (UID: "ed2eebef-e282-407b-8c6b-60277cca322a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:58:45.924120 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:45.920405    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f0278e8-613e-43c3-ba18-733ba28577db-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4f0278e8-613e-43c3-ba18-733ba28577db" (UID: "4f0278e8-613e-43c3-ba18-733ba28577db"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:58:45.924120 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:45.923749    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed2eebef-e282-407b-8c6b-60277cca322a-kube-api-access-9lctq" (OuterVolumeSpecName: "kube-api-access-9lctq") pod "ed2eebef-e282-407b-8c6b-60277cca322a" (UID: "ed2eebef-e282-407b-8c6b-60277cca322a"). InnerVolumeSpecName "kube-api-access-9lctq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:58:45.925439 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:45.925412    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f0278e8-613e-43c3-ba18-733ba28577db-kube-api-access-dkvzh" (OuterVolumeSpecName: "kube-api-access-dkvzh") pod "4f0278e8-613e-43c3-ba18-733ba28577db" (UID: "4f0278e8-613e-43c3-ba18-733ba28577db"). InnerVolumeSpecName "kube-api-access-dkvzh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:58:46.001800 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:46.001759    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9lctq\" (UniqueName: \"kubernetes.io/projected/ed2eebef-e282-407b-8c6b-60277cca322a-kube-api-access-9lctq\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:58:46.001800 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:46.001801    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f0278e8-613e-43c3-ba18-733ba28577db-plugins-config-volume\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:58:46.001994 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:46.001818    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dkvzh\" (UniqueName: \"kubernetes.io/projected/4f0278e8-613e-43c3-ba18-733ba28577db-kube-api-access-dkvzh\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:58:46.001994 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:46.001833    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed2eebef-e282-407b-8c6b-60277cca322a-plugins-config-volume\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:58:46.575661 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:46.575628    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-2d4edf-595d4b596f-swj8r"
Sep 09 20:58:46.575661 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:46.575634    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-2d4edf-595d4b596f-swj8r" event={"ID":"ed2eebef-e282-407b-8c6b-60277cca322a","Type":"ContainerDied","Data":"89d6d826f3e12b147970ebb34f6994addc3dd320adc4a0b24bfa63a0aa0ea13d"}
Sep 09 20:58:46.576232 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:46.575702    2581 scope.go:117] "RemoveContainer" containerID="71bbdca5fc8a8609f05c4d3c2282d878563408385cd092fa55009f8309ee57ad"
Sep 09 20:58:46.577288 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:46.577266    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-2d4edf-756bd96bbb-29lsh" event={"ID":"4f0278e8-613e-43c3-ba18-733ba28577db","Type":"ContainerDied","Data":"7ebc0618d578384a53887ecfeaa85e8b10065f782d199a70299e69ce6ee5b06e"}
Sep 09 20:58:46.577387 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:46.577275    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-2d4edf-756bd96bbb-29lsh"
Sep 09 20:58:46.585693 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:46.585675    2581 scope.go:117] "RemoveContainer" containerID="325ce7b78a2c408aa6c2a864d9464e6c3d6bd342931d8817e66819a5130be9cb"
Sep 09 20:58:46.599014 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:46.598986    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-2d4edf-595d4b596f-swj8r"]
Sep 09 20:58:46.610604 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:46.610519    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-2d4edf-595d4b596f-swj8r"]
Sep 09 20:58:46.625935 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:46.625907    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-2d4edf-756bd96bbb-29lsh"]
Sep 09 20:58:46.628800 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:46.628775    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-2d4edf-756bd96bbb-29lsh"]
Sep 09 20:58:46.825310 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:46.825275    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f0278e8-613e-43c3-ba18-733ba28577db" path="/var/lib/kubelet/pods/4f0278e8-613e-43c3-ba18-733ba28577db/volumes"
Sep 09 20:58:46.825847 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:46.825823    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed2eebef-e282-407b-8c6b-60277cca322a" path="/var/lib/kubelet/pods/ed2eebef-e282-407b-8c6b-60277cca322a/volumes"
Sep 09 20:58:46.826320 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:46.826299    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8fa2269-0606-42c2-8803-feab16d87374" path="/var/lib/kubelet/pods/f8fa2269-0606-42c2-8803-feab16d87374/volumes"
Sep 09 20:58:50.865413 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:50.865374    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-2d4edf/explicit-model-2d4edf-kserve-665bb8cb7-wf7sb"]
Sep 09 20:58:51.614801 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:51.614771    2581 generic.go:356] "Generic (PLEG): container finished" podID="2d1fc6e2-9473-40ed-b93a-dc1a3722d473" containerID="6d095799028f6da4228e0ddd522626f1941dea4f73570f2cd6fb2e198d0e5e69" exitCode=0
Sep 09 20:58:51.615009 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:51.614857    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h" event={"ID":"2d1fc6e2-9473-40ed-b93a-dc1a3722d473","Type":"ContainerDied","Data":"6d095799028f6da4228e0ddd522626f1941dea4f73570f2cd6fb2e198d0e5e69"}
Sep 09 20:58:51.615333 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:51.615318    2581 scope.go:117] "RemoveContainer" containerID="6d095799028f6da4228e0ddd522626f1941dea4f73570f2cd6fb2e198d0e5e69"
Sep 09 20:58:51.618931 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:51.617299    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:58:51.618931 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:51.617387    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:58:51.618931 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:51.617432    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:58:52.626614 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:52.626573    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h" event={"ID":"2d1fc6e2-9473-40ed-b93a-dc1a3722d473","Type":"ContainerStarted","Data":"ec8ac8149d4829c3a0ea63eef8d53c1335684a7d3a9b51f0a9c363f536dbad28"}
Sep 09 20:58:52.956750 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:52.956663    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h"
Sep 09 20:58:52.957990 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:52.957958    2581 patch_prober.go:28] interesting pod/e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Sep 09 20:58:52.958158 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:52.958011    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h" podUID="2d1fc6e2-9473-40ed-b93a-dc1a3722d473" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Sep 09 20:58:53.957225 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:53.957184    2581 patch_prober.go:28] interesting pod/e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Sep 09 20:58:53.957599 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:53.957242    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h" podUID="2d1fc6e2-9473-40ed-b93a-dc1a3722d473" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Sep 09 20:58:54.957286 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:54.957249    2581 patch_prober.go:28] interesting pod/e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Sep 09 20:58:54.957689 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:54.957308    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h" podUID="2d1fc6e2-9473-40ed-b93a-dc1a3722d473" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Sep 09 20:58:55.957824 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:55.957788    2581 patch_prober.go:28] interesting pod/e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Sep 09 20:58:55.958339 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:55.957844    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h" podUID="2d1fc6e2-9473-40ed-b93a-dc1a3722d473" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Sep 09 20:58:56.957471 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:56.957436    2581 patch_prober.go:28] interesting pod/e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Sep 09 20:58:56.957635 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:56.957491    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h" podUID="2d1fc6e2-9473-40ed-b93a-dc1a3722d473" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Sep 09 20:58:57.430277 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:57.430240    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h"
Sep 09 20:58:57.961833 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:57.961799    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h"
Sep 09 20:58:57.962702 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:58:57.962681    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h"
Sep 09 20:59:03.788603 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:03.788569    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e2f4649f-klhvn"]
Sep 09 20:59:03.789026 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:03.789013    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed2eebef-e282-407b-8c6b-60277cca322a" containerName="payload-processing"
Sep 09 20:59:03.789026 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:03.789027    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed2eebef-e282-407b-8c6b-60277cca322a" containerName="payload-processing"
Sep 09 20:59:03.789355 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:03.789050    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f0278e8-613e-43c3-ba18-733ba28577db" containerName="payload-pre-processing"
Sep 09 20:59:03.789355 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:03.789055    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f0278e8-613e-43c3-ba18-733ba28577db" containerName="payload-pre-processing"
Sep 09 20:59:03.789355 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:03.789063    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8fa2269-0606-42c2-8803-feab16d87374" containerName="revoke-keys"
Sep 09 20:59:03.789355 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:03.789069    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8fa2269-0606-42c2-8803-feab16d87374" containerName="revoke-keys"
Sep 09 20:59:03.789355 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:03.789144    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed2eebef-e282-407b-8c6b-60277cca322a" containerName="payload-processing"
Sep 09 20:59:03.789355 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:03.789153    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8fa2269-0606-42c2-8803-feab16d87374" containerName="revoke-keys"
Sep 09 20:59:03.789355 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:03.789165    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f0278e8-613e-43c3-ba18-733ba28577db" containerName="payload-pre-processing"
Sep 09 20:59:03.793545 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:03.793526    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e2f4649f-klhvn"
Sep 09 20:59:03.796404 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:03.796378    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-glz44\""
Sep 09 20:59:03.801377 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:03.800792    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e2f4649f-klhvn"]
Sep 09 20:59:03.808950 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:03.808923    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab1a6949-7d8c-4392-9642-526f0ff853ec-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-e2f4649f-klhvn\" (UID: \"ab1a6949-7d8c-4392-9642-526f0ff853ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e2f4649f-klhvn"
Sep 09 20:59:03.910858 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:03.910812    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab1a6949-7d8c-4392-9642-526f0ff853ec-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-e2f4649f-klhvn\" (UID: \"ab1a6949-7d8c-4392-9642-526f0ff853ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e2f4649f-klhvn"
Sep 09 20:59:03.911505 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:03.911480    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab1a6949-7d8c-4392-9642-526f0ff853ec-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-e2f4649f-klhvn\" (UID: \"ab1a6949-7d8c-4392-9642-526f0ff853ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e2f4649f-klhvn"
Sep 09 20:59:04.108490 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:04.108415    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e2f4649f-klhvn"
Sep 09 20:59:04.379140 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:04.379082    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e2f4649f-klhvn"]
Sep 09 20:59:04.380336 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:59:04.380308    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab1a6949_7d8c_4392_9642_526f0ff853ec.slice/crio-3131bdbfee441703bc4f355d68a9f1b52ff74d478570b1c5b5a288ca42603d49 WatchSource:0}: Error finding container 3131bdbfee441703bc4f355d68a9f1b52ff74d478570b1c5b5a288ca42603d49: Status 404 returned error can't find the container with id 3131bdbfee441703bc4f355d68a9f1b52ff74d478570b1c5b5a288ca42603d49
Sep 09 20:59:04.703274 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:04.703242    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e2f4649f-klhvn" event={"ID":"ab1a6949-7d8c-4392-9642-526f0ff853ec","Type":"ContainerStarted","Data":"3131bdbfee441703bc4f355d68a9f1b52ff74d478570b1c5b5a288ca42603d49"}
Sep 09 20:59:05.710006 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:05.709967    2581 generic.go:356] "Generic (PLEG): container finished" podID="ab1a6949-7d8c-4392-9642-526f0ff853ec" containerID="bb134a497ea012b3402dab57c753a990f8a215d8ff209b5442d7ae9d4652f437" exitCode=0
Sep 09 20:59:05.710412 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:05.710061    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e2f4649f-klhvn" event={"ID":"ab1a6949-7d8c-4392-9642-526f0ff853ec","Type":"ContainerDied","Data":"bb134a497ea012b3402dab57c753a990f8a215d8ff209b5442d7ae9d4652f437"}
Sep 09 20:59:05.885452 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:05.885407    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-explicit-2d4edf/explicit-model-2d4edf-kserve-665bb8cb7-wf7sb" podUID="027d2a28-eb79-405e-bef2-986ff736a808" containerName="main" containerID="cri-o://19a1dc62a84ec50b9b5555287675123eafc3f4aeac19f33b869f2523d31dbc47" gracePeriod=45
Sep 09 20:59:06.156373 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:06.156348    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-2d4edf/explicit-model-2d4edf-kserve-665bb8cb7-wf7sb"
Sep 09 20:59:06.238397 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:06.238375    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/027d2a28-eb79-405e-bef2-986ff736a808-dshm\") pod \"027d2a28-eb79-405e-bef2-986ff736a808\" (UID: \"027d2a28-eb79-405e-bef2-986ff736a808\") "
Sep 09 20:59:06.238542 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:06.238426    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/027d2a28-eb79-405e-bef2-986ff736a808-tls-certs\") pod \"027d2a28-eb79-405e-bef2-986ff736a808\" (UID: \"027d2a28-eb79-405e-bef2-986ff736a808\") "
Sep 09 20:59:06.238542 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:06.238459    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/027d2a28-eb79-405e-bef2-986ff736a808-home\") pod \"027d2a28-eb79-405e-bef2-986ff736a808\" (UID: \"027d2a28-eb79-405e-bef2-986ff736a808\") "
Sep 09 20:59:06.238542 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:06.238484    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/027d2a28-eb79-405e-bef2-986ff736a808-tmp-dir\") pod \"027d2a28-eb79-405e-bef2-986ff736a808\" (UID: \"027d2a28-eb79-405e-bef2-986ff736a808\") "
Sep 09 20:59:06.238542 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:06.238530    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/027d2a28-eb79-405e-bef2-986ff736a808-model-cache\") pod \"027d2a28-eb79-405e-bef2-986ff736a808\" (UID: \"027d2a28-eb79-405e-bef2-986ff736a808\") "
Sep 09 20:59:06.238720 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:06.238546    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8vphl\" (UniqueName: \"kubernetes.io/projected/027d2a28-eb79-405e-bef2-986ff736a808-kube-api-access-8vphl\") pod \"027d2a28-eb79-405e-bef2-986ff736a808\" (UID: \"027d2a28-eb79-405e-bef2-986ff736a808\") "
Sep 09 20:59:06.238791 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:06.238766    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/027d2a28-eb79-405e-bef2-986ff736a808-home" (OuterVolumeSpecName: "home") pod "027d2a28-eb79-405e-bef2-986ff736a808" (UID: "027d2a28-eb79-405e-bef2-986ff736a808"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:59:06.238791 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:06.238760    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/027d2a28-eb79-405e-bef2-986ff736a808-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "027d2a28-eb79-405e-bef2-986ff736a808" (UID: "027d2a28-eb79-405e-bef2-986ff736a808"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:59:06.238919 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:06.238795    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/027d2a28-eb79-405e-bef2-986ff736a808-model-cache" (OuterVolumeSpecName: "model-cache") pod "027d2a28-eb79-405e-bef2-986ff736a808" (UID: "027d2a28-eb79-405e-bef2-986ff736a808"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:59:06.238919 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:06.238900    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/027d2a28-eb79-405e-bef2-986ff736a808-home\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:59:06.238919 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:06.238915    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/027d2a28-eb79-405e-bef2-986ff736a808-tmp-dir\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:59:06.239031 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:06.238923    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/027d2a28-eb79-405e-bef2-986ff736a808-model-cache\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:59:06.240642 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:06.240616    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/027d2a28-eb79-405e-bef2-986ff736a808-dshm" (OuterVolumeSpecName: "dshm") pod "027d2a28-eb79-405e-bef2-986ff736a808" (UID: "027d2a28-eb79-405e-bef2-986ff736a808"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:59:06.240755 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:06.240665    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/027d2a28-eb79-405e-bef2-986ff736a808-kube-api-access-8vphl" (OuterVolumeSpecName: "kube-api-access-8vphl") pod "027d2a28-eb79-405e-bef2-986ff736a808" (UID: "027d2a28-eb79-405e-bef2-986ff736a808"). InnerVolumeSpecName "kube-api-access-8vphl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:59:06.240755 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:06.240673    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/027d2a28-eb79-405e-bef2-986ff736a808-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "027d2a28-eb79-405e-bef2-986ff736a808" (UID: "027d2a28-eb79-405e-bef2-986ff736a808"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 20:59:06.340231 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:06.340149    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8vphl\" (UniqueName: \"kubernetes.io/projected/027d2a28-eb79-405e-bef2-986ff736a808-kube-api-access-8vphl\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:59:06.340231 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:06.340178    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/027d2a28-eb79-405e-bef2-986ff736a808-dshm\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:59:06.340231 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:06.340188    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/027d2a28-eb79-405e-bef2-986ff736a808-tls-certs\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:59:06.716669 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:06.716585    2581 generic.go:356] "Generic (PLEG): container finished" podID="027d2a28-eb79-405e-bef2-986ff736a808" containerID="19a1dc62a84ec50b9b5555287675123eafc3f4aeac19f33b869f2523d31dbc47" exitCode=0
Sep 09 20:59:06.716669 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:06.716629    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-2d4edf/explicit-model-2d4edf-kserve-665bb8cb7-wf7sb"
Sep 09 20:59:06.717136 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:06.716671    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-2d4edf/explicit-model-2d4edf-kserve-665bb8cb7-wf7sb" event={"ID":"027d2a28-eb79-405e-bef2-986ff736a808","Type":"ContainerDied","Data":"19a1dc62a84ec50b9b5555287675123eafc3f4aeac19f33b869f2523d31dbc47"}
Sep 09 20:59:06.717136 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:06.716718    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-2d4edf/explicit-model-2d4edf-kserve-665bb8cb7-wf7sb" event={"ID":"027d2a28-eb79-405e-bef2-986ff736a808","Type":"ContainerDied","Data":"2190ed85bf2fcfe5be6d4847e1b504878c8fb750173d980663d5f69fe34666dc"}
Sep 09 20:59:06.717136 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:06.716737    2581 scope.go:117] "RemoveContainer" containerID="19a1dc62a84ec50b9b5555287675123eafc3f4aeac19f33b869f2523d31dbc47"
Sep 09 20:59:06.727927 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:06.727910    2581 scope.go:117] "RemoveContainer" containerID="19a1dc62a84ec50b9b5555287675123eafc3f4aeac19f33b869f2523d31dbc47"
Sep 09 20:59:06.728223 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:59:06.728204    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19a1dc62a84ec50b9b5555287675123eafc3f4aeac19f33b869f2523d31dbc47\": container with ID starting with 19a1dc62a84ec50b9b5555287675123eafc3f4aeac19f33b869f2523d31dbc47 not found: ID does not exist" containerID="19a1dc62a84ec50b9b5555287675123eafc3f4aeac19f33b869f2523d31dbc47"
Sep 09 20:59:06.728302 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:06.728230    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19a1dc62a84ec50b9b5555287675123eafc3f4aeac19f33b869f2523d31dbc47"} err="failed to get container status \"19a1dc62a84ec50b9b5555287675123eafc3f4aeac19f33b869f2523d31dbc47\": rpc error: code = NotFound desc = could not find container \"19a1dc62a84ec50b9b5555287675123eafc3f4aeac19f33b869f2523d31dbc47\": container with ID starting with 19a1dc62a84ec50b9b5555287675123eafc3f4aeac19f33b869f2523d31dbc47 not found: ID does not exist"
Sep 09 20:59:06.739907 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:06.739884    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-2d4edf/explicit-model-2d4edf-kserve-665bb8cb7-wf7sb"]
Sep 09 20:59:06.746335 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:06.745930    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-explicit-2d4edf/explicit-model-2d4edf-kserve-665bb8cb7-wf7sb"]
Sep 09 20:59:06.821883 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:06.821843    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="027d2a28-eb79-405e-bef2-986ff736a808" path="/var/lib/kubelet/pods/027d2a28-eb79-405e-bef2-986ff736a808/volumes"
Sep 09 20:59:06.837868 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:06.837844    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e2f4649f-klhvn"
Sep 09 20:59:06.946654 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:06.946623    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab1a6949-7d8c-4392-9642-526f0ff853ec-maas-api-service-ca\") pod \"ab1a6949-7d8c-4392-9642-526f0ff853ec\" (UID: \"ab1a6949-7d8c-4392-9642-526f0ff853ec\") "
Sep 09 20:59:06.946970 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:06.946948    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab1a6949-7d8c-4392-9642-526f0ff853ec-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ab1a6949-7d8c-4392-9642-526f0ff853ec" (UID: "ab1a6949-7d8c-4392-9642-526f0ff853ec"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:59:07.048396 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:07.048364    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab1a6949-7d8c-4392-9642-526f0ff853ec-maas-api-service-ca\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:59:07.724785 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:07.724753    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e2f4649f-klhvn" event={"ID":"ab1a6949-7d8c-4392-9642-526f0ff853ec","Type":"ContainerDied","Data":"3131bdbfee441703bc4f355d68a9f1b52ff74d478570b1c5b5a288ca42603d49"}
Sep 09 20:59:07.724785 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:07.724782    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e2f4649f-klhvn"
Sep 09 20:59:07.725245 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:07.724801    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3131bdbfee441703bc4f355d68a9f1b52ff74d478570b1c5b5a288ca42603d49"
Sep 09 20:59:14.342926 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:14.342887    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e2f4649f-6c8555c6f7-ns4bw"]
Sep 09 20:59:14.343429 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:14.343210    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-e2f4649f-6c8555c6f7-ns4bw" podUID="a602ad2d-daf0-418d-a4e8-841ff4a82438" containerName="payload-processing" containerID="cri-o://0974c8621941b948c53d838529093887af850548c79de42ac518abfb7a3b578f" gracePeriod=30
Sep 09 20:59:14.353724 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:14.353566    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e2f4649f-657d677566-bq9lx"]
Sep 09 20:59:14.354252 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:14.354163    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-e2f4649f-657d677566-bq9lx" podUID="ba3fb41e-2c50-4146-aeaf-063a3be8eeb1" containerName="payload-pre-processing" containerID="cri-o://26e9ef9b79ae370f0ebd34ab6168c12e30ec673529dba6d3ece1586223e1fd35" gracePeriod=30
Sep 09 20:59:14.520219 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:14.520188    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e2f4649f-klhvn"]
Sep 09 20:59:14.524288 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:14.524259    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e2f4649f-klhvn"]
Sep 09 20:59:14.624574 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:14.624548    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e2f4649f-6c8555c6f7-ns4bw"
Sep 09 20:59:14.675376 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:14.675345    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-e2f4649f-657d677566-bq9lx"
Sep 09 20:59:14.732287 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:14.732252    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dvrpj\" (UniqueName: \"kubernetes.io/projected/a602ad2d-daf0-418d-a4e8-841ff4a82438-kube-api-access-dvrpj\") pod \"a602ad2d-daf0-418d-a4e8-841ff4a82438\" (UID: \"a602ad2d-daf0-418d-a4e8-841ff4a82438\") "
Sep 09 20:59:14.732466 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:14.732351    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tx7d4\" (UniqueName: \"kubernetes.io/projected/ba3fb41e-2c50-4146-aeaf-063a3be8eeb1-kube-api-access-tx7d4\") pod \"ba3fb41e-2c50-4146-aeaf-063a3be8eeb1\" (UID: \"ba3fb41e-2c50-4146-aeaf-063a3be8eeb1\") "
Sep 09 20:59:14.732532 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:14.732494    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a602ad2d-daf0-418d-a4e8-841ff4a82438-plugins-config-volume\") pod \"a602ad2d-daf0-418d-a4e8-841ff4a82438\" (UID: \"a602ad2d-daf0-418d-a4e8-841ff4a82438\") "
Sep 09 20:59:14.732588 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:14.732545    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ba3fb41e-2c50-4146-aeaf-063a3be8eeb1-plugins-config-volume\") pod \"ba3fb41e-2c50-4146-aeaf-063a3be8eeb1\" (UID: \"ba3fb41e-2c50-4146-aeaf-063a3be8eeb1\") "
Sep 09 20:59:14.733036 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:14.733001    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a602ad2d-daf0-418d-a4e8-841ff4a82438-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a602ad2d-daf0-418d-a4e8-841ff4a82438" (UID: "a602ad2d-daf0-418d-a4e8-841ff4a82438"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:59:14.733036 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:14.733026    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba3fb41e-2c50-4146-aeaf-063a3be8eeb1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ba3fb41e-2c50-4146-aeaf-063a3be8eeb1" (UID: "ba3fb41e-2c50-4146-aeaf-063a3be8eeb1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:59:14.734564 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:14.734534    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba3fb41e-2c50-4146-aeaf-063a3be8eeb1-kube-api-access-tx7d4" (OuterVolumeSpecName: "kube-api-access-tx7d4") pod "ba3fb41e-2c50-4146-aeaf-063a3be8eeb1" (UID: "ba3fb41e-2c50-4146-aeaf-063a3be8eeb1"). InnerVolumeSpecName "kube-api-access-tx7d4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:59:14.735088 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:14.735063    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a602ad2d-daf0-418d-a4e8-841ff4a82438-kube-api-access-dvrpj" (OuterVolumeSpecName: "kube-api-access-dvrpj") pod "a602ad2d-daf0-418d-a4e8-841ff4a82438" (UID: "a602ad2d-daf0-418d-a4e8-841ff4a82438"). InnerVolumeSpecName "kube-api-access-dvrpj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:59:14.764871 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:14.764837    2581 generic.go:356] "Generic (PLEG): container finished" podID="a602ad2d-daf0-418d-a4e8-841ff4a82438" containerID="0974c8621941b948c53d838529093887af850548c79de42ac518abfb7a3b578f" exitCode=0
Sep 09 20:59:14.765037 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:14.764884    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e2f4649f-6c8555c6f7-ns4bw"
Sep 09 20:59:14.765037 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:14.764926    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e2f4649f-6c8555c6f7-ns4bw" event={"ID":"a602ad2d-daf0-418d-a4e8-841ff4a82438","Type":"ContainerDied","Data":"0974c8621941b948c53d838529093887af850548c79de42ac518abfb7a3b578f"}
Sep 09 20:59:14.765037 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:14.765016    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e2f4649f-6c8555c6f7-ns4bw" event={"ID":"a602ad2d-daf0-418d-a4e8-841ff4a82438","Type":"ContainerDied","Data":"0cce3cdf93ffac1f752e07c63b7639cb951116a4123970f524286dd2f9daac99"}
Sep 09 20:59:14.765237 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:14.765045    2581 scope.go:117] "RemoveContainer" containerID="0974c8621941b948c53d838529093887af850548c79de42ac518abfb7a3b578f"
Sep 09 20:59:14.766489 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:14.766466    2581 generic.go:356] "Generic (PLEG): container finished" podID="ba3fb41e-2c50-4146-aeaf-063a3be8eeb1" containerID="26e9ef9b79ae370f0ebd34ab6168c12e30ec673529dba6d3ece1586223e1fd35" exitCode=0
Sep 09 20:59:14.766598 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:14.766501    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-e2f4649f-657d677566-bq9lx"
Sep 09 20:59:14.766598 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:14.766535    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-e2f4649f-657d677566-bq9lx" event={"ID":"ba3fb41e-2c50-4146-aeaf-063a3be8eeb1","Type":"ContainerDied","Data":"26e9ef9b79ae370f0ebd34ab6168c12e30ec673529dba6d3ece1586223e1fd35"}
Sep 09 20:59:14.766598 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:14.766576    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-e2f4649f-657d677566-bq9lx" event={"ID":"ba3fb41e-2c50-4146-aeaf-063a3be8eeb1","Type":"ContainerDied","Data":"56d331b56cebbba3b4a57017a942421b743147fb6562f6987f522959109c58f8"}
Sep 09 20:59:14.776304 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:14.776286    2581 scope.go:117] "RemoveContainer" containerID="0974c8621941b948c53d838529093887af850548c79de42ac518abfb7a3b578f"
Sep 09 20:59:14.776727 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:59:14.776602    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0974c8621941b948c53d838529093887af850548c79de42ac518abfb7a3b578f\": container with ID starting with 0974c8621941b948c53d838529093887af850548c79de42ac518abfb7a3b578f not found: ID does not exist" containerID="0974c8621941b948c53d838529093887af850548c79de42ac518abfb7a3b578f"
Sep 09 20:59:14.776727 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:14.776635    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0974c8621941b948c53d838529093887af850548c79de42ac518abfb7a3b578f"} err="failed to get container status \"0974c8621941b948c53d838529093887af850548c79de42ac518abfb7a3b578f\": rpc error: code = NotFound desc = could not find container \"0974c8621941b948c53d838529093887af850548c79de42ac518abfb7a3b578f\": container with ID starting with 0974c8621941b948c53d838529093887af850548c79de42ac518abfb7a3b578f not found: ID does not exist"
Sep 09 20:59:14.776727 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:14.776659    2581 scope.go:117] "RemoveContainer" containerID="26e9ef9b79ae370f0ebd34ab6168c12e30ec673529dba6d3ece1586223e1fd35"
Sep 09 20:59:14.785402 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:14.785386    2581 scope.go:117] "RemoveContainer" containerID="26e9ef9b79ae370f0ebd34ab6168c12e30ec673529dba6d3ece1586223e1fd35"
Sep 09 20:59:14.785643 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:59:14.785625    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26e9ef9b79ae370f0ebd34ab6168c12e30ec673529dba6d3ece1586223e1fd35\": container with ID starting with 26e9ef9b79ae370f0ebd34ab6168c12e30ec673529dba6d3ece1586223e1fd35 not found: ID does not exist" containerID="26e9ef9b79ae370f0ebd34ab6168c12e30ec673529dba6d3ece1586223e1fd35"
Sep 09 20:59:14.785717 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:14.785649    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26e9ef9b79ae370f0ebd34ab6168c12e30ec673529dba6d3ece1586223e1fd35"} err="failed to get container status \"26e9ef9b79ae370f0ebd34ab6168c12e30ec673529dba6d3ece1586223e1fd35\": rpc error: code = NotFound desc = could not find container \"26e9ef9b79ae370f0ebd34ab6168c12e30ec673529dba6d3ece1586223e1fd35\": container with ID starting with 26e9ef9b79ae370f0ebd34ab6168c12e30ec673529dba6d3ece1586223e1fd35 not found: ID does not exist"
Sep 09 20:59:14.792936 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:14.792911    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e2f4649f-6c8555c6f7-ns4bw"]
Sep 09 20:59:14.797226 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:14.797201    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e2f4649f-6c8555c6f7-ns4bw"]
Sep 09 20:59:14.808930 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:14.808902    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e2f4649f-657d677566-bq9lx"]
Sep 09 20:59:14.811571 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:14.811551    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e2f4649f-657d677566-bq9lx"]
Sep 09 20:59:14.820461 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:14.820435    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a602ad2d-daf0-418d-a4e8-841ff4a82438" path="/var/lib/kubelet/pods/a602ad2d-daf0-418d-a4e8-841ff4a82438/volumes"
Sep 09 20:59:14.820870 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:14.820850    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab1a6949-7d8c-4392-9642-526f0ff853ec" path="/var/lib/kubelet/pods/ab1a6949-7d8c-4392-9642-526f0ff853ec/volumes"
Sep 09 20:59:14.821301 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:14.821283    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba3fb41e-2c50-4146-aeaf-063a3be8eeb1" path="/var/lib/kubelet/pods/ba3fb41e-2c50-4146-aeaf-063a3be8eeb1/volumes"
Sep 09 20:59:14.833959 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:14.833931    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ba3fb41e-2c50-4146-aeaf-063a3be8eeb1-plugins-config-volume\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:59:14.833959 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:14.833956    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dvrpj\" (UniqueName: \"kubernetes.io/projected/a602ad2d-daf0-418d-a4e8-841ff4a82438-kube-api-access-dvrpj\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:59:14.834084 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:14.833973    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tx7d4\" (UniqueName: \"kubernetes.io/projected/ba3fb41e-2c50-4146-aeaf-063a3be8eeb1-kube-api-access-tx7d4\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:59:14.834084 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:14.833987    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a602ad2d-daf0-418d-a4e8-841ff4a82438-plugins-config-volume\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:59:21.699183 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:21.699154    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h"]
Sep 09 20:59:21.699606 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:21.699367    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h" podUID="2d1fc6e2-9473-40ed-b93a-dc1a3722d473" containerName="istio-proxy" containerID="cri-o://ec8ac8149d4829c3a0ea63eef8d53c1335684a7d3a9b51f0a9c363f536dbad28" gracePeriod=30
Sep 09 20:59:26.843373 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:26.843341    2581 generic.go:356] "Generic (PLEG): container finished" podID="2d1fc6e2-9473-40ed-b93a-dc1a3722d473" containerID="ec8ac8149d4829c3a0ea63eef8d53c1335684a7d3a9b51f0a9c363f536dbad28" exitCode=0
Sep 09 20:59:26.843824 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:26.843401    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h" event={"ID":"2d1fc6e2-9473-40ed-b93a-dc1a3722d473","Type":"ContainerDied","Data":"ec8ac8149d4829c3a0ea63eef8d53c1335684a7d3a9b51f0a9c363f536dbad28"}
Sep 09 20:59:26.843824 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:26.843449    2581 scope.go:117] "RemoveContainer" containerID="6d095799028f6da4228e0ddd522626f1941dea4f73570f2cd6fb2e198d0e5e69"
Sep 09 20:59:26.954368 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:26.954341    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h"
Sep 09 20:59:27.061413 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:27.061372    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d1fc6e2-9473-40ed-b93a-dc1a3722d473-istiod-ca-cert\") pod \"2d1fc6e2-9473-40ed-b93a-dc1a3722d473\" (UID: \"2d1fc6e2-9473-40ed-b93a-dc1a3722d473\") "
Sep 09 20:59:27.061630 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:27.061449    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d1fc6e2-9473-40ed-b93a-dc1a3722d473-workload-socket\") pod \"2d1fc6e2-9473-40ed-b93a-dc1a3722d473\" (UID: \"2d1fc6e2-9473-40ed-b93a-dc1a3722d473\") "
Sep 09 20:59:27.061630 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:27.061475    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d1fc6e2-9473-40ed-b93a-dc1a3722d473-credential-socket\") pod \"2d1fc6e2-9473-40ed-b93a-dc1a3722d473\" (UID: \"2d1fc6e2-9473-40ed-b93a-dc1a3722d473\") "
Sep 09 20:59:27.061630 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:27.061507    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d1fc6e2-9473-40ed-b93a-dc1a3722d473-istio-envoy\") pod \"2d1fc6e2-9473-40ed-b93a-dc1a3722d473\" (UID: \"2d1fc6e2-9473-40ed-b93a-dc1a3722d473\") "
Sep 09 20:59:27.061630 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:27.061549    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d1fc6e2-9473-40ed-b93a-dc1a3722d473-istio-podinfo\") pod \"2d1fc6e2-9473-40ed-b93a-dc1a3722d473\" (UID: \"2d1fc6e2-9473-40ed-b93a-dc1a3722d473\") "
Sep 09 20:59:27.061630 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:27.061581    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l4l68\" (UniqueName: \"kubernetes.io/projected/2d1fc6e2-9473-40ed-b93a-dc1a3722d473-kube-api-access-l4l68\") pod \"2d1fc6e2-9473-40ed-b93a-dc1a3722d473\" (UID: \"2d1fc6e2-9473-40ed-b93a-dc1a3722d473\") "
Sep 09 20:59:27.061630 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:27.061631    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d1fc6e2-9473-40ed-b93a-dc1a3722d473-workload-certs\") pod \"2d1fc6e2-9473-40ed-b93a-dc1a3722d473\" (UID: \"2d1fc6e2-9473-40ed-b93a-dc1a3722d473\") "
Sep 09 20:59:27.061947 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:27.061659    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d1fc6e2-9473-40ed-b93a-dc1a3722d473-istio-data\") pod \"2d1fc6e2-9473-40ed-b93a-dc1a3722d473\" (UID: \"2d1fc6e2-9473-40ed-b93a-dc1a3722d473\") "
Sep 09 20:59:27.061947 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:27.061693    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d1fc6e2-9473-40ed-b93a-dc1a3722d473-istio-token\") pod \"2d1fc6e2-9473-40ed-b93a-dc1a3722d473\" (UID: \"2d1fc6e2-9473-40ed-b93a-dc1a3722d473\") "
Sep 09 20:59:27.061947 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:27.061755    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d1fc6e2-9473-40ed-b93a-dc1a3722d473-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2d1fc6e2-9473-40ed-b93a-dc1a3722d473" (UID: "2d1fc6e2-9473-40ed-b93a-dc1a3722d473"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:59:27.061947 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:27.061882    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d1fc6e2-9473-40ed-b93a-dc1a3722d473-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2d1fc6e2-9473-40ed-b93a-dc1a3722d473" (UID: "2d1fc6e2-9473-40ed-b93a-dc1a3722d473"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:59:27.061947 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:27.061913    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d1fc6e2-9473-40ed-b93a-dc1a3722d473-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2d1fc6e2-9473-40ed-b93a-dc1a3722d473" (UID: "2d1fc6e2-9473-40ed-b93a-dc1a3722d473"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:59:27.061947 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:27.061910    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d1fc6e2-9473-40ed-b93a-dc1a3722d473-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2d1fc6e2-9473-40ed-b93a-dc1a3722d473" (UID: "2d1fc6e2-9473-40ed-b93a-dc1a3722d473"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:59:27.062236 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:27.061991    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d1fc6e2-9473-40ed-b93a-dc1a3722d473-istio-data" (OuterVolumeSpecName: "istio-data") pod "2d1fc6e2-9473-40ed-b93a-dc1a3722d473" (UID: "2d1fc6e2-9473-40ed-b93a-dc1a3722d473"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:59:27.062236 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:27.062148    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d1fc6e2-9473-40ed-b93a-dc1a3722d473-workload-certs\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:59:27.062236 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:27.062175    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d1fc6e2-9473-40ed-b93a-dc1a3722d473-istio-data\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:59:27.062236 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:27.062191    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d1fc6e2-9473-40ed-b93a-dc1a3722d473-istiod-ca-cert\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:59:27.062236 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:27.062203    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d1fc6e2-9473-40ed-b93a-dc1a3722d473-workload-socket\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:59:27.062236 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:27.062218    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d1fc6e2-9473-40ed-b93a-dc1a3722d473-credential-socket\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:59:27.063822 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:27.063797    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2d1fc6e2-9473-40ed-b93a-dc1a3722d473-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2d1fc6e2-9473-40ed-b93a-dc1a3722d473" (UID: "2d1fc6e2-9473-40ed-b93a-dc1a3722d473"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 20:59:27.063949 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:27.063825    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d1fc6e2-9473-40ed-b93a-dc1a3722d473-kube-api-access-l4l68" (OuterVolumeSpecName: "kube-api-access-l4l68") pod "2d1fc6e2-9473-40ed-b93a-dc1a3722d473" (UID: "2d1fc6e2-9473-40ed-b93a-dc1a3722d473"). InnerVolumeSpecName "kube-api-access-l4l68". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:59:27.063949 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:27.063893    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d1fc6e2-9473-40ed-b93a-dc1a3722d473-istio-token" (OuterVolumeSpecName: "istio-token") pod "2d1fc6e2-9473-40ed-b93a-dc1a3722d473" (UID: "2d1fc6e2-9473-40ed-b93a-dc1a3722d473"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:59:27.063949 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:27.063914    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d1fc6e2-9473-40ed-b93a-dc1a3722d473-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2d1fc6e2-9473-40ed-b93a-dc1a3722d473" (UID: "2d1fc6e2-9473-40ed-b93a-dc1a3722d473"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:59:27.163438 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:27.163402    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d1fc6e2-9473-40ed-b93a-dc1a3722d473-istio-envoy\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:59:27.163438 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:27.163432    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d1fc6e2-9473-40ed-b93a-dc1a3722d473-istio-podinfo\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:59:27.163438 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:27.163441    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l4l68\" (UniqueName: \"kubernetes.io/projected/2d1fc6e2-9473-40ed-b93a-dc1a3722d473-kube-api-access-l4l68\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:59:27.163674 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:27.163449    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d1fc6e2-9473-40ed-b93a-dc1a3722d473-istio-token\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:59:27.850168 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:27.850141    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h"
Sep 09 20:59:27.850585 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:27.850181    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h" event={"ID":"2d1fc6e2-9473-40ed-b93a-dc1a3722d473","Type":"ContainerDied","Data":"a833d347c012648ff1888058a90fd950666d256ca6246dcebc89ddaefb1f1560"}
Sep 09 20:59:27.850585 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:27.850232    2581 scope.go:117] "RemoveContainer" containerID="ec8ac8149d4829c3a0ea63eef8d53c1335684a7d3a9b51f0a9c363f536dbad28"
Sep 09 20:59:27.876665 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:27.876641    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h"]
Sep 09 20:59:27.878470 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:27.878450    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-e2f4649f-openshift-default-77f867b948-8nf4h"]
Sep 09 20:59:28.235644 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:28.235611    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4"]
Sep 09 20:59:28.235858 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:28.235836    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4" podUID="d8184bd0-d46c-41e0-9e1d-11ae37508fcc" containerName="istio-proxy" containerID="cri-o://16ae552310fd1d4b83501d2eeaa0bcba3fb38203fb8a131aa3d84ec520226a24" gracePeriod=30
Sep 09 20:59:28.820715 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:28.820679    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d1fc6e2-9473-40ed-b93a-dc1a3722d473" path="/var/lib/kubelet/pods/2d1fc6e2-9473-40ed-b93a-dc1a3722d473/volumes"
Sep 09 20:59:29.052758 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:29.052729    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm"]
Sep 09 20:59:29.053189 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:29.053172    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d1fc6e2-9473-40ed-b93a-dc1a3722d473" containerName="istio-proxy"
Sep 09 20:59:29.053189 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:29.053190    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d1fc6e2-9473-40ed-b93a-dc1a3722d473" containerName="istio-proxy"
Sep 09 20:59:29.053275 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:29.053206    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="027d2a28-eb79-405e-bef2-986ff736a808" containerName="main"
Sep 09 20:59:29.053275 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:29.053212    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="027d2a28-eb79-405e-bef2-986ff736a808" containerName="main"
Sep 09 20:59:29.053275 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:29.053220    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba3fb41e-2c50-4146-aeaf-063a3be8eeb1" containerName="payload-pre-processing"
Sep 09 20:59:29.053275 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:29.053226    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba3fb41e-2c50-4146-aeaf-063a3be8eeb1" containerName="payload-pre-processing"
Sep 09 20:59:29.053275 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:29.053237    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab1a6949-7d8c-4392-9642-526f0ff853ec" containerName="revoke-keys"
Sep 09 20:59:29.053275 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:29.053241    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab1a6949-7d8c-4392-9642-526f0ff853ec" containerName="revoke-keys"
Sep 09 20:59:29.053275 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:29.053248    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a602ad2d-daf0-418d-a4e8-841ff4a82438" containerName="payload-processing"
Sep 09 20:59:29.053275 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:29.053254    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a602ad2d-daf0-418d-a4e8-841ff4a82438" containerName="payload-processing"
Sep 09 20:59:29.053538 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:29.053320    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="027d2a28-eb79-405e-bef2-986ff736a808" containerName="main"
Sep 09 20:59:29.053538 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:29.053327    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab1a6949-7d8c-4392-9642-526f0ff853ec" containerName="revoke-keys"
Sep 09 20:59:29.053538 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:29.053336    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba3fb41e-2c50-4146-aeaf-063a3be8eeb1" containerName="payload-pre-processing"
Sep 09 20:59:29.053538 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:29.053342    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d1fc6e2-9473-40ed-b93a-dc1a3722d473" containerName="istio-proxy"
Sep 09 20:59:29.053538 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:29.053348    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d1fc6e2-9473-40ed-b93a-dc1a3722d473" containerName="istio-proxy"
Sep 09 20:59:29.053538 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:29.053354    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a602ad2d-daf0-418d-a4e8-841ff4a82438" containerName="payload-processing"
Sep 09 20:59:29.057999 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:29.057980    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm"
Sep 09 20:59:29.060629 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:29.060609    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-noten-e7c6aa-openshift-default-dockercfg-5jgsc\""
Sep 09 20:59:29.068296 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:29.068275    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm"]
Sep 09 20:59:29.186511 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:29.186425    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fd3a041-c3a6-43cb-b02c-55edd6d4fbff-workload-socket\") pod \"e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm\" (UID: \"5fd3a041-c3a6-43cb-b02c-55edd6d4fbff\") " pod="openshift-ingress/e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm"
Sep 09 20:59:29.186511 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:29.186483    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5fd3a041-c3a6-43cb-b02c-55edd6d4fbff-istio-envoy\") pod \"e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm\" (UID: \"5fd3a041-c3a6-43cb-b02c-55edd6d4fbff\") " pod="openshift-ingress/e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm"
Sep 09 20:59:29.186748 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:29.186567    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fd3a041-c3a6-43cb-b02c-55edd6d4fbff-credential-socket\") pod \"e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm\" (UID: \"5fd3a041-c3a6-43cb-b02c-55edd6d4fbff\") " pod="openshift-ingress/e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm"
Sep 09 20:59:29.186748 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:29.186594    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56rrb\" (UniqueName: \"kubernetes.io/projected/5fd3a041-c3a6-43cb-b02c-55edd6d4fbff-kube-api-access-56rrb\") pod \"e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm\" (UID: \"5fd3a041-c3a6-43cb-b02c-55edd6d4fbff\") " pod="openshift-ingress/e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm"
Sep 09 20:59:29.186748 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:29.186613    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5fd3a041-c3a6-43cb-b02c-55edd6d4fbff-istio-podinfo\") pod \"e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm\" (UID: \"5fd3a041-c3a6-43cb-b02c-55edd6d4fbff\") " pod="openshift-ingress/e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm"
Sep 09 20:59:29.186748 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:29.186702    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5fd3a041-c3a6-43cb-b02c-55edd6d4fbff-istio-token\") pod \"e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm\" (UID: \"5fd3a041-c3a6-43cb-b02c-55edd6d4fbff\") " pod="openshift-ingress/e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm"
Sep 09 20:59:29.186884 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:29.186749    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5fd3a041-c3a6-43cb-b02c-55edd6d4fbff-workload-certs\") pod \"e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm\" (UID: \"5fd3a041-c3a6-43cb-b02c-55edd6d4fbff\") " pod="openshift-ingress/e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm"
Sep 09 20:59:29.186884 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:29.186796    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5fd3a041-c3a6-43cb-b02c-55edd6d4fbff-istio-data\") pod \"e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm\" (UID: \"5fd3a041-c3a6-43cb-b02c-55edd6d4fbff\") " pod="openshift-ingress/e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm"
Sep 09 20:59:29.186884 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:29.186822    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5fd3a041-c3a6-43cb-b02c-55edd6d4fbff-istiod-ca-cert\") pod \"e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm\" (UID: \"5fd3a041-c3a6-43cb-b02c-55edd6d4fbff\") " pod="openshift-ingress/e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm"
Sep 09 20:59:29.288192 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:29.288152    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fd3a041-c3a6-43cb-b02c-55edd6d4fbff-workload-socket\") pod \"e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm\" (UID: \"5fd3a041-c3a6-43cb-b02c-55edd6d4fbff\") " pod="openshift-ingress/e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm"
Sep 09 20:59:29.288392 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:29.288230    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5fd3a041-c3a6-43cb-b02c-55edd6d4fbff-istio-envoy\") pod \"e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm\" (UID: \"5fd3a041-c3a6-43cb-b02c-55edd6d4fbff\") " pod="openshift-ingress/e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm"
Sep 09 20:59:29.288392 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:29.288322    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fd3a041-c3a6-43cb-b02c-55edd6d4fbff-credential-socket\") pod \"e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm\" (UID: \"5fd3a041-c3a6-43cb-b02c-55edd6d4fbff\") " pod="openshift-ingress/e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm"
Sep 09 20:59:29.288392 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:29.288352    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56rrb\" (UniqueName: \"kubernetes.io/projected/5fd3a041-c3a6-43cb-b02c-55edd6d4fbff-kube-api-access-56rrb\") pod \"e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm\" (UID: \"5fd3a041-c3a6-43cb-b02c-55edd6d4fbff\") " pod="openshift-ingress/e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm"
Sep 09 20:59:29.288593 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:29.288419    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5fd3a041-c3a6-43cb-b02c-55edd6d4fbff-istio-podinfo\") pod \"e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm\" (UID: \"5fd3a041-c3a6-43cb-b02c-55edd6d4fbff\") " pod="openshift-ingress/e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm"
Sep 09 20:59:29.288593 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:29.288576    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fd3a041-c3a6-43cb-b02c-55edd6d4fbff-workload-socket\") pod \"e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm\" (UID: \"5fd3a041-c3a6-43cb-b02c-55edd6d4fbff\") " pod="openshift-ingress/e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm"
Sep 09 20:59:29.288715 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:29.288691    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fd3a041-c3a6-43cb-b02c-55edd6d4fbff-credential-socket\") pod \"e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm\" (UID: \"5fd3a041-c3a6-43cb-b02c-55edd6d4fbff\") " pod="openshift-ingress/e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm"
Sep 09 20:59:29.288785 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:29.288741    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5fd3a041-c3a6-43cb-b02c-55edd6d4fbff-istio-token\") pod \"e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm\" (UID: \"5fd3a041-c3a6-43cb-b02c-55edd6d4fbff\") " pod="openshift-ingress/e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm"
Sep 09 20:59:29.288849 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:29.288789    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5fd3a041-c3a6-43cb-b02c-55edd6d4fbff-workload-certs\") pod \"e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm\" (UID: \"5fd3a041-c3a6-43cb-b02c-55edd6d4fbff\") " pod="openshift-ingress/e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm"
Sep 09 20:59:29.288849 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:29.288829    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5fd3a041-c3a6-43cb-b02c-55edd6d4fbff-istio-data\") pod \"e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm\" (UID: \"5fd3a041-c3a6-43cb-b02c-55edd6d4fbff\") " pod="openshift-ingress/e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm"
Sep 09 20:59:29.288956 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:29.288865    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5fd3a041-c3a6-43cb-b02c-55edd6d4fbff-istiod-ca-cert\") pod \"e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm\" (UID: \"5fd3a041-c3a6-43cb-b02c-55edd6d4fbff\") " pod="openshift-ingress/e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm"
Sep 09 20:59:29.289229 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:29.289205    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5fd3a041-c3a6-43cb-b02c-55edd6d4fbff-istio-data\") pod \"e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm\" (UID: \"5fd3a041-c3a6-43cb-b02c-55edd6d4fbff\") " pod="openshift-ingress/e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm"
Sep 09 20:59:29.289439 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:29.289415    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5fd3a041-c3a6-43cb-b02c-55edd6d4fbff-workload-certs\") pod \"e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm\" (UID: \"5fd3a041-c3a6-43cb-b02c-55edd6d4fbff\") " pod="openshift-ingress/e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm"
Sep 09 20:59:29.289795 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:29.289773    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5fd3a041-c3a6-43cb-b02c-55edd6d4fbff-istiod-ca-cert\") pod \"e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm\" (UID: \"5fd3a041-c3a6-43cb-b02c-55edd6d4fbff\") " pod="openshift-ingress/e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm"
Sep 09 20:59:29.290839 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:29.290818    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5fd3a041-c3a6-43cb-b02c-55edd6d4fbff-istio-envoy\") pod \"e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm\" (UID: \"5fd3a041-c3a6-43cb-b02c-55edd6d4fbff\") " pod="openshift-ingress/e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm"
Sep 09 20:59:29.291043 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:29.291027    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5fd3a041-c3a6-43cb-b02c-55edd6d4fbff-istio-podinfo\") pod \"e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm\" (UID: \"5fd3a041-c3a6-43cb-b02c-55edd6d4fbff\") " pod="openshift-ingress/e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm"
Sep 09 20:59:29.298759 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:29.298734    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5fd3a041-c3a6-43cb-b02c-55edd6d4fbff-istio-token\") pod \"e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm\" (UID: \"5fd3a041-c3a6-43cb-b02c-55edd6d4fbff\") " pod="openshift-ingress/e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm"
Sep 09 20:59:29.299331 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:29.299312    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-56rrb\" (UniqueName: \"kubernetes.io/projected/5fd3a041-c3a6-43cb-b02c-55edd6d4fbff-kube-api-access-56rrb\") pod \"e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm\" (UID: \"5fd3a041-c3a6-43cb-b02c-55edd6d4fbff\") " pod="openshift-ingress/e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm"
Sep 09 20:59:29.369085 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:29.369049    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm"
Sep 09 20:59:29.656560 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:29.656477    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm"]
Sep 09 20:59:29.659660 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:59:29.659634    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fd3a041_c3a6_43cb_b02c_55edd6d4fbff.slice/crio-e3de29c12a20e834b048db16756b560899409049d044cec5b321d7d2c3347f5f WatchSource:0}: Error finding container e3de29c12a20e834b048db16756b560899409049d044cec5b321d7d2c3347f5f: Status 404 returned error can't find the container with id e3de29c12a20e834b048db16756b560899409049d044cec5b321d7d2c3347f5f
Sep 09 20:59:29.661895 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:29.661860    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:59:29.662004 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:29.661940    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:59:29.662004 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:29.661988    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:59:29.863650 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:29.863613    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm" event={"ID":"5fd3a041-c3a6-43cb-b02c-55edd6d4fbff","Type":"ContainerStarted","Data":"726838450e3a0216431d27dfb59da3a0caef7e5b8437113163ac7d40e44575fa"}
Sep 09 20:59:29.863835 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:29.863669    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm" event={"ID":"5fd3a041-c3a6-43cb-b02c-55edd6d4fbff","Type":"ContainerStarted","Data":"e3de29c12a20e834b048db16756b560899409049d044cec5b321d7d2c3347f5f"}
Sep 09 20:59:29.886130 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:29.886053    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm" podStartSLOduration=0.886038449 podStartE2EDuration="886.038449ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:59:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:59:29.884436418 +0000 UTC m=+1131.512064306" watchObservedRunningTime="2026-09-09 20:59:29.886038449 +0000 UTC m=+1131.513666375"
Sep 09 20:59:30.370443 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:30.370409    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm"
Sep 09 20:59:30.375736 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:30.375713    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm"
Sep 09 20:59:30.869238 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:30.869211    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm"
Sep 09 20:59:30.870058 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:30.870038    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm"
Sep 09 20:59:32.173241 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:32.173207    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-47e41c/distinct-2-w4-47e41c-kserve-67c87c7696-slxnj"]
Sep 09 20:59:32.173700 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:32.173685    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d1fc6e2-9473-40ed-b93a-dc1a3722d473" containerName="istio-proxy"
Sep 09 20:59:32.173748 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:32.173702    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d1fc6e2-9473-40ed-b93a-dc1a3722d473" containerName="istio-proxy"
Sep 09 20:59:32.177086 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:32.177066    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-47e41c/distinct-2-w4-47e41c-kserve-67c87c7696-slxnj"
Sep 09 20:59:32.179707 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:32.179684    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-47e41c\"/\"distinct-2-w4-47e41c-kserve-self-signed-certs\""
Sep 09 20:59:32.187930 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:32.187908    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-47e41c/distinct-2-w4-47e41c-kserve-67c87c7696-slxnj"]
Sep 09 20:59:32.216247 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:32.216221    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6500905e-d7dd-4d93-bfd9-c92b320c2817-tls-certs\") pod \"distinct-2-w4-47e41c-kserve-67c87c7696-slxnj\" (UID: \"6500905e-d7dd-4d93-bfd9-c92b320c2817\") " pod="e2e-models-e2e-worker-w4-47e41c/distinct-2-w4-47e41c-kserve-67c87c7696-slxnj"
Sep 09 20:59:32.216368 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:32.216315    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6500905e-d7dd-4d93-bfd9-c92b320c2817-dshm\") pod \"distinct-2-w4-47e41c-kserve-67c87c7696-slxnj\" (UID: \"6500905e-d7dd-4d93-bfd9-c92b320c2817\") " pod="e2e-models-e2e-worker-w4-47e41c/distinct-2-w4-47e41c-kserve-67c87c7696-slxnj"
Sep 09 20:59:32.216438 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:32.216374    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drwd9\" (UniqueName: \"kubernetes.io/projected/6500905e-d7dd-4d93-bfd9-c92b320c2817-kube-api-access-drwd9\") pod \"distinct-2-w4-47e41c-kserve-67c87c7696-slxnj\" (UID: \"6500905e-d7dd-4d93-bfd9-c92b320c2817\") " pod="e2e-models-e2e-worker-w4-47e41c/distinct-2-w4-47e41c-kserve-67c87c7696-slxnj"
Sep 09 20:59:32.216483 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:32.216460    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6500905e-d7dd-4d93-bfd9-c92b320c2817-model-cache\") pod \"distinct-2-w4-47e41c-kserve-67c87c7696-slxnj\" (UID: \"6500905e-d7dd-4d93-bfd9-c92b320c2817\") " pod="e2e-models-e2e-worker-w4-47e41c/distinct-2-w4-47e41c-kserve-67c87c7696-slxnj"
Sep 09 20:59:32.216527 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:32.216483    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6500905e-d7dd-4d93-bfd9-c92b320c2817-tmp-dir\") pod \"distinct-2-w4-47e41c-kserve-67c87c7696-slxnj\" (UID: \"6500905e-d7dd-4d93-bfd9-c92b320c2817\") " pod="e2e-models-e2e-worker-w4-47e41c/distinct-2-w4-47e41c-kserve-67c87c7696-slxnj"
Sep 09 20:59:32.216527 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:32.216507    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6500905e-d7dd-4d93-bfd9-c92b320c2817-home\") pod \"distinct-2-w4-47e41c-kserve-67c87c7696-slxnj\" (UID: \"6500905e-d7dd-4d93-bfd9-c92b320c2817\") " pod="e2e-models-e2e-worker-w4-47e41c/distinct-2-w4-47e41c-kserve-67c87c7696-slxnj"
Sep 09 20:59:32.317517 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:32.317485    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6500905e-d7dd-4d93-bfd9-c92b320c2817-model-cache\") pod \"distinct-2-w4-47e41c-kserve-67c87c7696-slxnj\" (UID: \"6500905e-d7dd-4d93-bfd9-c92b320c2817\") " pod="e2e-models-e2e-worker-w4-47e41c/distinct-2-w4-47e41c-kserve-67c87c7696-slxnj"
Sep 09 20:59:32.317517 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:32.317521    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6500905e-d7dd-4d93-bfd9-c92b320c2817-tmp-dir\") pod \"distinct-2-w4-47e41c-kserve-67c87c7696-slxnj\" (UID: \"6500905e-d7dd-4d93-bfd9-c92b320c2817\") " pod="e2e-models-e2e-worker-w4-47e41c/distinct-2-w4-47e41c-kserve-67c87c7696-slxnj"
Sep 09 20:59:32.317704 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:32.317542    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6500905e-d7dd-4d93-bfd9-c92b320c2817-home\") pod \"distinct-2-w4-47e41c-kserve-67c87c7696-slxnj\" (UID: \"6500905e-d7dd-4d93-bfd9-c92b320c2817\") " pod="e2e-models-e2e-worker-w4-47e41c/distinct-2-w4-47e41c-kserve-67c87c7696-slxnj"
Sep 09 20:59:32.317704 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:32.317570    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6500905e-d7dd-4d93-bfd9-c92b320c2817-tls-certs\") pod \"distinct-2-w4-47e41c-kserve-67c87c7696-slxnj\" (UID: \"6500905e-d7dd-4d93-bfd9-c92b320c2817\") " pod="e2e-models-e2e-worker-w4-47e41c/distinct-2-w4-47e41c-kserve-67c87c7696-slxnj"
Sep 09 20:59:32.317704 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:32.317605    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6500905e-d7dd-4d93-bfd9-c92b320c2817-dshm\") pod \"distinct-2-w4-47e41c-kserve-67c87c7696-slxnj\" (UID: \"6500905e-d7dd-4d93-bfd9-c92b320c2817\") " pod="e2e-models-e2e-worker-w4-47e41c/distinct-2-w4-47e41c-kserve-67c87c7696-slxnj"
Sep 09 20:59:32.317704 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:32.317634    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drwd9\" (UniqueName: \"kubernetes.io/projected/6500905e-d7dd-4d93-bfd9-c92b320c2817-kube-api-access-drwd9\") pod \"distinct-2-w4-47e41c-kserve-67c87c7696-slxnj\" (UID: \"6500905e-d7dd-4d93-bfd9-c92b320c2817\") " pod="e2e-models-e2e-worker-w4-47e41c/distinct-2-w4-47e41c-kserve-67c87c7696-slxnj"
Sep 09 20:59:32.317934 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:32.317858    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6500905e-d7dd-4d93-bfd9-c92b320c2817-tmp-dir\") pod \"distinct-2-w4-47e41c-kserve-67c87c7696-slxnj\" (UID: \"6500905e-d7dd-4d93-bfd9-c92b320c2817\") " pod="e2e-models-e2e-worker-w4-47e41c/distinct-2-w4-47e41c-kserve-67c87c7696-slxnj"
Sep 09 20:59:32.318005 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:32.317931    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6500905e-d7dd-4d93-bfd9-c92b320c2817-home\") pod \"distinct-2-w4-47e41c-kserve-67c87c7696-slxnj\" (UID: \"6500905e-d7dd-4d93-bfd9-c92b320c2817\") " pod="e2e-models-e2e-worker-w4-47e41c/distinct-2-w4-47e41c-kserve-67c87c7696-slxnj"
Sep 09 20:59:32.318005 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:32.317950    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6500905e-d7dd-4d93-bfd9-c92b320c2817-model-cache\") pod \"distinct-2-w4-47e41c-kserve-67c87c7696-slxnj\" (UID: \"6500905e-d7dd-4d93-bfd9-c92b320c2817\") " pod="e2e-models-e2e-worker-w4-47e41c/distinct-2-w4-47e41c-kserve-67c87c7696-slxnj"
Sep 09 20:59:32.319816 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:32.319795    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6500905e-d7dd-4d93-bfd9-c92b320c2817-dshm\") pod \"distinct-2-w4-47e41c-kserve-67c87c7696-slxnj\" (UID: \"6500905e-d7dd-4d93-bfd9-c92b320c2817\") " pod="e2e-models-e2e-worker-w4-47e41c/distinct-2-w4-47e41c-kserve-67c87c7696-slxnj"
Sep 09 20:59:32.319977 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:32.319960    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6500905e-d7dd-4d93-bfd9-c92b320c2817-tls-certs\") pod \"distinct-2-w4-47e41c-kserve-67c87c7696-slxnj\" (UID: \"6500905e-d7dd-4d93-bfd9-c92b320c2817\") " pod="e2e-models-e2e-worker-w4-47e41c/distinct-2-w4-47e41c-kserve-67c87c7696-slxnj"
Sep 09 20:59:32.325516 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:32.325497    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-drwd9\" (UniqueName: \"kubernetes.io/projected/6500905e-d7dd-4d93-bfd9-c92b320c2817-kube-api-access-drwd9\") pod \"distinct-2-w4-47e41c-kserve-67c87c7696-slxnj\" (UID: \"6500905e-d7dd-4d93-bfd9-c92b320c2817\") " pod="e2e-models-e2e-worker-w4-47e41c/distinct-2-w4-47e41c-kserve-67c87c7696-slxnj"
Sep 09 20:59:32.487079 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:32.487037    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-47e41c/distinct-2-w4-47e41c-kserve-67c87c7696-slxnj"
Sep 09 20:59:32.758803 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:32.758777    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-47e41c/distinct-2-w4-47e41c-kserve-67c87c7696-slxnj"]
Sep 09 20:59:32.761079 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:59:32.761051    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6500905e_d7dd_4d93_bfd9_c92b320c2817.slice/crio-059f6562a2efad9a8532ac78e9beb2d5533c9df1c23d353e007f0117d17301f7 WatchSource:0}: Error finding container 059f6562a2efad9a8532ac78e9beb2d5533c9df1c23d353e007f0117d17301f7: Status 404 returned error can't find the container with id 059f6562a2efad9a8532ac78e9beb2d5533c9df1c23d353e007f0117d17301f7
Sep 09 20:59:32.884065 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:32.884027    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-47e41c/distinct-2-w4-47e41c-kserve-67c87c7696-slxnj" event={"ID":"6500905e-d7dd-4d93-bfd9-c92b320c2817","Type":"ContainerStarted","Data":"63dad889e0e472d47c31644a35057577053bbbedb89a8601b152f1de881ad359"}
Sep 09 20:59:32.884233 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:32.884075    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-47e41c/distinct-2-w4-47e41c-kserve-67c87c7696-slxnj" event={"ID":"6500905e-d7dd-4d93-bfd9-c92b320c2817","Type":"ContainerStarted","Data":"059f6562a2efad9a8532ac78e9beb2d5533c9df1c23d353e007f0117d17301f7"}
Sep 09 20:59:32.901052 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:32.901001    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-47e41c/distinct-2-w4-47e41c-kserve-67c87c7696-slxnj" podStartSLOduration=0.900984506 podStartE2EDuration="900.984506ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:59:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:59:32.900785675 +0000 UTC m=+1134.528413553" watchObservedRunningTime="2026-09-09 20:59:32.900984506 +0000 UTC m=+1134.528612402"
Sep 09 20:59:33.484770 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:33.484745    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4"
Sep 09 20:59:33.527552 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:33.527457    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d8184bd0-d46c-41e0-9e1d-11ae37508fcc-istiod-ca-cert\") pod \"d8184bd0-d46c-41e0-9e1d-11ae37508fcc\" (UID: \"d8184bd0-d46c-41e0-9e1d-11ae37508fcc\") "
Sep 09 20:59:33.527552 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:33.527531    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d8184bd0-d46c-41e0-9e1d-11ae37508fcc-istio-data\") pod \"d8184bd0-d46c-41e0-9e1d-11ae37508fcc\" (UID: \"d8184bd0-d46c-41e0-9e1d-11ae37508fcc\") "
Sep 09 20:59:33.527787 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:33.527562    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d8184bd0-d46c-41e0-9e1d-11ae37508fcc-istio-podinfo\") pod \"d8184bd0-d46c-41e0-9e1d-11ae37508fcc\" (UID: \"d8184bd0-d46c-41e0-9e1d-11ae37508fcc\") "
Sep 09 20:59:33.527787 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:33.527604    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d8184bd0-d46c-41e0-9e1d-11ae37508fcc-istio-envoy\") pod \"d8184bd0-d46c-41e0-9e1d-11ae37508fcc\" (UID: \"d8184bd0-d46c-41e0-9e1d-11ae37508fcc\") "
Sep 09 20:59:33.527787 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:33.527683    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d8184bd0-d46c-41e0-9e1d-11ae37508fcc-istio-token\") pod \"d8184bd0-d46c-41e0-9e1d-11ae37508fcc\" (UID: \"d8184bd0-d46c-41e0-9e1d-11ae37508fcc\") "
Sep 09 20:59:33.527787 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:33.527712    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8184bd0-d46c-41e0-9e1d-11ae37508fcc-workload-socket\") pod \"d8184bd0-d46c-41e0-9e1d-11ae37508fcc\" (UID: \"d8184bd0-d46c-41e0-9e1d-11ae37508fcc\") "
Sep 09 20:59:33.527787 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:33.527745    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jkfqr\" (UniqueName: \"kubernetes.io/projected/d8184bd0-d46c-41e0-9e1d-11ae37508fcc-kube-api-access-jkfqr\") pod \"d8184bd0-d46c-41e0-9e1d-11ae37508fcc\" (UID: \"d8184bd0-d46c-41e0-9e1d-11ae37508fcc\") "
Sep 09 20:59:33.528053 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:33.527798    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d8184bd0-d46c-41e0-9e1d-11ae37508fcc-workload-certs\") pod \"d8184bd0-d46c-41e0-9e1d-11ae37508fcc\" (UID: \"d8184bd0-d46c-41e0-9e1d-11ae37508fcc\") "
Sep 09 20:59:33.528053 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:33.527852    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8184bd0-d46c-41e0-9e1d-11ae37508fcc-credential-socket\") pod \"d8184bd0-d46c-41e0-9e1d-11ae37508fcc\" (UID: \"d8184bd0-d46c-41e0-9e1d-11ae37508fcc\") "
Sep 09 20:59:33.528053 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:33.527873    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8184bd0-d46c-41e0-9e1d-11ae37508fcc-istio-data" (OuterVolumeSpecName: "istio-data") pod "d8184bd0-d46c-41e0-9e1d-11ae37508fcc" (UID: "d8184bd0-d46c-41e0-9e1d-11ae37508fcc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:59:33.528053 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:33.527907    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8184bd0-d46c-41e0-9e1d-11ae37508fcc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d8184bd0-d46c-41e0-9e1d-11ae37508fcc" (UID: "d8184bd0-d46c-41e0-9e1d-11ae37508fcc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:59:33.528303 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:33.528053    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8184bd0-d46c-41e0-9e1d-11ae37508fcc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d8184bd0-d46c-41e0-9e1d-11ae37508fcc" (UID: "d8184bd0-d46c-41e0-9e1d-11ae37508fcc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:59:33.528303 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:33.528256    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8184bd0-d46c-41e0-9e1d-11ae37508fcc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d8184bd0-d46c-41e0-9e1d-11ae37508fcc" (UID: "d8184bd0-d46c-41e0-9e1d-11ae37508fcc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:59:33.528482 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:33.528450    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8184bd0-d46c-41e0-9e1d-11ae37508fcc-credential-socket\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:59:33.528482 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:33.528460    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8184bd0-d46c-41e0-9e1d-11ae37508fcc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d8184bd0-d46c-41e0-9e1d-11ae37508fcc" (UID: "d8184bd0-d46c-41e0-9e1d-11ae37508fcc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:59:33.528482 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:33.528474    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d8184bd0-d46c-41e0-9e1d-11ae37508fcc-istiod-ca-cert\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:59:33.528620 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:33.528489    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d8184bd0-d46c-41e0-9e1d-11ae37508fcc-istio-data\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:59:33.528620 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:33.528502    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8184bd0-d46c-41e0-9e1d-11ae37508fcc-workload-socket\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:59:33.530468 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:33.530427    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8184bd0-d46c-41e0-9e1d-11ae37508fcc-istio-token" (OuterVolumeSpecName: "istio-token") pod "d8184bd0-d46c-41e0-9e1d-11ae37508fcc" (UID: "d8184bd0-d46c-41e0-9e1d-11ae37508fcc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:59:33.530574 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:33.530533    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8184bd0-d46c-41e0-9e1d-11ae37508fcc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d8184bd0-d46c-41e0-9e1d-11ae37508fcc" (UID: "d8184bd0-d46c-41e0-9e1d-11ae37508fcc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:59:33.530574 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:33.530536    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8184bd0-d46c-41e0-9e1d-11ae37508fcc-kube-api-access-jkfqr" (OuterVolumeSpecName: "kube-api-access-jkfqr") pod "d8184bd0-d46c-41e0-9e1d-11ae37508fcc" (UID: "d8184bd0-d46c-41e0-9e1d-11ae37508fcc"). InnerVolumeSpecName "kube-api-access-jkfqr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:59:33.530912 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:33.530891    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d8184bd0-d46c-41e0-9e1d-11ae37508fcc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d8184bd0-d46c-41e0-9e1d-11ae37508fcc" (UID: "d8184bd0-d46c-41e0-9e1d-11ae37508fcc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 20:59:33.629665 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:33.629628    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d8184bd0-d46c-41e0-9e1d-11ae37508fcc-istio-podinfo\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:59:33.629665 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:33.629660    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d8184bd0-d46c-41e0-9e1d-11ae37508fcc-istio-envoy\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:59:33.629665 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:33.629668    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d8184bd0-d46c-41e0-9e1d-11ae37508fcc-istio-token\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:59:33.629665 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:33.629676    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jkfqr\" (UniqueName: \"kubernetes.io/projected/d8184bd0-d46c-41e0-9e1d-11ae37508fcc-kube-api-access-jkfqr\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:59:33.629927 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:33.629685    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d8184bd0-d46c-41e0-9e1d-11ae37508fcc-workload-certs\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 20:59:33.891289 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:33.891206    2581 generic.go:356] "Generic (PLEG): container finished" podID="d8184bd0-d46c-41e0-9e1d-11ae37508fcc" containerID="16ae552310fd1d4b83501d2eeaa0bcba3fb38203fb8a131aa3d84ec520226a24" exitCode=0
Sep 09 20:59:33.891289 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:33.891248    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4"
Sep 09 20:59:33.891473 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:33.891293    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4" event={"ID":"d8184bd0-d46c-41e0-9e1d-11ae37508fcc","Type":"ContainerDied","Data":"16ae552310fd1d4b83501d2eeaa0bcba3fb38203fb8a131aa3d84ec520226a24"}
Sep 09 20:59:33.891473 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:33.891343    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4" event={"ID":"d8184bd0-d46c-41e0-9e1d-11ae37508fcc","Type":"ContainerDied","Data":"83cce5558d40c908c04127122993c28f4f31a7d0883118f00891d6d3ade383b7"}
Sep 09 20:59:33.891473 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:33.891366    2581 scope.go:117] "RemoveContainer" containerID="16ae552310fd1d4b83501d2eeaa0bcba3fb38203fb8a131aa3d84ec520226a24"
Sep 09 20:59:33.903731 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:33.903705    2581 scope.go:117] "RemoveContainer" containerID="e88f4843a1aaf0e61ac2131e4fcaf8d0df367b7ebaca415297c47280ccfb0e07"
Sep 09 20:59:33.912798 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:33.912781    2581 scope.go:117] "RemoveContainer" containerID="16ae552310fd1d4b83501d2eeaa0bcba3fb38203fb8a131aa3d84ec520226a24"
Sep 09 20:59:33.913061 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:59:33.913037    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"16ae552310fd1d4b83501d2eeaa0bcba3fb38203fb8a131aa3d84ec520226a24\": container with ID starting with 16ae552310fd1d4b83501d2eeaa0bcba3fb38203fb8a131aa3d84ec520226a24 not found: ID does not exist" containerID="16ae552310fd1d4b83501d2eeaa0bcba3fb38203fb8a131aa3d84ec520226a24"
Sep 09 20:59:33.913194 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:33.913067    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16ae552310fd1d4b83501d2eeaa0bcba3fb38203fb8a131aa3d84ec520226a24"} err="failed to get container status \"16ae552310fd1d4b83501d2eeaa0bcba3fb38203fb8a131aa3d84ec520226a24\": rpc error: code = NotFound desc = could not find container \"16ae552310fd1d4b83501d2eeaa0bcba3fb38203fb8a131aa3d84ec520226a24\": container with ID starting with 16ae552310fd1d4b83501d2eeaa0bcba3fb38203fb8a131aa3d84ec520226a24 not found: ID does not exist"
Sep 09 20:59:33.913194 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:33.913084    2581 scope.go:117] "RemoveContainer" containerID="e88f4843a1aaf0e61ac2131e4fcaf8d0df367b7ebaca415297c47280ccfb0e07"
Sep 09 20:59:33.913364 ip-10-0-129-99 kubenswrapper[2581]: E0909 20:59:33.913343    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e88f4843a1aaf0e61ac2131e4fcaf8d0df367b7ebaca415297c47280ccfb0e07\": container with ID starting with e88f4843a1aaf0e61ac2131e4fcaf8d0df367b7ebaca415297c47280ccfb0e07 not found: ID does not exist" containerID="e88f4843a1aaf0e61ac2131e4fcaf8d0df367b7ebaca415297c47280ccfb0e07"
Sep 09 20:59:33.913421 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:33.913372    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e88f4843a1aaf0e61ac2131e4fcaf8d0df367b7ebaca415297c47280ccfb0e07"} err="failed to get container status \"e88f4843a1aaf0e61ac2131e4fcaf8d0df367b7ebaca415297c47280ccfb0e07\": rpc error: code = NotFound desc = could not find container \"e88f4843a1aaf0e61ac2131e4fcaf8d0df367b7ebaca415297c47280ccfb0e07\": container with ID starting with e88f4843a1aaf0e61ac2131e4fcaf8d0df367b7ebaca415297c47280ccfb0e07 not found: ID does not exist"
Sep 09 20:59:33.919270 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:33.919251    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4"]
Sep 09 20:59:33.925957 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:33.925938    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-explicit-2d4edf-openshift-default-545bcf5bd4-sgmm4"]
Sep 09 20:59:34.820193 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:34.820164    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d8184bd0-d46c-41e0-9e1d-11ae37508fcc" path="/var/lib/kubelet/pods/d8184bd0-d46c-41e0-9e1d-11ae37508fcc/volumes"
Sep 09 20:59:36.122928 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:36.122891    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9"]
Sep 09 20:59:36.123378 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:36.123364    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8184bd0-d46c-41e0-9e1d-11ae37508fcc" containerName="istio-proxy"
Sep 09 20:59:36.123431 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:36.123379    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8184bd0-d46c-41e0-9e1d-11ae37508fcc" containerName="istio-proxy"
Sep 09 20:59:36.123431 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:36.123394    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8184bd0-d46c-41e0-9e1d-11ae37508fcc" containerName="istio-proxy"
Sep 09 20:59:36.123431 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:36.123401    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8184bd0-d46c-41e0-9e1d-11ae37508fcc" containerName="istio-proxy"
Sep 09 20:59:36.123537 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:36.123470    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8184bd0-d46c-41e0-9e1d-11ae37508fcc" containerName="istio-proxy"
Sep 09 20:59:36.123537 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:36.123479    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8184bd0-d46c-41e0-9e1d-11ae37508fcc" containerName="istio-proxy"
Sep 09 20:59:36.127997 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:36.127970    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9"
Sep 09 20:59:36.131001 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:36.130976    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-e73aab-openshift-default-dockercfg-mntlg\""
Sep 09 20:59:36.140080 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:36.140057    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9"]
Sep 09 20:59:36.154472 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:36.154447    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkf8k\" (UniqueName: \"kubernetes.io/projected/4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb-kube-api-access-wkf8k\") pod \"e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9\" (UID: \"4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb\") " pod="openshift-ingress/e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9"
Sep 09 20:59:36.154582 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:36.154542    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb-workload-certs\") pod \"e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9\" (UID: \"4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb\") " pod="openshift-ingress/e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9"
Sep 09 20:59:36.154582 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:36.154567    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb-istiod-ca-cert\") pod \"e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9\" (UID: \"4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb\") " pod="openshift-ingress/e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9"
Sep 09 20:59:36.154685 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:36.154593    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb-istio-data\") pod \"e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9\" (UID: \"4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb\") " pod="openshift-ingress/e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9"
Sep 09 20:59:36.154685 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:36.154655    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb-istio-podinfo\") pod \"e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9\" (UID: \"4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb\") " pod="openshift-ingress/e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9"
Sep 09 20:59:36.154779 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:36.154738    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb-workload-socket\") pod \"e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9\" (UID: \"4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb\") " pod="openshift-ingress/e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9"
Sep 09 20:59:36.154828 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:36.154808    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb-istio-envoy\") pod \"e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9\" (UID: \"4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb\") " pod="openshift-ingress/e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9"
Sep 09 20:59:36.154883 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:36.154842    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb-credential-socket\") pod \"e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9\" (UID: \"4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb\") " pod="openshift-ingress/e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9"
Sep 09 20:59:36.154883 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:36.154869    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb-istio-token\") pod \"e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9\" (UID: \"4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb\") " pod="openshift-ingress/e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9"
Sep 09 20:59:36.256095 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:36.256061    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb-workload-certs\") pod \"e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9\" (UID: \"4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb\") " pod="openshift-ingress/e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9"
Sep 09 20:59:36.256095 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:36.256095    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb-istiod-ca-cert\") pod \"e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9\" (UID: \"4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb\") " pod="openshift-ingress/e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9"
Sep 09 20:59:36.256355 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:36.256144    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb-istio-data\") pod \"e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9\" (UID: \"4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb\") " pod="openshift-ingress/e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9"
Sep 09 20:59:36.256355 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:36.256161    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb-istio-podinfo\") pod \"e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9\" (UID: \"4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb\") " pod="openshift-ingress/e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9"
Sep 09 20:59:36.256355 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:36.256201    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb-workload-socket\") pod \"e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9\" (UID: \"4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb\") " pod="openshift-ingress/e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9"
Sep 09 20:59:36.256355 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:36.256266    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb-istio-envoy\") pod \"e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9\" (UID: \"4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb\") " pod="openshift-ingress/e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9"
Sep 09 20:59:36.256355 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:36.256297    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb-credential-socket\") pod \"e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9\" (UID: \"4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb\") " pod="openshift-ingress/e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9"
Sep 09 20:59:36.256355 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:36.256324    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb-istio-token\") pod \"e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9\" (UID: \"4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb\") " pod="openshift-ingress/e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9"
Sep 09 20:59:36.256637 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:36.256389    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkf8k\" (UniqueName: \"kubernetes.io/projected/4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb-kube-api-access-wkf8k\") pod \"e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9\" (UID: \"4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb\") " pod="openshift-ingress/e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9"
Sep 09 20:59:36.256637 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:36.256523    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb-workload-certs\") pod \"e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9\" (UID: \"4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb\") " pod="openshift-ingress/e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9"
Sep 09 20:59:36.256637 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:36.256609    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb-istio-data\") pod \"e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9\" (UID: \"4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb\") " pod="openshift-ingress/e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9"
Sep 09 20:59:36.256924 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:36.256897    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb-workload-socket\") pod \"e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9\" (UID: \"4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb\") " pod="openshift-ingress/e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9"
Sep 09 20:59:36.257012 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:36.256985    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb-istiod-ca-cert\") pod \"e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9\" (UID: \"4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb\") " pod="openshift-ingress/e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9"
Sep 09 20:59:36.257072 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:36.256985    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb-credential-socket\") pod \"e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9\" (UID: \"4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb\") " pod="openshift-ingress/e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9"
Sep 09 20:59:36.258529 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:36.258507    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb-istio-envoy\") pod \"e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9\" (UID: \"4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb\") " pod="openshift-ingress/e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9"
Sep 09 20:59:36.258834 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:36.258818    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb-istio-podinfo\") pod \"e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9\" (UID: \"4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb\") " pod="openshift-ingress/e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9"
Sep 09 20:59:36.264664 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:36.264641    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb-istio-token\") pod \"e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9\" (UID: \"4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb\") " pod="openshift-ingress/e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9"
Sep 09 20:59:36.264780 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:36.264673    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkf8k\" (UniqueName: \"kubernetes.io/projected/4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb-kube-api-access-wkf8k\") pod \"e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9\" (UID: \"4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb\") " pod="openshift-ingress/e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9"
Sep 09 20:59:36.439614 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:36.439541    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9"
Sep 09 20:59:36.721962 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:36.721935    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9"]
Sep 09 20:59:36.724021 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:59:36.723989    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4dc50419_8643_4bd0_8c7a_c3bd6c4d91eb.slice/crio-2e33616cc2a7c16dbd49c0e45b95e76dfed4a5818200673279d9afdb16e0cb36 WatchSource:0}: Error finding container 2e33616cc2a7c16dbd49c0e45b95e76dfed4a5818200673279d9afdb16e0cb36: Status 404 returned error can't find the container with id 2e33616cc2a7c16dbd49c0e45b95e76dfed4a5818200673279d9afdb16e0cb36
Sep 09 20:59:36.726782 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:36.726741    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:59:36.726871 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:36.726837    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:59:36.726912 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:36.726879    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 20:59:36.914159 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:36.914118    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9" event={"ID":"4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb","Type":"ContainerStarted","Data":"5d436b45429bb66f9650eecb0af1270bffdc7659eb46d6c5efa5887817d51b04"}
Sep 09 20:59:36.914356 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:36.914190    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9" event={"ID":"4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb","Type":"ContainerStarted","Data":"2e33616cc2a7c16dbd49c0e45b95e76dfed4a5818200673279d9afdb16e0cb36"}
Sep 09 20:59:36.939943 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:36.939882    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9" podStartSLOduration=0.939867679 podStartE2EDuration="939.867679ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:59:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:59:36.935364356 +0000 UTC m=+1138.562992228" watchObservedRunningTime="2026-09-09 20:59:36.939867679 +0000 UTC m=+1138.567495552"
Sep 09 20:59:37.440186 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:37.440149    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9"
Sep 09 20:59:37.445526 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:37.445496    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9"
Sep 09 20:59:37.919670 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:37.919641    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9"
Sep 09 20:59:37.920766 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:37.920743    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9"
Sep 09 20:59:40.622549 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:40.622514    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-e73aab-6678d9b6cf-6rqh2"]
Sep 09 20:59:40.627400 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:40.627382    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e73aab-6678d9b6cf-6rqh2"
Sep 09 20:59:40.629923 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:40.629907    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-e73aab-dockercfg-xzkpg\""
Sep 09 20:59:40.630653 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:40.630638    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-e73aab\""
Sep 09 20:59:40.639859 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:40.639834    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-e73aab-6678d9b6cf-6rqh2"]
Sep 09 20:59:40.653964 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:40.653933    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e73aab-7f86bb8d78-5q7mc"]
Sep 09 20:59:40.658230 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:40.658212    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e73aab-7f86bb8d78-5q7mc"
Sep 09 20:59:40.667218 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:40.667017    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e73aab-7f86bb8d78-5q7mc"]
Sep 09 20:59:40.706691 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:40.706640    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xc69v\" (UniqueName: \"kubernetes.io/projected/254f0e4f-9e77-4b6c-82e4-fe37777ad5bd-kube-api-access-xc69v\") pod \"payload-pre-processing-e2e-shared-a-w2-e73aab-7f86bb8d78-5q7mc\" (UID: \"254f0e4f-9e77-4b6c-82e4-fe37777ad5bd\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e73aab-7f86bb8d78-5q7mc"
Sep 09 20:59:40.707019 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:40.706996    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f8d283b-a4b9-4e8c-8b3d-e793c73e7f71-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-e73aab-6678d9b6cf-6rqh2\" (UID: \"5f8d283b-a4b9-4e8c-8b3d-e793c73e7f71\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e73aab-6678d9b6cf-6rqh2"
Sep 09 20:59:40.707232 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:40.707215    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqzzv\" (UniqueName: \"kubernetes.io/projected/5f8d283b-a4b9-4e8c-8b3d-e793c73e7f71-kube-api-access-fqzzv\") pod \"payload-processing-e2e-shared-a-w2-e73aab-6678d9b6cf-6rqh2\" (UID: \"5f8d283b-a4b9-4e8c-8b3d-e793c73e7f71\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e73aab-6678d9b6cf-6rqh2"
Sep 09 20:59:40.707691 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:40.707672    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/254f0e4f-9e77-4b6c-82e4-fe37777ad5bd-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-e73aab-7f86bb8d78-5q7mc\" (UID: \"254f0e4f-9e77-4b6c-82e4-fe37777ad5bd\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e73aab-7f86bb8d78-5q7mc"
Sep 09 20:59:40.808607 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:40.808572    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/254f0e4f-9e77-4b6c-82e4-fe37777ad5bd-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-e73aab-7f86bb8d78-5q7mc\" (UID: \"254f0e4f-9e77-4b6c-82e4-fe37777ad5bd\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e73aab-7f86bb8d78-5q7mc"
Sep 09 20:59:40.808793 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:40.808613    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xc69v\" (UniqueName: \"kubernetes.io/projected/254f0e4f-9e77-4b6c-82e4-fe37777ad5bd-kube-api-access-xc69v\") pod \"payload-pre-processing-e2e-shared-a-w2-e73aab-7f86bb8d78-5q7mc\" (UID: \"254f0e4f-9e77-4b6c-82e4-fe37777ad5bd\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e73aab-7f86bb8d78-5q7mc"
Sep 09 20:59:40.808793 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:40.808639    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f8d283b-a4b9-4e8c-8b3d-e793c73e7f71-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-e73aab-6678d9b6cf-6rqh2\" (UID: \"5f8d283b-a4b9-4e8c-8b3d-e793c73e7f71\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e73aab-6678d9b6cf-6rqh2"
Sep 09 20:59:40.808793 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:40.808678    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqzzv\" (UniqueName: \"kubernetes.io/projected/5f8d283b-a4b9-4e8c-8b3d-e793c73e7f71-kube-api-access-fqzzv\") pod \"payload-processing-e2e-shared-a-w2-e73aab-6678d9b6cf-6rqh2\" (UID: \"5f8d283b-a4b9-4e8c-8b3d-e793c73e7f71\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e73aab-6678d9b6cf-6rqh2"
Sep 09 20:59:40.809483 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:40.809448    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/254f0e4f-9e77-4b6c-82e4-fe37777ad5bd-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-e73aab-7f86bb8d78-5q7mc\" (UID: \"254f0e4f-9e77-4b6c-82e4-fe37777ad5bd\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e73aab-7f86bb8d78-5q7mc"
Sep 09 20:59:40.809586 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:40.809536    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f8d283b-a4b9-4e8c-8b3d-e793c73e7f71-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-e73aab-6678d9b6cf-6rqh2\" (UID: \"5f8d283b-a4b9-4e8c-8b3d-e793c73e7f71\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e73aab-6678d9b6cf-6rqh2"
Sep 09 20:59:40.822728 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:40.822692    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xc69v\" (UniqueName: \"kubernetes.io/projected/254f0e4f-9e77-4b6c-82e4-fe37777ad5bd-kube-api-access-xc69v\") pod \"payload-pre-processing-e2e-shared-a-w2-e73aab-7f86bb8d78-5q7mc\" (UID: \"254f0e4f-9e77-4b6c-82e4-fe37777ad5bd\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e73aab-7f86bb8d78-5q7mc"
Sep 09 20:59:40.822852 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:40.822698    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqzzv\" (UniqueName: \"kubernetes.io/projected/5f8d283b-a4b9-4e8c-8b3d-e793c73e7f71-kube-api-access-fqzzv\") pod \"payload-processing-e2e-shared-a-w2-e73aab-6678d9b6cf-6rqh2\" (UID: \"5f8d283b-a4b9-4e8c-8b3d-e793c73e7f71\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e73aab-6678d9b6cf-6rqh2"
Sep 09 20:59:40.937944 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:40.937833    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e73aab-6678d9b6cf-6rqh2"
Sep 09 20:59:40.971148 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:40.971095    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e73aab-7f86bb8d78-5q7mc"
Sep 09 20:59:41.217388 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:41.217360    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-e73aab-6678d9b6cf-6rqh2"]
Sep 09 20:59:41.219810 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:59:41.219774    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f8d283b_a4b9_4e8c_8b3d_e793c73e7f71.slice/crio-9c604c8007c619c3270942e0b2eb6f13dfe187cae43af9185cb1b085f4f4f3de WatchSource:0}: Error finding container 9c604c8007c619c3270942e0b2eb6f13dfe187cae43af9185cb1b085f4f4f3de: Status 404 returned error can't find the container with id 9c604c8007c619c3270942e0b2eb6f13dfe187cae43af9185cb1b085f4f4f3de
Sep 09 20:59:41.275279 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:41.275248    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e73aab-7f86bb8d78-5q7mc"]
Sep 09 20:59:41.277332 ip-10-0-129-99 kubenswrapper[2581]: W0909 20:59:41.277304    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod254f0e4f_9e77_4b6c_82e4_fe37777ad5bd.slice/crio-a7ca85c6ede25fd1deb3a9ca5c308d35c9f66918d88cd27828ee0fdadf58cef3 WatchSource:0}: Error finding container a7ca85c6ede25fd1deb3a9ca5c308d35c9f66918d88cd27828ee0fdadf58cef3: Status 404 returned error can't find the container with id a7ca85c6ede25fd1deb3a9ca5c308d35c9f66918d88cd27828ee0fdadf58cef3
Sep 09 20:59:41.945540 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:41.945443    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e73aab-6678d9b6cf-6rqh2" event={"ID":"5f8d283b-a4b9-4e8c-8b3d-e793c73e7f71","Type":"ContainerStarted","Data":"2160abc018e3bdf52afc07805dfa83c7b4ce86afcce864d39ac57234a23723fb"}
Sep 09 20:59:41.945540 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:41.945503    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e73aab-6678d9b6cf-6rqh2" event={"ID":"5f8d283b-a4b9-4e8c-8b3d-e793c73e7f71","Type":"ContainerStarted","Data":"9c604c8007c619c3270942e0b2eb6f13dfe187cae43af9185cb1b085f4f4f3de"}
Sep 09 20:59:41.945540 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:41.945533    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e73aab-6678d9b6cf-6rqh2"
Sep 09 20:59:41.947215 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:41.947189    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e73aab-7f86bb8d78-5q7mc" event={"ID":"254f0e4f-9e77-4b6c-82e4-fe37777ad5bd","Type":"ContainerStarted","Data":"d5c3a5ed4d386757887a2022958b92564ea57735aeaf1c1e3b55189f5ed86f5a"}
Sep 09 20:59:41.947351 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:41.947235    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e73aab-7f86bb8d78-5q7mc" event={"ID":"254f0e4f-9e77-4b6c-82e4-fe37777ad5bd","Type":"ContainerStarted","Data":"a7ca85c6ede25fd1deb3a9ca5c308d35c9f66918d88cd27828ee0fdadf58cef3"}
Sep 09 20:59:41.947351 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:41.947322    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e73aab-7f86bb8d78-5q7mc"
Sep 09 20:59:41.962878 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:41.962830    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e73aab-6678d9b6cf-6rqh2" podStartSLOduration=1.962815379 podStartE2EDuration="1.962815379s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:59:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:59:41.961896642 +0000 UTC m=+1143.589524529" watchObservedRunningTime="2026-09-09 20:59:41.962815379 +0000 UTC m=+1143.590443251"
Sep 09 20:59:41.980968 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:41.980928    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e73aab-7f86bb8d78-5q7mc" podStartSLOduration=1.980915464 podStartE2EDuration="1.980915464s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:59:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:59:41.978590737 +0000 UTC m=+1143.606218611" watchObservedRunningTime="2026-09-09 20:59:41.980915464 +0000 UTC m=+1143.608543342"
Sep 09 20:59:42.487395 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:42.487355    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-47e41c/distinct-2-w4-47e41c-kserve-67c87c7696-slxnj"
Sep 09 20:59:42.487600 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:42.487448    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-47e41c/distinct-2-w4-47e41c-kserve-67c87c7696-slxnj"
Sep 09 20:59:42.500131 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:42.500090    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-47e41c/distinct-2-w4-47e41c-kserve-67c87c7696-slxnj"
Sep 09 20:59:42.970773 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:42.970694    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-47e41c/distinct-2-w4-47e41c-kserve-67c87c7696-slxnj"
Sep 09 20:59:52.955224 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:52.955192    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e73aab-6678d9b6cf-6rqh2"
Sep 09 20:59:52.955616 ip-10-0-129-99 kubenswrapper[2581]: I0909 20:59:52.955269    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e73aab-7f86bb8d78-5q7mc"
Sep 09 21:00:00.137760 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:00.137726    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-47e41c-29816460-95t86"]
Sep 09 21:00:00.141595 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:00.141575    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-e73aab-29816460-z97p9"]
Sep 09 21:00:00.141721 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:00.141689    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-47e41c-29816460-95t86"
Sep 09 21:00:00.144261 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:00.144241    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-glz44\""
Sep 09 21:00:00.145005 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:00.144990    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-5eef90-29816460-djtcd"]
Sep 09 21:00:00.145134 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:00.145121    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-e73aab-29816460-z97p9"
Sep 09 21:00:00.148653 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:00.148631    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-5eef90-29816460-djtcd"
Sep 09 21:00:00.150819 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:00.150800    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-423e6a-29816460-kf6wm"]
Sep 09 21:00:00.155131 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:00.154331    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-423e6a-29816460-kf6wm"
Sep 09 21:00:00.155131 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:00.154828    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-47e41c-29816460-95t86"
Sep 09 21:00:00.155833 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:00.155814    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-47e41c-29816460-95t86"]
Sep 09 21:00:00.158532 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:00.158519    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-5eef90-29816460-djtcd"]
Sep 09 21:00:00.161354 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:00.161165    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-e73aab-29816460-z97p9"]
Sep 09 21:00:00.166408 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:00.164995    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-e73aab-29816460-z97p9"
Sep 09 21:00:00.173866 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:00.172593    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-423e6a-29816460-kf6wm"]
Sep 09 21:00:00.232438 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:00.231901    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-5eef90-29816460-djtcd"
Sep 09 21:00:00.245187 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:00.244836    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-423e6a-29816460-kf6wm"
Sep 09 21:00:00.478181 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:00.478158    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-47e41c-29816460-95t86"]
Sep 09 21:00:00.480180 ip-10-0-129-99 kubenswrapper[2581]: W0909 21:00:00.480149    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc45a510c_8af8_45c9_8499_c11b11ae4bb2.slice/crio-3837dc71c0d4e392226b6875e262de352adc9c1ad6b9a6d426c65e461e625000 WatchSource:0}: Error finding container 3837dc71c0d4e392226b6875e262de352adc9c1ad6b9a6d426c65e461e625000: Status 404 returned error can't find the container with id 3837dc71c0d4e392226b6875e262de352adc9c1ad6b9a6d426c65e461e625000
Sep 09 21:00:00.482771 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:00.482747    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-e73aab-29816460-z97p9"]
Sep 09 21:00:00.483336 ip-10-0-129-99 kubenswrapper[2581]: W0909 21:00:00.483315    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6653cb7_c4bf_4703_87b7_40ea28390c44.slice/crio-1d9b57c5ad242fd83254897acfd7197399288da564ae9d01cff047e501bf81ea WatchSource:0}: Error finding container 1d9b57c5ad242fd83254897acfd7197399288da564ae9d01cff047e501bf81ea: Status 404 returned error can't find the container with id 1d9b57c5ad242fd83254897acfd7197399288da564ae9d01cff047e501bf81ea
Sep 09 21:00:00.549681 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:00.549229    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-5eef90-29816460-djtcd"]
Sep 09 21:00:00.576219 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:00.576196    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-423e6a-29816460-kf6wm"]
Sep 09 21:00:00.581327 ip-10-0-129-99 kubenswrapper[2581]: W0909 21:00:00.581304    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f69d8aa_4cee_4e44_a1dc_186c32bdd357.slice/crio-0b997a513c83ae999eb8f9e5aa82183eed4b376cd31ef86cfc94eff0d3fe04fa WatchSource:0}: Error finding container 0b997a513c83ae999eb8f9e5aa82183eed4b376cd31ef86cfc94eff0d3fe04fa: Status 404 returned error can't find the container with id 0b997a513c83ae999eb8f9e5aa82183eed4b376cd31ef86cfc94eff0d3fe04fa
Sep 09 21:00:01.054717 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:01.054684    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-5eef90-29816460-djtcd" event={"ID":"8e464858-076a-44ef-8c80-368eb08ca836","Type":"ContainerStarted","Data":"b5137f94e95cdbc14ef40cbb1c3bba0f72e837216f9b4b67a0cbd6b82613287a"}
Sep 09 21:00:01.054864 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:01.054743    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-5eef90-29816460-djtcd" event={"ID":"8e464858-076a-44ef-8c80-368eb08ca836","Type":"ContainerStarted","Data":"dc6d3e7be79223eca66977552f77ff5377b4ca998e0ca60c7b93a09ad9c32717"}
Sep 09 21:00:01.056289 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:01.056262    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-47e41c-29816460-95t86" event={"ID":"c45a510c-8af8-45c9-8499-c11b11ae4bb2","Type":"ContainerStarted","Data":"09d4c34209a386a6be05b6baea5cba452bd1ddcfb1ecb4bd9b40f65336955d16"}
Sep 09 21:00:01.056404 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:01.056313    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-47e41c-29816460-95t86" event={"ID":"c45a510c-8af8-45c9-8499-c11b11ae4bb2","Type":"ContainerStarted","Data":"3837dc71c0d4e392226b6875e262de352adc9c1ad6b9a6d426c65e461e625000"}
Sep 09 21:00:01.057756 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:01.057730    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-423e6a-29816460-kf6wm" event={"ID":"9f69d8aa-4cee-4e44-a1dc-186c32bdd357","Type":"ContainerStarted","Data":"ff8eb15373989afc543474758a7341b5edd7dab6c281b76b9d740e39e590c9bc"}
Sep 09 21:00:01.057882 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:01.057815    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-423e6a-29816460-kf6wm" event={"ID":"9f69d8aa-4cee-4e44-a1dc-186c32bdd357","Type":"ContainerStarted","Data":"0b997a513c83ae999eb8f9e5aa82183eed4b376cd31ef86cfc94eff0d3fe04fa"}
Sep 09 21:00:01.059286 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:01.059266    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-e73aab-29816460-z97p9" event={"ID":"b6653cb7-c4bf-4703-87b7-40ea28390c44","Type":"ContainerStarted","Data":"ebedddd971fc496b4f091052890d0e42645433022a85b977df0ea55668ac1de3"}
Sep 09 21:00:01.059381 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:01.059307    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-e73aab-29816460-z97p9" event={"ID":"b6653cb7-c4bf-4703-87b7-40ea28390c44","Type":"ContainerStarted","Data":"1d9b57c5ad242fd83254897acfd7197399288da564ae9d01cff047e501bf81ea"}
Sep 09 21:00:01.070795 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:01.070752    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-5eef90-29816460-djtcd" podStartSLOduration=1.070740101 podStartE2EDuration="1.070740101s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:00:01.06913206 +0000 UTC m=+1162.696759931" watchObservedRunningTime="2026-09-09 21:00:01.070740101 +0000 UTC m=+1162.698367974"
Sep 09 21:00:01.085429 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:01.085386    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-423e6a-29816460-kf6wm" podStartSLOduration=1.085371922 podStartE2EDuration="1.085371922s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:00:01.082850894 +0000 UTC m=+1162.710478768" watchObservedRunningTime="2026-09-09 21:00:01.085371922 +0000 UTC m=+1162.712999794"
Sep 09 21:00:01.098759 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:01.098713    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-47e41c-29816460-95t86" podStartSLOduration=1.098700867 podStartE2EDuration="1.098700867s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:00:01.097203744 +0000 UTC m=+1162.724831618" watchObservedRunningTime="2026-09-09 21:00:01.098700867 +0000 UTC m=+1162.726328739"
Sep 09 21:00:01.113337 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:01.113281    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-e73aab-29816460-z97p9" podStartSLOduration=1.113246516 podStartE2EDuration="1.113246516s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:00:01.112308084 +0000 UTC m=+1162.739935962" watchObservedRunningTime="2026-09-09 21:00:01.113246516 +0000 UTC m=+1162.740874390"
Sep 09 21:00:02.071008 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:02.070973    2581 generic.go:356] "Generic (PLEG): container finished" podID="8e464858-076a-44ef-8c80-368eb08ca836" containerID="b5137f94e95cdbc14ef40cbb1c3bba0f72e837216f9b4b67a0cbd6b82613287a" exitCode=0
Sep 09 21:00:02.071424 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:02.071069    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-5eef90-29816460-djtcd" event={"ID":"8e464858-076a-44ef-8c80-368eb08ca836","Type":"ContainerDied","Data":"b5137f94e95cdbc14ef40cbb1c3bba0f72e837216f9b4b67a0cbd6b82613287a"}
Sep 09 21:00:02.072959 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:02.072933    2581 generic.go:356] "Generic (PLEG): container finished" podID="c45a510c-8af8-45c9-8499-c11b11ae4bb2" containerID="09d4c34209a386a6be05b6baea5cba452bd1ddcfb1ecb4bd9b40f65336955d16" exitCode=0
Sep 09 21:00:02.073145 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:02.073014    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-47e41c-29816460-95t86" event={"ID":"c45a510c-8af8-45c9-8499-c11b11ae4bb2","Type":"ContainerDied","Data":"09d4c34209a386a6be05b6baea5cba452bd1ddcfb1ecb4bd9b40f65336955d16"}
Sep 09 21:00:02.074607 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:02.074583    2581 generic.go:356] "Generic (PLEG): container finished" podID="9f69d8aa-4cee-4e44-a1dc-186c32bdd357" containerID="ff8eb15373989afc543474758a7341b5edd7dab6c281b76b9d740e39e590c9bc" exitCode=0
Sep 09 21:00:02.074746 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:02.074666    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-423e6a-29816460-kf6wm" event={"ID":"9f69d8aa-4cee-4e44-a1dc-186c32bdd357","Type":"ContainerDied","Data":"ff8eb15373989afc543474758a7341b5edd7dab6c281b76b9d740e39e590c9bc"}
Sep 09 21:00:02.076284 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:02.076259    2581 generic.go:356] "Generic (PLEG): container finished" podID="b6653cb7-c4bf-4703-87b7-40ea28390c44" containerID="ebedddd971fc496b4f091052890d0e42645433022a85b977df0ea55668ac1de3" exitCode=0
Sep 09 21:00:02.076380 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:02.076300    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-e73aab-29816460-z97p9" event={"ID":"b6653cb7-c4bf-4703-87b7-40ea28390c44","Type":"ContainerDied","Data":"ebedddd971fc496b4f091052890d0e42645433022a85b977df0ea55668ac1de3"}
Sep 09 21:00:03.274430 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:03.274405    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-423e6a-29816460-kf6wm"
Sep 09 21:00:03.352995 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:03.352877    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-5eef90-29816460-djtcd"
Sep 09 21:00:03.367002 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:03.366980    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-e73aab-29816460-z97p9"
Sep 09 21:00:03.370237 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:03.370221    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-47e41c-29816460-95t86"
Sep 09 21:00:04.090175 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:04.090148    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-5eef90-29816460-djtcd"
Sep 09 21:00:04.090375 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:04.090144    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-5eef90-29816460-djtcd" event={"ID":"8e464858-076a-44ef-8c80-368eb08ca836","Type":"ContainerDied","Data":"dc6d3e7be79223eca66977552f77ff5377b4ca998e0ca60c7b93a09ad9c32717"}
Sep 09 21:00:04.090375 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:04.090277    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dc6d3e7be79223eca66977552f77ff5377b4ca998e0ca60c7b93a09ad9c32717"
Sep 09 21:00:04.091450 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:04.091430    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-47e41c-29816460-95t86"
Sep 09 21:00:04.091563 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:04.091454    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-47e41c-29816460-95t86" event={"ID":"c45a510c-8af8-45c9-8499-c11b11ae4bb2","Type":"ContainerDied","Data":"3837dc71c0d4e392226b6875e262de352adc9c1ad6b9a6d426c65e461e625000"}
Sep 09 21:00:04.091563 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:04.091493    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3837dc71c0d4e392226b6875e262de352adc9c1ad6b9a6d426c65e461e625000"
Sep 09 21:00:04.092905 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:04.092882    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-423e6a-29816460-kf6wm" event={"ID":"9f69d8aa-4cee-4e44-a1dc-186c32bdd357","Type":"ContainerDied","Data":"0b997a513c83ae999eb8f9e5aa82183eed4b376cd31ef86cfc94eff0d3fe04fa"}
Sep 09 21:00:04.092998 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:04.092904    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-423e6a-29816460-kf6wm"
Sep 09 21:00:04.092998 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:04.092925    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0b997a513c83ae999eb8f9e5aa82183eed4b376cd31ef86cfc94eff0d3fe04fa"
Sep 09 21:00:04.094185 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:04.094161    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-e73aab-29816460-z97p9" event={"ID":"b6653cb7-c4bf-4703-87b7-40ea28390c44","Type":"ContainerDied","Data":"1d9b57c5ad242fd83254897acfd7197399288da564ae9d01cff047e501bf81ea"}
Sep 09 21:00:04.094292 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:04.094194    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1d9b57c5ad242fd83254897acfd7197399288da564ae9d01cff047e501bf81ea"
Sep 09 21:00:04.094292 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:04.094169    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-e73aab-29816460-z97p9"
Sep 09 21:00:04.441733 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:04.441641    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d"]
Sep 09 21:00:04.442410 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:04.442388    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6653cb7-c4bf-4703-87b7-40ea28390c44" containerName="cleanup"
Sep 09 21:00:04.442542 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:04.442420    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6653cb7-c4bf-4703-87b7-40ea28390c44" containerName="cleanup"
Sep 09 21:00:04.442542 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:04.442432    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c45a510c-8af8-45c9-8499-c11b11ae4bb2" containerName="cleanup"
Sep 09 21:00:04.442542 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:04.442442    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c45a510c-8af8-45c9-8499-c11b11ae4bb2" containerName="cleanup"
Sep 09 21:00:04.442542 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:04.442496    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f69d8aa-4cee-4e44-a1dc-186c32bdd357" containerName="cleanup"
Sep 09 21:00:04.442542 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:04.442506    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f69d8aa-4cee-4e44-a1dc-186c32bdd357" containerName="cleanup"
Sep 09 21:00:04.442542 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:04.442515    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e464858-076a-44ef-8c80-368eb08ca836" containerName="cleanup"
Sep 09 21:00:04.442542 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:04.442523    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e464858-076a-44ef-8c80-368eb08ca836" containerName="cleanup"
Sep 09 21:00:04.442893 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:04.442651    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f69d8aa-4cee-4e44-a1dc-186c32bdd357" containerName="cleanup"
Sep 09 21:00:04.442893 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:04.442670    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c45a510c-8af8-45c9-8499-c11b11ae4bb2" containerName="cleanup"
Sep 09 21:00:04.442893 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:04.442683    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6653cb7-c4bf-4703-87b7-40ea28390c44" containerName="cleanup"
Sep 09 21:00:04.442893 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:04.442692    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e464858-076a-44ef-8c80-368eb08ca836" containerName="cleanup"
Sep 09 21:00:04.446782 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:04.446763    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d"
Sep 09 21:00:04.450857 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:04.450679    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-06e1ff-openshift-default-dockercfg-5lrrt\""
Sep 09 21:00:04.468509 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:04.468486    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19-workload-certs\") pod \"e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d\" (UID: \"0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19\") " pod="openshift-ingress/e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d"
Sep 09 21:00:04.468623 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:04.468523    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19-credential-socket\") pod \"e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d\" (UID: \"0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19\") " pod="openshift-ingress/e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d"
Sep 09 21:00:04.468623 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:04.468562    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19-istio-envoy\") pod \"e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d\" (UID: \"0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19\") " pod="openshift-ingress/e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d"
Sep 09 21:00:04.468623 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:04.468606    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19-istiod-ca-cert\") pod \"e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d\" (UID: \"0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19\") " pod="openshift-ingress/e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d"
Sep 09 21:00:04.468795 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:04.468724    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19-istio-data\") pod \"e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d\" (UID: \"0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19\") " pod="openshift-ingress/e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d"
Sep 09 21:00:04.468855 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:04.468794    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19-istio-podinfo\") pod \"e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d\" (UID: \"0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19\") " pod="openshift-ingress/e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d"
Sep 09 21:00:04.468855 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:04.468826    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19-istio-token\") pod \"e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d\" (UID: \"0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19\") " pod="openshift-ingress/e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d"
Sep 09 21:00:04.468949 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:04.468883    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqc5g\" (UniqueName: \"kubernetes.io/projected/0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19-kube-api-access-lqc5g\") pod \"e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d\" (UID: \"0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19\") " pod="openshift-ingress/e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d"
Sep 09 21:00:04.468996 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:04.468959    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19-workload-socket\") pod \"e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d\" (UID: \"0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19\") " pod="openshift-ingress/e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d"
Sep 09 21:00:04.474178 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:04.474096    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d"]
Sep 09 21:00:04.571598 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:04.571545    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19-istio-envoy\") pod \"e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d\" (UID: \"0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19\") " pod="openshift-ingress/e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d"
Sep 09 21:00:04.571791 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:04.571618    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19-istiod-ca-cert\") pod \"e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d\" (UID: \"0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19\") " pod="openshift-ingress/e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d"
Sep 09 21:00:04.571791 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:04.571697    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19-istio-data\") pod \"e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d\" (UID: \"0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19\") " pod="openshift-ingress/e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d"
Sep 09 21:00:04.571791 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:04.571722    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19-istio-podinfo\") pod \"e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d\" (UID: \"0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19\") " pod="openshift-ingress/e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d"
Sep 09 21:00:04.571791 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:04.571748    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19-istio-token\") pod \"e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d\" (UID: \"0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19\") " pod="openshift-ingress/e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d"
Sep 09 21:00:04.572021 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:04.571808    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqc5g\" (UniqueName: \"kubernetes.io/projected/0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19-kube-api-access-lqc5g\") pod \"e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d\" (UID: \"0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19\") " pod="openshift-ingress/e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d"
Sep 09 21:00:04.572021 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:04.571877    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19-workload-socket\") pod \"e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d\" (UID: \"0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19\") " pod="openshift-ingress/e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d"
Sep 09 21:00:04.572021 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:04.571912    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19-workload-certs\") pod \"e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d\" (UID: \"0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19\") " pod="openshift-ingress/e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d"
Sep 09 21:00:04.572021 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:04.571952    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19-credential-socket\") pod \"e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d\" (UID: \"0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19\") " pod="openshift-ingress/e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d"
Sep 09 21:00:04.572496 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:04.572465    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19-credential-socket\") pod \"e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d\" (UID: \"0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19\") " pod="openshift-ingress/e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d"
Sep 09 21:00:04.573839 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:04.573812    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19-istiod-ca-cert\") pod \"e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d\" (UID: \"0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19\") " pod="openshift-ingress/e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d"
Sep 09 21:00:04.575338 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:04.574274    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19-istio-data\") pod \"e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d\" (UID: \"0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19\") " pod="openshift-ingress/e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d"
Sep 09 21:00:04.575459 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:04.574897    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19-workload-socket\") pod \"e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d\" (UID: \"0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19\") " pod="openshift-ingress/e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d"
Sep 09 21:00:04.575522 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:04.575472    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19-workload-certs\") pod \"e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d\" (UID: \"0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19\") " pod="openshift-ingress/e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d"
Sep 09 21:00:04.578881 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:04.578278    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19-istio-envoy\") pod \"e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d\" (UID: \"0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19\") " pod="openshift-ingress/e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d"
Sep 09 21:00:04.579590 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:04.579564    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19-istio-podinfo\") pod \"e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d\" (UID: \"0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19\") " pod="openshift-ingress/e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d"
Sep 09 21:00:04.582770 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:04.582744    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19-istio-token\") pod \"e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d\" (UID: \"0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19\") " pod="openshift-ingress/e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d"
Sep 09 21:00:04.584646 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:04.584624    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqc5g\" (UniqueName: \"kubernetes.io/projected/0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19-kube-api-access-lqc5g\") pod \"e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d\" (UID: \"0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19\") " pod="openshift-ingress/e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d"
Sep 09 21:00:04.757817 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:04.757778    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d"
Sep 09 21:00:05.063692 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:05.063668    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d"]
Sep 09 21:00:05.066033 ip-10-0-129-99 kubenswrapper[2581]: W0909 21:00:05.066005    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0110caa6_0efd_4f2e_8ac4_ca1c09b0cc19.slice/crio-bc3a225477954c3cfb5f7193d2ccbd02fe3be074e2273e74d2b8e56360fff661 WatchSource:0}: Error finding container bc3a225477954c3cfb5f7193d2ccbd02fe3be074e2273e74d2b8e56360fff661: Status 404 returned error can't find the container with id bc3a225477954c3cfb5f7193d2ccbd02fe3be074e2273e74d2b8e56360fff661
Sep 09 21:00:05.068985 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:05.068945    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:00:05.069084 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:05.069018    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:00:05.069084 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:05.069052    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:00:05.100864 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:05.100837    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d" event={"ID":"0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19","Type":"ContainerStarted","Data":"bc3a225477954c3cfb5f7193d2ccbd02fe3be074e2273e74d2b8e56360fff661"}
Sep 09 21:00:06.109818 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:06.109783    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d" event={"ID":"0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19","Type":"ContainerStarted","Data":"f216c804a79fbbecd1c0b8efb736e59ea13ff0e38cbb24279f3032ae76994df3"}
Sep 09 21:00:06.758374 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:06.758339    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d"
Sep 09 21:00:06.763525 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:06.763497    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d"
Sep 09 21:00:06.785301 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:06.785243    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d" podStartSLOduration=2.785225145 podStartE2EDuration="2.785225145s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:00:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:00:06.154867158 +0000 UTC m=+1167.782495024" watchObservedRunningTime="2026-09-09 21:00:06.785225145 +0000 UTC m=+1168.412853017"
Sep 09 21:00:07.116187 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:07.116076    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d"
Sep 09 21:00:07.117229 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:07.117208    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d"
Sep 09 21:00:08.202668 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:08.202023    2581 patch_prober.go:28] interesting pod/e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": read tcp 10.133.0.2:60838->10.133.0.62:15021: read: connection reset by peer" start-of-body=
Sep 09 21:00:08.202668 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:08.202133    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9" podUID="4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": read tcp 10.133.0.2:60838->10.133.0.62:15021: read: connection reset by peer"
Sep 09 21:00:09.136401 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:09.136367    2581 generic.go:356] "Generic (PLEG): container finished" podID="4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb" containerID="5d436b45429bb66f9650eecb0af1270bffdc7659eb46d6c5efa5887817d51b04" exitCode=0
Sep 09 21:00:09.136584 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:09.136442    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9" event={"ID":"4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb","Type":"ContainerDied","Data":"5d436b45429bb66f9650eecb0af1270bffdc7659eb46d6c5efa5887817d51b04"}
Sep 09 21:00:09.136968 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:09.136947    2581 scope.go:117] "RemoveContainer" containerID="5d436b45429bb66f9650eecb0af1270bffdc7659eb46d6c5efa5887817d51b04"
Sep 09 21:00:09.138133 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:09.138076    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:00:09.138236 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:09.138171    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:00:09.138236 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:09.138205    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:00:09.959216 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:09.959183    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-06e1ff-768dd8d94c-8t6lm"]
Sep 09 21:00:09.962909 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:09.962889    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-06e1ff-768dd8d94c-8t6lm"
Sep 09 21:00:09.969073 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:09.969052    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-06e1ff\""
Sep 09 21:00:09.977773 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:09.977747    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-06e1ff-768dd8d94c-8t6lm"]
Sep 09 21:00:10.030920 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:10.030885    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea95b990-0e24-4d67-b761-c811612e7a78-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-06e1ff-768dd8d94c-8t6lm\" (UID: \"ea95b990-0e24-4d67-b761-c811612e7a78\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-06e1ff-768dd8d94c-8t6lm"
Sep 09 21:00:10.031149 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:10.030947    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwmnp\" (UniqueName: \"kubernetes.io/projected/ea95b990-0e24-4d67-b761-c811612e7a78-kube-api-access-cwmnp\") pod \"payload-processing-e2e-shared-b-w2-06e1ff-768dd8d94c-8t6lm\" (UID: \"ea95b990-0e24-4d67-b761-c811612e7a78\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-06e1ff-768dd8d94c-8t6lm"
Sep 09 21:00:10.136123 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:10.133063    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea95b990-0e24-4d67-b761-c811612e7a78-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-06e1ff-768dd8d94c-8t6lm\" (UID: \"ea95b990-0e24-4d67-b761-c811612e7a78\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-06e1ff-768dd8d94c-8t6lm"
Sep 09 21:00:10.136123 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:10.133185    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwmnp\" (UniqueName: \"kubernetes.io/projected/ea95b990-0e24-4d67-b761-c811612e7a78-kube-api-access-cwmnp\") pod \"payload-processing-e2e-shared-b-w2-06e1ff-768dd8d94c-8t6lm\" (UID: \"ea95b990-0e24-4d67-b761-c811612e7a78\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-06e1ff-768dd8d94c-8t6lm"
Sep 09 21:00:10.136123 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:10.134017    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea95b990-0e24-4d67-b761-c811612e7a78-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-06e1ff-768dd8d94c-8t6lm\" (UID: \"ea95b990-0e24-4d67-b761-c811612e7a78\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-06e1ff-768dd8d94c-8t6lm"
Sep 09 21:00:10.147152 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:10.144555    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwmnp\" (UniqueName: \"kubernetes.io/projected/ea95b990-0e24-4d67-b761-c811612e7a78-kube-api-access-cwmnp\") pod \"payload-processing-e2e-shared-b-w2-06e1ff-768dd8d94c-8t6lm\" (UID: \"ea95b990-0e24-4d67-b761-c811612e7a78\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-06e1ff-768dd8d94c-8t6lm"
Sep 09 21:00:10.154989 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:10.154956    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9" event={"ID":"4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb","Type":"ContainerStarted","Data":"4189c01731023c93ec96e6dca32e3e8710cfd22b16f3e76fc4c0505ce87ee97c"}
Sep 09 21:00:10.273938 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:10.273894    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-06e1ff-768dd8d94c-8t6lm"
Sep 09 21:00:10.440724 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:10.440688    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9"
Sep 09 21:00:10.440933 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:10.440908    2581 patch_prober.go:28] interesting pod/e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Sep 09 21:00:10.440981 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:10.440960    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9" podUID="4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Sep 09 21:00:10.572962 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:10.572929    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-06e1ff-768dd8d94c-8t6lm"]
Sep 09 21:00:10.574027 ip-10-0-129-99 kubenswrapper[2581]: W0909 21:00:10.574001    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea95b990_0e24_4d67_b761_c811612e7a78.slice/crio-a1eedc906e724554918d70db442c7e920b0819951c555cac939ab5b9a17e96f9 WatchSource:0}: Error finding container a1eedc906e724554918d70db442c7e920b0819951c555cac939ab5b9a17e96f9: Status 404 returned error can't find the container with id a1eedc906e724554918d70db442c7e920b0819951c555cac939ab5b9a17e96f9
Sep 09 21:00:11.163078 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:11.163033    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-06e1ff-768dd8d94c-8t6lm" event={"ID":"ea95b990-0e24-4d67-b761-c811612e7a78","Type":"ContainerStarted","Data":"38a89293a00f29712331c3a618b6e2d1558c30b37961f6ca6f4346d096129b01"}
Sep 09 21:00:11.163500 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:11.163091    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-06e1ff-768dd8d94c-8t6lm" event={"ID":"ea95b990-0e24-4d67-b761-c811612e7a78","Type":"ContainerStarted","Data":"a1eedc906e724554918d70db442c7e920b0819951c555cac939ab5b9a17e96f9"}
Sep 09 21:00:11.163500 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:11.163136    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-06e1ff-768dd8d94c-8t6lm"
Sep 09 21:00:11.184207 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:11.184154    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-06e1ff-768dd8d94c-8t6lm" podStartSLOduration=2.184134081 podStartE2EDuration="2.184134081s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:00:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:00:11.183138868 +0000 UTC m=+1172.810766744" watchObservedRunningTime="2026-09-09 21:00:11.184134081 +0000 UTC m=+1172.811761956"
Sep 09 21:00:11.440551 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:11.440461    2581 patch_prober.go:28] interesting pod/e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Sep 09 21:00:11.440551 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:11.440527    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9" podUID="4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Sep 09 21:00:12.440584 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:12.440551    2581 patch_prober.go:28] interesting pod/e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Sep 09 21:00:12.441027 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:12.441001    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9" podUID="4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Sep 09 21:00:13.440849 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:13.440809    2581 patch_prober.go:28] interesting pod/e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Sep 09 21:00:13.441314 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:13.440880    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9" podUID="4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Sep 09 21:00:14.440042 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:14.439996    2581 patch_prober.go:28] interesting pod/e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Sep 09 21:00:14.440245 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:14.440077    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9" podUID="4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Sep 09 21:00:15.443833 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:15.443800    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9"
Sep 09 21:00:15.444224 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:15.444092    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9"
Sep 09 21:00:15.444990 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:15.444969    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9"
Sep 09 21:00:21.804524 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:21.804488    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5eef90-lbnj4"]
Sep 09 21:00:21.810759 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:21.810738    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5eef90-lbnj4"
Sep 09 21:00:21.813707 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:21.813674    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-glz44\""
Sep 09 21:00:21.815079 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:21.815052    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5eef90-lbnj4"]
Sep 09 21:00:21.862225 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:21.862193    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcf974a4-3d84-43b7-883a-dd023afb882b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-5eef90-lbnj4\" (UID: \"fcf974a4-3d84-43b7-883a-dd023afb882b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5eef90-lbnj4"
Sep 09 21:00:21.962833 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:21.962798    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcf974a4-3d84-43b7-883a-dd023afb882b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-5eef90-lbnj4\" (UID: \"fcf974a4-3d84-43b7-883a-dd023afb882b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5eef90-lbnj4"
Sep 09 21:00:21.963473 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:21.963448    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcf974a4-3d84-43b7-883a-dd023afb882b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-5eef90-lbnj4\" (UID: \"fcf974a4-3d84-43b7-883a-dd023afb882b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5eef90-lbnj4"
Sep 09 21:00:22.135899 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:22.135805    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5eef90-lbnj4"
Sep 09 21:00:22.171300 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:22.171263    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-06e1ff-768dd8d94c-8t6lm"
Sep 09 21:00:22.426186 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:22.418729    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5eef90-lbnj4"]
Sep 09 21:00:22.427435 ip-10-0-129-99 kubenswrapper[2581]: W0909 21:00:22.427403    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcf974a4_3d84_43b7_883a_dd023afb882b.slice/crio-a7023f8db3f182b0a45e3af753fcddfa2a6357aa436a9a4b4863a8ceda5df200 WatchSource:0}: Error finding container a7023f8db3f182b0a45e3af753fcddfa2a6357aa436a9a4b4863a8ceda5df200: Status 404 returned error can't find the container with id a7023f8db3f182b0a45e3af753fcddfa2a6357aa436a9a4b4863a8ceda5df200
Sep 09 21:00:23.249707 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:23.249668    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5eef90-lbnj4" event={"ID":"fcf974a4-3d84-43b7-883a-dd023afb882b","Type":"ContainerStarted","Data":"c40ac317c2654cf70dc170cf164d33213b81a778d74b604e3bd230f24594cee8"}
Sep 09 21:00:23.250200 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:23.249725    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5eef90-lbnj4" event={"ID":"fcf974a4-3d84-43b7-883a-dd023afb882b","Type":"ContainerStarted","Data":"a7023f8db3f182b0a45e3af753fcddfa2a6357aa436a9a4b4863a8ceda5df200"}
Sep 09 21:00:23.268526 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:23.268477    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5eef90-lbnj4" podStartSLOduration=2.268459576 podStartE2EDuration="2.268459576s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:00:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:00:23.266983283 +0000 UTC m=+1184.894611157" watchObservedRunningTime="2026-09-09 21:00:23.268459576 +0000 UTC m=+1184.896087450"
Sep 09 21:00:24.260113 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:24.260074    2581 generic.go:356] "Generic (PLEG): container finished" podID="fcf974a4-3d84-43b7-883a-dd023afb882b" containerID="c40ac317c2654cf70dc170cf164d33213b81a778d74b604e3bd230f24594cee8" exitCode=0
Sep 09 21:00:24.260510 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:24.260159    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5eef90-lbnj4" event={"ID":"fcf974a4-3d84-43b7-883a-dd023afb882b","Type":"ContainerDied","Data":"c40ac317c2654cf70dc170cf164d33213b81a778d74b604e3bd230f24594cee8"}
Sep 09 21:00:25.408491 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:25.408464    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5eef90-lbnj4"
Sep 09 21:00:25.505634 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:25.505602    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcf974a4-3d84-43b7-883a-dd023afb882b-maas-api-service-ca\") pod \"fcf974a4-3d84-43b7-883a-dd023afb882b\" (UID: \"fcf974a4-3d84-43b7-883a-dd023afb882b\") "
Sep 09 21:00:25.506050 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:25.506021    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fcf974a4-3d84-43b7-883a-dd023afb882b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fcf974a4-3d84-43b7-883a-dd023afb882b" (UID: "fcf974a4-3d84-43b7-883a-dd023afb882b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:00:25.607174 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:25.607051    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcf974a4-3d84-43b7-883a-dd023afb882b-maas-api-service-ca\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:00:26.274793 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:26.274763    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5eef90-lbnj4"
Sep 09 21:00:26.275040 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:26.274800    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5eef90-lbnj4" event={"ID":"fcf974a4-3d84-43b7-883a-dd023afb882b","Type":"ContainerDied","Data":"a7023f8db3f182b0a45e3af753fcddfa2a6357aa436a9a4b4863a8ceda5df200"}
Sep 09 21:00:26.275040 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:26.274847    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a7023f8db3f182b0a45e3af753fcddfa2a6357aa436a9a4b4863a8ceda5df200"
Sep 09 21:00:27.282071 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:27.282042    2581 generic.go:356] "Generic (PLEG): container finished" podID="0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19" containerID="f216c804a79fbbecd1c0b8efb736e59ea13ff0e38cbb24279f3032ae76994df3" exitCode=0
Sep 09 21:00:27.282504 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:27.282131    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d" event={"ID":"0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19","Type":"ContainerDied","Data":"f216c804a79fbbecd1c0b8efb736e59ea13ff0e38cbb24279f3032ae76994df3"}
Sep 09 21:00:27.282561 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:27.282534    2581 scope.go:117] "RemoveContainer" containerID="f216c804a79fbbecd1c0b8efb736e59ea13ff0e38cbb24279f3032ae76994df3"
Sep 09 21:00:27.283756 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:27.283726    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:00:27.283826 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:27.283797    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:00:27.283863 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:27.283830    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:00:28.290022 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:28.289985    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d" event={"ID":"0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19","Type":"ContainerStarted","Data":"846efa2d5a2296e2eff4f2973251fa588abe8b374146e456001b3f2cb1b1b8f6"}
Sep 09 21:00:28.758448 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:28.758413    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d"
Sep 09 21:00:28.758685 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:28.758660    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Sep 09 21:00:28.758779 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:28.758722    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d" podUID="0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Sep 09 21:00:29.758934 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:29.758897    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Sep 09 21:00:29.759381 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:29.758957    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d" podUID="0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Sep 09 21:00:30.758770 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:30.758734    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Sep 09 21:00:30.758974 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:30.758795    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d" podUID="0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Sep 09 21:00:31.758988 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:31.758953    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Sep 09 21:00:31.759405 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:31.759010    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d" podUID="0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Sep 09 21:00:32.036483 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:32.036391    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-5eef90-29816460-djtcd"]
Sep 09 21:00:32.038840 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:32.038808    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-5eef90-29816460-djtcd"]
Sep 09 21:00:32.248192 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:32.248147    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5eef90-lbnj4"]
Sep 09 21:00:32.251228 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:32.251201    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5eef90-lbnj4"]
Sep 09 21:00:32.758657 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:32.758627    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Sep 09 21:00:32.758836 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:32.758683    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d" podUID="0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Sep 09 21:00:32.825248 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:32.825205    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8e464858-076a-44ef-8c80-368eb08ca836" path="/var/lib/kubelet/pods/8e464858-076a-44ef-8c80-368eb08ca836/volumes"
Sep 09 21:00:32.825711 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:32.825690    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fcf974a4-3d84-43b7-883a-dd023afb882b" path="/var/lib/kubelet/pods/fcf974a4-3d84-43b7-883a-dd023afb882b/volumes"
Sep 09 21:00:33.763150 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:33.763091    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d"
Sep 09 21:00:33.763536 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:33.763512    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d"
Sep 09 21:00:33.764323 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:33.764292    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d"
Sep 09 21:00:38.872913 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:38.872884    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-wsv4v_2b1f9a6e-7a79-4e89-ac3f-4658d6d91550/console-operator/1.log"
Sep 09 21:00:38.875535 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:38.875513    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-wsv4v_2b1f9a6e-7a79-4e89-ac3f-4658d6d91550/console-operator/1.log"
Sep 09 21:00:38.880647 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:38.880624    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-75wj4_55b4fbae-29dc-41bf-a907-b1412b852cef/ovn-acl-logging/0.log"
Sep 09 21:00:38.883359 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:38.883340    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-75wj4_55b4fbae-29dc-41bf-a907-b1412b852cef/ovn-acl-logging/0.log"
Sep 09 21:00:39.299782 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:39.299751    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-423e6a-6smh5"]
Sep 09 21:00:39.300210 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:39.300197    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fcf974a4-3d84-43b7-883a-dd023afb882b" containerName="revoke-keys"
Sep 09 21:00:39.300265 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:39.300212    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="fcf974a4-3d84-43b7-883a-dd023afb882b" containerName="revoke-keys"
Sep 09 21:00:39.300305 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:39.300298    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="fcf974a4-3d84-43b7-883a-dd023afb882b" containerName="revoke-keys"
Sep 09 21:00:39.303696 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:39.303680    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-423e6a-6smh5"
Sep 09 21:00:39.306354 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:39.306335    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-glz44\""
Sep 09 21:00:39.313531 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:39.313497    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-423e6a-6smh5"]
Sep 09 21:00:39.457397 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:39.457360    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1af3e7c-583a-41f8-bebe-f66a415dca65-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-423e6a-6smh5\" (UID: \"e1af3e7c-583a-41f8-bebe-f66a415dca65\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-423e6a-6smh5"
Sep 09 21:00:39.558779 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:39.558685    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1af3e7c-583a-41f8-bebe-f66a415dca65-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-423e6a-6smh5\" (UID: \"e1af3e7c-583a-41f8-bebe-f66a415dca65\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-423e6a-6smh5"
Sep 09 21:00:39.559416 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:39.559392    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1af3e7c-583a-41f8-bebe-f66a415dca65-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-423e6a-6smh5\" (UID: \"e1af3e7c-583a-41f8-bebe-f66a415dca65\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-423e6a-6smh5"
Sep 09 21:00:39.615273 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:39.615238    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-423e6a-6smh5"
Sep 09 21:00:39.888210 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:39.888181    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-423e6a-6smh5"]
Sep 09 21:00:39.889869 ip-10-0-129-99 kubenswrapper[2581]: W0909 21:00:39.889845    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1af3e7c_583a_41f8_bebe_f66a415dca65.slice/crio-9e56de5fb9d29921b64232ad3a6746b4aa871fcfbcb539ee8239b319ab0c854c WatchSource:0}: Error finding container 9e56de5fb9d29921b64232ad3a6746b4aa871fcfbcb539ee8239b319ab0c854c: Status 404 returned error can't find the container with id 9e56de5fb9d29921b64232ad3a6746b4aa871fcfbcb539ee8239b319ab0c854c
Sep 09 21:00:40.389539 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:40.389493    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-423e6a-6smh5" event={"ID":"e1af3e7c-583a-41f8-bebe-f66a415dca65","Type":"ContainerStarted","Data":"3b74befa341add30d924710aa0c0104cf089b1eb850a37f3f303a0879d60e26a"}
Sep 09 21:00:40.389719 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:40.389594    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-423e6a-6smh5" event={"ID":"e1af3e7c-583a-41f8-bebe-f66a415dca65","Type":"ContainerStarted","Data":"9e56de5fb9d29921b64232ad3a6746b4aa871fcfbcb539ee8239b319ab0c854c"}
Sep 09 21:00:40.406460 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:40.406413    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-423e6a-6smh5" podStartSLOduration=1.406397212 podStartE2EDuration="1.406397212s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:00:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:00:40.405403841 +0000 UTC m=+1202.033031715" watchObservedRunningTime="2026-09-09 21:00:40.406397212 +0000 UTC m=+1202.034025118"
Sep 09 21:00:41.396838 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:41.396805    2581 generic.go:356] "Generic (PLEG): container finished" podID="e1af3e7c-583a-41f8-bebe-f66a415dca65" containerID="3b74befa341add30d924710aa0c0104cf089b1eb850a37f3f303a0879d60e26a" exitCode=0
Sep 09 21:00:41.397247 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:41.396845    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-423e6a-6smh5" event={"ID":"e1af3e7c-583a-41f8-bebe-f66a415dca65","Type":"ContainerDied","Data":"3b74befa341add30d924710aa0c0104cf089b1eb850a37f3f303a0879d60e26a"}
Sep 09 21:00:42.549084 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:42.549061    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-423e6a-6smh5"
Sep 09 21:00:42.701483 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:42.701388    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1af3e7c-583a-41f8-bebe-f66a415dca65-maas-api-service-ca\") pod \"e1af3e7c-583a-41f8-bebe-f66a415dca65\" (UID: \"e1af3e7c-583a-41f8-bebe-f66a415dca65\") "
Sep 09 21:00:42.701745 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:42.701719    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1af3e7c-583a-41f8-bebe-f66a415dca65-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e1af3e7c-583a-41f8-bebe-f66a415dca65" (UID: "e1af3e7c-583a-41f8-bebe-f66a415dca65"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:00:42.701922 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:42.701902    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1af3e7c-583a-41f8-bebe-f66a415dca65-maas-api-service-ca\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:00:43.409936 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:43.409908    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-423e6a-6smh5"
Sep 09 21:00:43.410167 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:43.409950    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-423e6a-6smh5" event={"ID":"e1af3e7c-583a-41f8-bebe-f66a415dca65","Type":"ContainerDied","Data":"9e56de5fb9d29921b64232ad3a6746b4aa871fcfbcb539ee8239b319ab0c854c"}
Sep 09 21:00:43.410167 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:43.410009    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9e56de5fb9d29921b64232ad3a6746b4aa871fcfbcb539ee8239b319ab0c854c"
Sep 09 21:00:50.306715 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:50.306647    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-423e6a-29816460-kf6wm"]
Sep 09 21:00:50.309230 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:50.309208    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-423e6a-29816460-kf6wm"]
Sep 09 21:00:50.543003 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:50.542967    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-423e6a-6smh5"]
Sep 09 21:00:50.547438 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:50.547408    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-423e6a-6smh5"]
Sep 09 21:00:50.820441 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:50.820410    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f69d8aa-4cee-4e44-a1dc-186c32bdd357" path="/var/lib/kubelet/pods/9f69d8aa-4cee-4e44-a1dc-186c32bdd357/volumes"
Sep 09 21:00:50.820761 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:50.820738    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1af3e7c-583a-41f8-bebe-f66a415dca65" path="/var/lib/kubelet/pods/e1af3e7c-583a-41f8-bebe-f66a415dca65/volumes"
Sep 09 21:00:52.998457 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:52.998425    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm"]
Sep 09 21:00:52.998855 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:52.998638    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm" podUID="5fd3a041-c3a6-43cb-b02c-55edd6d4fbff" containerName="istio-proxy" containerID="cri-o://726838450e3a0216431d27dfb59da3a0caef7e5b8437113163ac7d40e44575fa" gracePeriod=30
Sep 09 21:00:56.941879 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:56.941845    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx"]
Sep 09 21:00:56.942383 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:56.942061    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx" podUID="67436bca-5ef3-400f-9892-6d1e3403349b" containerName="istio-proxy" containerID="cri-o://244a3d0fa614f74a6432e2e15e01f17d36ec41e9e57c66e305ef3b9ed2e5c14e" gracePeriod=30
Sep 09 21:00:58.254982 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:58.254959    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm"
Sep 09 21:00:58.387749 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:58.387658    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5fd3a041-c3a6-43cb-b02c-55edd6d4fbff-istiod-ca-cert\") pod \"5fd3a041-c3a6-43cb-b02c-55edd6d4fbff\" (UID: \"5fd3a041-c3a6-43cb-b02c-55edd6d4fbff\") "
Sep 09 21:00:58.387910 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:58.387789    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5fd3a041-c3a6-43cb-b02c-55edd6d4fbff-workload-certs\") pod \"5fd3a041-c3a6-43cb-b02c-55edd6d4fbff\" (UID: \"5fd3a041-c3a6-43cb-b02c-55edd6d4fbff\") "
Sep 09 21:00:58.387910 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:58.387826    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5fd3a041-c3a6-43cb-b02c-55edd6d4fbff-istio-envoy\") pod \"5fd3a041-c3a6-43cb-b02c-55edd6d4fbff\" (UID: \"5fd3a041-c3a6-43cb-b02c-55edd6d4fbff\") "
Sep 09 21:00:58.387910 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:58.387860    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fd3a041-c3a6-43cb-b02c-55edd6d4fbff-credential-socket\") pod \"5fd3a041-c3a6-43cb-b02c-55edd6d4fbff\" (UID: \"5fd3a041-c3a6-43cb-b02c-55edd6d4fbff\") "
Sep 09 21:00:58.388096 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:58.387916    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-56rrb\" (UniqueName: \"kubernetes.io/projected/5fd3a041-c3a6-43cb-b02c-55edd6d4fbff-kube-api-access-56rrb\") pod \"5fd3a041-c3a6-43cb-b02c-55edd6d4fbff\" (UID: \"5fd3a041-c3a6-43cb-b02c-55edd6d4fbff\") "
Sep 09 21:00:58.388096 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:58.387952    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5fd3a041-c3a6-43cb-b02c-55edd6d4fbff-istio-data\") pod \"5fd3a041-c3a6-43cb-b02c-55edd6d4fbff\" (UID: \"5fd3a041-c3a6-43cb-b02c-55edd6d4fbff\") "
Sep 09 21:00:58.388096 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:58.387975    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5fd3a041-c3a6-43cb-b02c-55edd6d4fbff-istio-token\") pod \"5fd3a041-c3a6-43cb-b02c-55edd6d4fbff\" (UID: \"5fd3a041-c3a6-43cb-b02c-55edd6d4fbff\") "
Sep 09 21:00:58.388096 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:58.388026    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5fd3a041-c3a6-43cb-b02c-55edd6d4fbff-istio-podinfo\") pod \"5fd3a041-c3a6-43cb-b02c-55edd6d4fbff\" (UID: \"5fd3a041-c3a6-43cb-b02c-55edd6d4fbff\") "
Sep 09 21:00:58.388096 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:58.388042    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5fd3a041-c3a6-43cb-b02c-55edd6d4fbff-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5fd3a041-c3a6-43cb-b02c-55edd6d4fbff" (UID: "5fd3a041-c3a6-43cb-b02c-55edd6d4fbff"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:00:58.388096 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:58.388066    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fd3a041-c3a6-43cb-b02c-55edd6d4fbff-workload-socket\") pod \"5fd3a041-c3a6-43cb-b02c-55edd6d4fbff\" (UID: \"5fd3a041-c3a6-43cb-b02c-55edd6d4fbff\") "
Sep 09 21:00:58.388096 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:58.388078    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fd3a041-c3a6-43cb-b02c-55edd6d4fbff-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5fd3a041-c3a6-43cb-b02c-55edd6d4fbff" (UID: "5fd3a041-c3a6-43cb-b02c-55edd6d4fbff"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:00:58.388515 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:58.388211    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fd3a041-c3a6-43cb-b02c-55edd6d4fbff-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5fd3a041-c3a6-43cb-b02c-55edd6d4fbff" (UID: "5fd3a041-c3a6-43cb-b02c-55edd6d4fbff"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:00:58.388515 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:58.388234    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fd3a041-c3a6-43cb-b02c-55edd6d4fbff-istio-data" (OuterVolumeSpecName: "istio-data") pod "5fd3a041-c3a6-43cb-b02c-55edd6d4fbff" (UID: "5fd3a041-c3a6-43cb-b02c-55edd6d4fbff"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:00:58.388515 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:58.388427    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fd3a041-c3a6-43cb-b02c-55edd6d4fbff-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5fd3a041-c3a6-43cb-b02c-55edd6d4fbff" (UID: "5fd3a041-c3a6-43cb-b02c-55edd6d4fbff"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:00:58.388680 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:58.388638    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5fd3a041-c3a6-43cb-b02c-55edd6d4fbff-workload-certs\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:00:58.388680 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:58.388657    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fd3a041-c3a6-43cb-b02c-55edd6d4fbff-credential-socket\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:00:58.388680 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:58.388672    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5fd3a041-c3a6-43cb-b02c-55edd6d4fbff-istio-data\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:00:58.388838 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:58.388686    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fd3a041-c3a6-43cb-b02c-55edd6d4fbff-workload-socket\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:00:58.388838 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:58.388701    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5fd3a041-c3a6-43cb-b02c-55edd6d4fbff-istiod-ca-cert\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:00:58.390238 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:58.390210    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fd3a041-c3a6-43cb-b02c-55edd6d4fbff-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5fd3a041-c3a6-43cb-b02c-55edd6d4fbff" (UID: "5fd3a041-c3a6-43cb-b02c-55edd6d4fbff"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:00:58.390238 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:58.390227    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5fd3a041-c3a6-43cb-b02c-55edd6d4fbff-kube-api-access-56rrb" (OuterVolumeSpecName: "kube-api-access-56rrb") pod "5fd3a041-c3a6-43cb-b02c-55edd6d4fbff" (UID: "5fd3a041-c3a6-43cb-b02c-55edd6d4fbff"). InnerVolumeSpecName "kube-api-access-56rrb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:00:58.390392 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:58.390244    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5fd3a041-c3a6-43cb-b02c-55edd6d4fbff-istio-token" (OuterVolumeSpecName: "istio-token") pod "5fd3a041-c3a6-43cb-b02c-55edd6d4fbff" (UID: "5fd3a041-c3a6-43cb-b02c-55edd6d4fbff"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:00:58.390392 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:58.390246    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5fd3a041-c3a6-43cb-b02c-55edd6d4fbff-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5fd3a041-c3a6-43cb-b02c-55edd6d4fbff" (UID: "5fd3a041-c3a6-43cb-b02c-55edd6d4fbff"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 21:00:58.489810 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:58.489770    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5fd3a041-c3a6-43cb-b02c-55edd6d4fbff-istio-envoy\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:00:58.489810 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:58.489804    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-56rrb\" (UniqueName: \"kubernetes.io/projected/5fd3a041-c3a6-43cb-b02c-55edd6d4fbff-kube-api-access-56rrb\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:00:58.489810 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:58.489817    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5fd3a041-c3a6-43cb-b02c-55edd6d4fbff-istio-token\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:00:58.490058 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:58.489831    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5fd3a041-c3a6-43cb-b02c-55edd6d4fbff-istio-podinfo\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:00:58.501146 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:58.501091    2581 generic.go:356] "Generic (PLEG): container finished" podID="5fd3a041-c3a6-43cb-b02c-55edd6d4fbff" containerID="726838450e3a0216431d27dfb59da3a0caef7e5b8437113163ac7d40e44575fa" exitCode=0
Sep 09 21:00:58.501146 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:58.501136    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm" event={"ID":"5fd3a041-c3a6-43cb-b02c-55edd6d4fbff","Type":"ContainerDied","Data":"726838450e3a0216431d27dfb59da3a0caef7e5b8437113163ac7d40e44575fa"}
Sep 09 21:00:58.501318 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:58.501197    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm" event={"ID":"5fd3a041-c3a6-43cb-b02c-55edd6d4fbff","Type":"ContainerDied","Data":"e3de29c12a20e834b048db16756b560899409049d044cec5b321d7d2c3347f5f"}
Sep 09 21:00:58.501318 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:58.501221    2581 scope.go:117] "RemoveContainer" containerID="726838450e3a0216431d27dfb59da3a0caef7e5b8437113163ac7d40e44575fa"
Sep 09 21:00:58.501318 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:58.501150    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm"
Sep 09 21:00:58.511394 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:58.511379    2581 scope.go:117] "RemoveContainer" containerID="726838450e3a0216431d27dfb59da3a0caef7e5b8437113163ac7d40e44575fa"
Sep 09 21:00:58.511662 ip-10-0-129-99 kubenswrapper[2581]: E0909 21:00:58.511644    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"726838450e3a0216431d27dfb59da3a0caef7e5b8437113163ac7d40e44575fa\": container with ID starting with 726838450e3a0216431d27dfb59da3a0caef7e5b8437113163ac7d40e44575fa not found: ID does not exist" containerID="726838450e3a0216431d27dfb59da3a0caef7e5b8437113163ac7d40e44575fa"
Sep 09 21:00:58.511731 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:58.511667    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"726838450e3a0216431d27dfb59da3a0caef7e5b8437113163ac7d40e44575fa"} err="failed to get container status \"726838450e3a0216431d27dfb59da3a0caef7e5b8437113163ac7d40e44575fa\": rpc error: code = NotFound desc = could not find container \"726838450e3a0216431d27dfb59da3a0caef7e5b8437113163ac7d40e44575fa\": container with ID starting with 726838450e3a0216431d27dfb59da3a0caef7e5b8437113163ac7d40e44575fa not found: ID does not exist"
Sep 09 21:00:58.534821 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:58.534785    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm"]
Sep 09 21:00:58.539695 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:58.539671    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-noten-e7c6aa-openshift-default-dd66d99b6-zb4sm"]
Sep 09 21:00:58.820660 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:00:58.820625    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5fd3a041-c3a6-43cb-b02c-55edd6d4fbff" path="/var/lib/kubelet/pods/5fd3a041-c3a6-43cb-b02c-55edd6d4fbff/volumes"
Sep 09 21:01:02.204041 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:02.203978    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx"
Sep 09 21:01:02.326655 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:02.326616    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67436bca-5ef3-400f-9892-6d1e3403349b-workload-socket\") pod \"67436bca-5ef3-400f-9892-6d1e3403349b\" (UID: \"67436bca-5ef3-400f-9892-6d1e3403349b\") "
Sep 09 21:01:02.326837 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:02.326696    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67436bca-5ef3-400f-9892-6d1e3403349b-istiod-ca-cert\") pod \"67436bca-5ef3-400f-9892-6d1e3403349b\" (UID: \"67436bca-5ef3-400f-9892-6d1e3403349b\") "
Sep 09 21:01:02.326837 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:02.326718    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k4ft7\" (UniqueName: \"kubernetes.io/projected/67436bca-5ef3-400f-9892-6d1e3403349b-kube-api-access-k4ft7\") pod \"67436bca-5ef3-400f-9892-6d1e3403349b\" (UID: \"67436bca-5ef3-400f-9892-6d1e3403349b\") "
Sep 09 21:01:02.326837 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:02.326779    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67436bca-5ef3-400f-9892-6d1e3403349b-credential-socket\") pod \"67436bca-5ef3-400f-9892-6d1e3403349b\" (UID: \"67436bca-5ef3-400f-9892-6d1e3403349b\") "
Sep 09 21:01:02.327001 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:02.326957    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67436bca-5ef3-400f-9892-6d1e3403349b-workload-certs\") pod \"67436bca-5ef3-400f-9892-6d1e3403349b\" (UID: \"67436bca-5ef3-400f-9892-6d1e3403349b\") "
Sep 09 21:01:02.327075 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:02.327013    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67436bca-5ef3-400f-9892-6d1e3403349b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "67436bca-5ef3-400f-9892-6d1e3403349b" (UID: "67436bca-5ef3-400f-9892-6d1e3403349b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:01:02.327171 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:02.327049    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67436bca-5ef3-400f-9892-6d1e3403349b-istio-podinfo\") pod \"67436bca-5ef3-400f-9892-6d1e3403349b\" (UID: \"67436bca-5ef3-400f-9892-6d1e3403349b\") "
Sep 09 21:01:02.327171 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:02.327077    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67436bca-5ef3-400f-9892-6d1e3403349b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "67436bca-5ef3-400f-9892-6d1e3403349b" (UID: "67436bca-5ef3-400f-9892-6d1e3403349b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:01:02.327287 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:02.327161    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67436bca-5ef3-400f-9892-6d1e3403349b-istio-data\") pod \"67436bca-5ef3-400f-9892-6d1e3403349b\" (UID: \"67436bca-5ef3-400f-9892-6d1e3403349b\") "
Sep 09 21:01:02.327287 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:02.327209    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67436bca-5ef3-400f-9892-6d1e3403349b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "67436bca-5ef3-400f-9892-6d1e3403349b" (UID: "67436bca-5ef3-400f-9892-6d1e3403349b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:01:02.327287 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:02.327216    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67436bca-5ef3-400f-9892-6d1e3403349b-istio-token\") pod \"67436bca-5ef3-400f-9892-6d1e3403349b\" (UID: \"67436bca-5ef3-400f-9892-6d1e3403349b\") "
Sep 09 21:01:02.327287 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:02.327255    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67436bca-5ef3-400f-9892-6d1e3403349b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "67436bca-5ef3-400f-9892-6d1e3403349b" (UID: "67436bca-5ef3-400f-9892-6d1e3403349b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:01:02.327287 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:02.327284    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67436bca-5ef3-400f-9892-6d1e3403349b-istio-envoy\") pod \"67436bca-5ef3-400f-9892-6d1e3403349b\" (UID: \"67436bca-5ef3-400f-9892-6d1e3403349b\") "
Sep 09 21:01:02.327560 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:02.327400    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67436bca-5ef3-400f-9892-6d1e3403349b-istio-data" (OuterVolumeSpecName: "istio-data") pod "67436bca-5ef3-400f-9892-6d1e3403349b" (UID: "67436bca-5ef3-400f-9892-6d1e3403349b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:01:02.327834 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:02.327812    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67436bca-5ef3-400f-9892-6d1e3403349b-credential-socket\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:01:02.327919 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:02.327840    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67436bca-5ef3-400f-9892-6d1e3403349b-workload-certs\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:01:02.327919 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:02.327855    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67436bca-5ef3-400f-9892-6d1e3403349b-istio-data\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:01:02.327919 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:02.327868    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67436bca-5ef3-400f-9892-6d1e3403349b-workload-socket\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:01:02.327919 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:02.327882    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67436bca-5ef3-400f-9892-6d1e3403349b-istiod-ca-cert\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:01:02.329135 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:02.329088    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67436bca-5ef3-400f-9892-6d1e3403349b-kube-api-access-k4ft7" (OuterVolumeSpecName: "kube-api-access-k4ft7") pod "67436bca-5ef3-400f-9892-6d1e3403349b" (UID: "67436bca-5ef3-400f-9892-6d1e3403349b"). InnerVolumeSpecName "kube-api-access-k4ft7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:01:02.329263 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:02.329243    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/67436bca-5ef3-400f-9892-6d1e3403349b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "67436bca-5ef3-400f-9892-6d1e3403349b" (UID: "67436bca-5ef3-400f-9892-6d1e3403349b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 21:01:02.329374 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:02.329356    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67436bca-5ef3-400f-9892-6d1e3403349b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "67436bca-5ef3-400f-9892-6d1e3403349b" (UID: "67436bca-5ef3-400f-9892-6d1e3403349b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:01:02.329572 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:02.329550    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67436bca-5ef3-400f-9892-6d1e3403349b-istio-token" (OuterVolumeSpecName: "istio-token") pod "67436bca-5ef3-400f-9892-6d1e3403349b" (UID: "67436bca-5ef3-400f-9892-6d1e3403349b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:01:02.428616 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:02.428586    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67436bca-5ef3-400f-9892-6d1e3403349b-istio-envoy\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:01:02.428616 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:02.428614    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k4ft7\" (UniqueName: \"kubernetes.io/projected/67436bca-5ef3-400f-9892-6d1e3403349b-kube-api-access-k4ft7\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:01:02.428796 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:02.428626    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67436bca-5ef3-400f-9892-6d1e3403349b-istio-podinfo\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:01:02.428796 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:02.428635    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67436bca-5ef3-400f-9892-6d1e3403349b-istio-token\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:01:02.529777 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:02.529742    2581 generic.go:356] "Generic (PLEG): container finished" podID="67436bca-5ef3-400f-9892-6d1e3403349b" containerID="244a3d0fa614f74a6432e2e15e01f17d36ec41e9e57c66e305ef3b9ed2e5c14e" exitCode=0
Sep 09 21:01:02.529961 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:02.529791    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx"
Sep 09 21:01:02.529961 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:02.529817    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx" event={"ID":"67436bca-5ef3-400f-9892-6d1e3403349b","Type":"ContainerDied","Data":"244a3d0fa614f74a6432e2e15e01f17d36ec41e9e57c66e305ef3b9ed2e5c14e"}
Sep 09 21:01:02.529961 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:02.529897    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx" event={"ID":"67436bca-5ef3-400f-9892-6d1e3403349b","Type":"ContainerDied","Data":"5c0882badf99b1d7f2b9b63f2ca40a82fc93dc3e86b565056ec453d5d0d0231b"}
Sep 09 21:01:02.529961 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:02.529923    2581 scope.go:117] "RemoveContainer" containerID="244a3d0fa614f74a6432e2e15e01f17d36ec41e9e57c66e305ef3b9ed2e5c14e"
Sep 09 21:01:02.540886 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:02.540864    2581 scope.go:117] "RemoveContainer" containerID="81fa1fe91f1684b322720d3bdeed556f730e4e2a53df649adcb43beaa9c74660"
Sep 09 21:01:02.549680 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:02.549660    2581 scope.go:117] "RemoveContainer" containerID="244a3d0fa614f74a6432e2e15e01f17d36ec41e9e57c66e305ef3b9ed2e5c14e"
Sep 09 21:01:02.550048 ip-10-0-129-99 kubenswrapper[2581]: E0909 21:01:02.550006    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"244a3d0fa614f74a6432e2e15e01f17d36ec41e9e57c66e305ef3b9ed2e5c14e\": container with ID starting with 244a3d0fa614f74a6432e2e15e01f17d36ec41e9e57c66e305ef3b9ed2e5c14e not found: ID does not exist" containerID="244a3d0fa614f74a6432e2e15e01f17d36ec41e9e57c66e305ef3b9ed2e5c14e"
Sep 09 21:01:02.550181 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:02.550042    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"244a3d0fa614f74a6432e2e15e01f17d36ec41e9e57c66e305ef3b9ed2e5c14e"} err="failed to get container status \"244a3d0fa614f74a6432e2e15e01f17d36ec41e9e57c66e305ef3b9ed2e5c14e\": rpc error: code = NotFound desc = could not find container \"244a3d0fa614f74a6432e2e15e01f17d36ec41e9e57c66e305ef3b9ed2e5c14e\": container with ID starting with 244a3d0fa614f74a6432e2e15e01f17d36ec41e9e57c66e305ef3b9ed2e5c14e not found: ID does not exist"
Sep 09 21:01:02.550181 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:02.550068    2581 scope.go:117] "RemoveContainer" containerID="81fa1fe91f1684b322720d3bdeed556f730e4e2a53df649adcb43beaa9c74660"
Sep 09 21:01:02.550586 ip-10-0-129-99 kubenswrapper[2581]: E0909 21:01:02.550561    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81fa1fe91f1684b322720d3bdeed556f730e4e2a53df649adcb43beaa9c74660\": container with ID starting with 81fa1fe91f1684b322720d3bdeed556f730e4e2a53df649adcb43beaa9c74660 not found: ID does not exist" containerID="81fa1fe91f1684b322720d3bdeed556f730e4e2a53df649adcb43beaa9c74660"
Sep 09 21:01:02.550680 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:02.550595    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81fa1fe91f1684b322720d3bdeed556f730e4e2a53df649adcb43beaa9c74660"} err="failed to get container status \"81fa1fe91f1684b322720d3bdeed556f730e4e2a53df649adcb43beaa9c74660\": rpc error: code = NotFound desc = could not find container \"81fa1fe91f1684b322720d3bdeed556f730e4e2a53df649adcb43beaa9c74660\": container with ID starting with 81fa1fe91f1684b322720d3bdeed556f730e4e2a53df649adcb43beaa9c74660 not found: ID does not exist"
Sep 09 21:01:02.553521 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:02.553501    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx"]
Sep 09 21:01:02.560702 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:02.560678    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-423e6a-openshift-default-766d9b8f86-c2dbx"]
Sep 09 21:01:02.763153 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:02.763119    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-615c6f-778b5c6b5d-qk8b7"]
Sep 09 21:01:02.763561 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:02.763548    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1af3e7c-583a-41f8-bebe-f66a415dca65" containerName="revoke-keys"
Sep 09 21:01:02.763601 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:02.763562    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1af3e7c-583a-41f8-bebe-f66a415dca65" containerName="revoke-keys"
Sep 09 21:01:02.763601 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:02.763575    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5fd3a041-c3a6-43cb-b02c-55edd6d4fbff" containerName="istio-proxy"
Sep 09 21:01:02.763601 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:02.763580    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="5fd3a041-c3a6-43cb-b02c-55edd6d4fbff" containerName="istio-proxy"
Sep 09 21:01:02.763601 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:02.763592    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67436bca-5ef3-400f-9892-6d1e3403349b" containerName="istio-proxy"
Sep 09 21:01:02.763601 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:02.763598    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="67436bca-5ef3-400f-9892-6d1e3403349b" containerName="istio-proxy"
Sep 09 21:01:02.763758 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:02.763608    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67436bca-5ef3-400f-9892-6d1e3403349b" containerName="istio-proxy"
Sep 09 21:01:02.763758 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:02.763615    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="67436bca-5ef3-400f-9892-6d1e3403349b" containerName="istio-proxy"
Sep 09 21:01:02.763758 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:02.763674    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="67436bca-5ef3-400f-9892-6d1e3403349b" containerName="istio-proxy"
Sep 09 21:01:02.763758 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:02.763683    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="67436bca-5ef3-400f-9892-6d1e3403349b" containerName="istio-proxy"
Sep 09 21:01:02.763758 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:02.763689    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="5fd3a041-c3a6-43cb-b02c-55edd6d4fbff" containerName="istio-proxy"
Sep 09 21:01:02.763758 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:02.763700    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1af3e7c-583a-41f8-bebe-f66a415dca65" containerName="revoke-keys"
Sep 09 21:01:02.768454 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:02.768435    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-615c6f-778b5c6b5d-qk8b7"
Sep 09 21:01:02.771308 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:02.771287    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-615c6f\""
Sep 09 21:01:02.771405 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:02.771287    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-615c6f-dockercfg-8x69k\""
Sep 09 21:01:02.776030 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:02.776003    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-615c6f-778b5c6b5d-qk8b7"]
Sep 09 21:01:02.830888 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:02.824737    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67436bca-5ef3-400f-9892-6d1e3403349b" path="/var/lib/kubelet/pods/67436bca-5ef3-400f-9892-6d1e3403349b/volumes"
Sep 09 21:01:02.849487 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:02.849452    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-615c6f-6d489cd67c-twlch"]
Sep 09 21:01:02.853426 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:02.853406    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-615c6f-6d489cd67c-twlch"
Sep 09 21:01:02.867759 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:02.867736    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-615c6f-6d489cd67c-twlch"]
Sep 09 21:01:02.934596 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:02.934566    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9cc52881-d895-4ab9-8bd3-49273342b95c-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-615c6f-778b5c6b5d-qk8b7\" (UID: \"9cc52881-d895-4ab9-8bd3-49273342b95c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-615c6f-778b5c6b5d-qk8b7"
Sep 09 21:01:02.934750 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:02.934611    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xf7sz\" (UniqueName: \"kubernetes.io/projected/9cc52881-d895-4ab9-8bd3-49273342b95c-kube-api-access-xf7sz\") pod \"payload-pre-processing-e2e-shared-a-w3-615c6f-778b5c6b5d-qk8b7\" (UID: \"9cc52881-d895-4ab9-8bd3-49273342b95c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-615c6f-778b5c6b5d-qk8b7"
Sep 09 21:01:02.934750 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:02.934701    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e02cebef-4376-4266-b958-2556fd55e0c3-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-615c6f-6d489cd67c-twlch\" (UID: \"e02cebef-4376-4266-b958-2556fd55e0c3\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-615c6f-6d489cd67c-twlch"
Sep 09 21:01:02.934896 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:02.934763    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vz8tb\" (UniqueName: \"kubernetes.io/projected/e02cebef-4376-4266-b958-2556fd55e0c3-kube-api-access-vz8tb\") pod \"payload-processing-e2e-shared-a-w3-615c6f-6d489cd67c-twlch\" (UID: \"e02cebef-4376-4266-b958-2556fd55e0c3\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-615c6f-6d489cd67c-twlch"
Sep 09 21:01:03.035831 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:03.035806    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vz8tb\" (UniqueName: \"kubernetes.io/projected/e02cebef-4376-4266-b958-2556fd55e0c3-kube-api-access-vz8tb\") pod \"payload-processing-e2e-shared-a-w3-615c6f-6d489cd67c-twlch\" (UID: \"e02cebef-4376-4266-b958-2556fd55e0c3\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-615c6f-6d489cd67c-twlch"
Sep 09 21:01:03.035988 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:03.035871    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9cc52881-d895-4ab9-8bd3-49273342b95c-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-615c6f-778b5c6b5d-qk8b7\" (UID: \"9cc52881-d895-4ab9-8bd3-49273342b95c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-615c6f-778b5c6b5d-qk8b7"
Sep 09 21:01:03.035988 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:03.035899    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xf7sz\" (UniqueName: \"kubernetes.io/projected/9cc52881-d895-4ab9-8bd3-49273342b95c-kube-api-access-xf7sz\") pod \"payload-pre-processing-e2e-shared-a-w3-615c6f-778b5c6b5d-qk8b7\" (UID: \"9cc52881-d895-4ab9-8bd3-49273342b95c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-615c6f-778b5c6b5d-qk8b7"
Sep 09 21:01:03.035988 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:03.035944    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e02cebef-4376-4266-b958-2556fd55e0c3-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-615c6f-6d489cd67c-twlch\" (UID: \"e02cebef-4376-4266-b958-2556fd55e0c3\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-615c6f-6d489cd67c-twlch"
Sep 09 21:01:03.036684 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:03.036664    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9cc52881-d895-4ab9-8bd3-49273342b95c-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-615c6f-778b5c6b5d-qk8b7\" (UID: \"9cc52881-d895-4ab9-8bd3-49273342b95c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-615c6f-778b5c6b5d-qk8b7"
Sep 09 21:01:03.036733 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:03.036662    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e02cebef-4376-4266-b958-2556fd55e0c3-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-615c6f-6d489cd67c-twlch\" (UID: \"e02cebef-4376-4266-b958-2556fd55e0c3\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-615c6f-6d489cd67c-twlch"
Sep 09 21:01:03.044090 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:03.044064    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vz8tb\" (UniqueName: \"kubernetes.io/projected/e02cebef-4376-4266-b958-2556fd55e0c3-kube-api-access-vz8tb\") pod \"payload-processing-e2e-shared-a-w3-615c6f-6d489cd67c-twlch\" (UID: \"e02cebef-4376-4266-b958-2556fd55e0c3\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-615c6f-6d489cd67c-twlch"
Sep 09 21:01:03.044198 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:03.044171    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xf7sz\" (UniqueName: \"kubernetes.io/projected/9cc52881-d895-4ab9-8bd3-49273342b95c-kube-api-access-xf7sz\") pod \"payload-pre-processing-e2e-shared-a-w3-615c6f-778b5c6b5d-qk8b7\" (UID: \"9cc52881-d895-4ab9-8bd3-49273342b95c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-615c6f-778b5c6b5d-qk8b7"
Sep 09 21:01:03.082246 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:03.082170    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-615c6f-778b5c6b5d-qk8b7"
Sep 09 21:01:03.163612 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:03.163568    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-615c6f-6d489cd67c-twlch"
Sep 09 21:01:03.357827 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:03.357798    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-615c6f-778b5c6b5d-qk8b7"]
Sep 09 21:01:03.359344 ip-10-0-129-99 kubenswrapper[2581]: W0909 21:01:03.359321    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cc52881_d895_4ab9_8bd3_49273342b95c.slice/crio-eb54b0bc9b6046538e40c28d91706d16bcc0167d14d87118a57275c7f4abd296 WatchSource:0}: Error finding container eb54b0bc9b6046538e40c28d91706d16bcc0167d14d87118a57275c7f4abd296: Status 404 returned error can't find the container with id eb54b0bc9b6046538e40c28d91706d16bcc0167d14d87118a57275c7f4abd296
Sep 09 21:01:03.438353 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:03.438321    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-615c6f-6d489cd67c-twlch"]
Sep 09 21:01:03.440096 ip-10-0-129-99 kubenswrapper[2581]: W0909 21:01:03.440065    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode02cebef_4376_4266_b958_2556fd55e0c3.slice/crio-1cfd8d2acb79f735b2843f7f7df379277602bd95bda3f8af12a1f8e6901f930f WatchSource:0}: Error finding container 1cfd8d2acb79f735b2843f7f7df379277602bd95bda3f8af12a1f8e6901f930f: Status 404 returned error can't find the container with id 1cfd8d2acb79f735b2843f7f7df379277602bd95bda3f8af12a1f8e6901f930f
Sep 09 21:01:03.541530 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:03.541489    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-615c6f-6d489cd67c-twlch" event={"ID":"e02cebef-4376-4266-b958-2556fd55e0c3","Type":"ContainerStarted","Data":"4533f96f04c5045c1e05d8777ae7ad4441ef6c39209a0d502a4483062087a7d0"}
Sep 09 21:01:03.541771 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:03.541563    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-615c6f-6d489cd67c-twlch" event={"ID":"e02cebef-4376-4266-b958-2556fd55e0c3","Type":"ContainerStarted","Data":"1cfd8d2acb79f735b2843f7f7df379277602bd95bda3f8af12a1f8e6901f930f"}
Sep 09 21:01:03.541771 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:03.541593    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-615c6f-6d489cd67c-twlch"
Sep 09 21:01:03.543167 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:03.543142    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-615c6f-778b5c6b5d-qk8b7" event={"ID":"9cc52881-d895-4ab9-8bd3-49273342b95c","Type":"ContainerStarted","Data":"1c35a6dda8cdfab997341e84ce98760d0fc5930d75b65036cb805bc467d2a356"}
Sep 09 21:01:03.543266 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:03.543188    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-615c6f-778b5c6b5d-qk8b7" event={"ID":"9cc52881-d895-4ab9-8bd3-49273342b95c","Type":"ContainerStarted","Data":"eb54b0bc9b6046538e40c28d91706d16bcc0167d14d87118a57275c7f4abd296"}
Sep 09 21:01:03.543266 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:03.543215    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-615c6f-778b5c6b5d-qk8b7"
Sep 09 21:01:03.560984 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:03.560930    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-615c6f-6d489cd67c-twlch" podStartSLOduration=1.5609099 podStartE2EDuration="1.5609099s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:01:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:01:03.559182538 +0000 UTC m=+1225.186810411" watchObservedRunningTime="2026-09-09 21:01:03.5609099 +0000 UTC m=+1225.188537775"
Sep 09 21:01:03.576010 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:03.575963    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-615c6f-778b5c6b5d-qk8b7" podStartSLOduration=1.575949558 podStartE2EDuration="1.575949558s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:01:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:01:03.574393207 +0000 UTC m=+1225.202021081" watchObservedRunningTime="2026-09-09 21:01:03.575949558 +0000 UTC m=+1225.203577431"
Sep 09 21:01:14.551326 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:14.551289    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-615c6f-778b5c6b5d-qk8b7"
Sep 09 21:01:14.551938 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:14.551613    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-615c6f-6d489cd67c-twlch"
Sep 09 21:01:26.283384 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:26.283350    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm"]
Sep 09 21:01:26.287344 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:26.287327    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm"
Sep 09 21:01:26.292916 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:26.292894    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-0b5d4c-openshift-default-dockercfg-ftpzt\""
Sep 09 21:01:26.313778 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:26.313751    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm"]
Sep 09 21:01:26.382222 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:26.382188    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbb20d4d-d468-4972-ac78-1678e69b548b-credential-socket\") pod \"e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm\" (UID: \"cbb20d4d-d468-4972-ac78-1678e69b548b\") " pod="openshift-ingress/e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm"
Sep 09 21:01:26.382545 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:26.382523    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cbb20d4d-d468-4972-ac78-1678e69b548b-istio-podinfo\") pod \"e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm\" (UID: \"cbb20d4d-d468-4972-ac78-1678e69b548b\") " pod="openshift-ingress/e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm"
Sep 09 21:01:26.382733 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:26.382713    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cbb20d4d-d468-4972-ac78-1678e69b548b-workload-certs\") pod \"e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm\" (UID: \"cbb20d4d-d468-4972-ac78-1678e69b548b\") " pod="openshift-ingress/e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm"
Sep 09 21:01:26.382882 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:26.382865    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbb20d4d-d468-4972-ac78-1678e69b548b-workload-socket\") pod \"e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm\" (UID: \"cbb20d4d-d468-4972-ac78-1678e69b548b\") " pod="openshift-ingress/e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm"
Sep 09 21:01:26.383037 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:26.383016    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cbb20d4d-d468-4972-ac78-1678e69b548b-istio-envoy\") pod \"e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm\" (UID: \"cbb20d4d-d468-4972-ac78-1678e69b548b\") " pod="openshift-ingress/e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm"
Sep 09 21:01:26.383176 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:26.383160    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vj2w5\" (UniqueName: \"kubernetes.io/projected/cbb20d4d-d468-4972-ac78-1678e69b548b-kube-api-access-vj2w5\") pod \"e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm\" (UID: \"cbb20d4d-d468-4972-ac78-1678e69b548b\") " pod="openshift-ingress/e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm"
Sep 09 21:01:26.383344 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:26.383323    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cbb20d4d-d468-4972-ac78-1678e69b548b-istio-data\") pod \"e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm\" (UID: \"cbb20d4d-d468-4972-ac78-1678e69b548b\") " pod="openshift-ingress/e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm"
Sep 09 21:01:26.383497 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:26.383483    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cbb20d4d-d468-4972-ac78-1678e69b548b-istiod-ca-cert\") pod \"e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm\" (UID: \"cbb20d4d-d468-4972-ac78-1678e69b548b\") " pod="openshift-ingress/e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm"
Sep 09 21:01:26.383616 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:26.383598    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cbb20d4d-d468-4972-ac78-1678e69b548b-istio-token\") pod \"e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm\" (UID: \"cbb20d4d-d468-4972-ac78-1678e69b548b\") " pod="openshift-ingress/e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm"
Sep 09 21:01:26.484622 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:26.484579    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cbb20d4d-d468-4972-ac78-1678e69b548b-istio-podinfo\") pod \"e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm\" (UID: \"cbb20d4d-d468-4972-ac78-1678e69b548b\") " pod="openshift-ingress/e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm"
Sep 09 21:01:26.484805 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:26.484653    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cbb20d4d-d468-4972-ac78-1678e69b548b-workload-certs\") pod \"e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm\" (UID: \"cbb20d4d-d468-4972-ac78-1678e69b548b\") " pod="openshift-ingress/e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm"
Sep 09 21:01:26.484805 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:26.484679    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbb20d4d-d468-4972-ac78-1678e69b548b-workload-socket\") pod \"e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm\" (UID: \"cbb20d4d-d468-4972-ac78-1678e69b548b\") " pod="openshift-ingress/e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm"
Sep 09 21:01:26.484805 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:26.484720    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cbb20d4d-d468-4972-ac78-1678e69b548b-istio-envoy\") pod \"e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm\" (UID: \"cbb20d4d-d468-4972-ac78-1678e69b548b\") " pod="openshift-ingress/e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm"
Sep 09 21:01:26.484805 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:26.484754    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vj2w5\" (UniqueName: \"kubernetes.io/projected/cbb20d4d-d468-4972-ac78-1678e69b548b-kube-api-access-vj2w5\") pod \"e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm\" (UID: \"cbb20d4d-d468-4972-ac78-1678e69b548b\") " pod="openshift-ingress/e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm"
Sep 09 21:01:26.484991 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:26.484814    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cbb20d4d-d468-4972-ac78-1678e69b548b-istio-data\") pod \"e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm\" (UID: \"cbb20d4d-d468-4972-ac78-1678e69b548b\") " pod="openshift-ingress/e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm"
Sep 09 21:01:26.484991 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:26.484970    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cbb20d4d-d468-4972-ac78-1678e69b548b-istiod-ca-cert\") pod \"e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm\" (UID: \"cbb20d4d-d468-4972-ac78-1678e69b548b\") " pod="openshift-ingress/e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm"
Sep 09 21:01:26.485121 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:26.485036    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cbb20d4d-d468-4972-ac78-1678e69b548b-istio-token\") pod \"e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm\" (UID: \"cbb20d4d-d468-4972-ac78-1678e69b548b\") " pod="openshift-ingress/e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm"
Sep 09 21:01:26.485121 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:26.485081    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbb20d4d-d468-4972-ac78-1678e69b548b-credential-socket\") pod \"e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm\" (UID: \"cbb20d4d-d468-4972-ac78-1678e69b548b\") " pod="openshift-ingress/e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm"
Sep 09 21:01:26.485245 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:26.485203    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cbb20d4d-d468-4972-ac78-1678e69b548b-workload-certs\") pod \"e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm\" (UID: \"cbb20d4d-d468-4972-ac78-1678e69b548b\") " pod="openshift-ingress/e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm"
Sep 09 21:01:26.485302 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:26.485253    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cbb20d4d-d468-4972-ac78-1678e69b548b-istio-data\") pod \"e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm\" (UID: \"cbb20d4d-d468-4972-ac78-1678e69b548b\") " pod="openshift-ingress/e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm"
Sep 09 21:01:26.485359 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:26.485305    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbb20d4d-d468-4972-ac78-1678e69b548b-workload-socket\") pod \"e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm\" (UID: \"cbb20d4d-d468-4972-ac78-1678e69b548b\") " pod="openshift-ingress/e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm"
Sep 09 21:01:26.485584 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:26.485559    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbb20d4d-d468-4972-ac78-1678e69b548b-credential-socket\") pod \"e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm\" (UID: \"cbb20d4d-d468-4972-ac78-1678e69b548b\") " pod="openshift-ingress/e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm"
Sep 09 21:01:26.485700 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:26.485681    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cbb20d4d-d468-4972-ac78-1678e69b548b-istiod-ca-cert\") pod \"e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm\" (UID: \"cbb20d4d-d468-4972-ac78-1678e69b548b\") " pod="openshift-ingress/e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm"
Sep 09 21:01:26.487073 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:26.487050    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cbb20d4d-d468-4972-ac78-1678e69b548b-istio-envoy\") pod \"e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm\" (UID: \"cbb20d4d-d468-4972-ac78-1678e69b548b\") " pod="openshift-ingress/e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm"
Sep 09 21:01:26.487286 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:26.487270    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cbb20d4d-d468-4972-ac78-1678e69b548b-istio-podinfo\") pod \"e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm\" (UID: \"cbb20d4d-d468-4972-ac78-1678e69b548b\") " pod="openshift-ingress/e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm"
Sep 09 21:01:26.498187 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:26.498161    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vj2w5\" (UniqueName: \"kubernetes.io/projected/cbb20d4d-d468-4972-ac78-1678e69b548b-kube-api-access-vj2w5\") pod \"e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm\" (UID: \"cbb20d4d-d468-4972-ac78-1678e69b548b\") " pod="openshift-ingress/e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm"
Sep 09 21:01:26.501345 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:26.501326    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cbb20d4d-d468-4972-ac78-1678e69b548b-istio-token\") pod \"e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm\" (UID: \"cbb20d4d-d468-4972-ac78-1678e69b548b\") " pod="openshift-ingress/e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm"
Sep 09 21:01:26.598093 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:26.598014    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm"
Sep 09 21:01:26.885568 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:26.885543    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm"]
Sep 09 21:01:26.887066 ip-10-0-129-99 kubenswrapper[2581]: W0909 21:01:26.887044    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbb20d4d_d468_4972_ac78_1678e69b548b.slice/crio-a1fd198886ffa5f7ff4df117b30542f1b5240cc9ecc8867e3c384073a51a64dc WatchSource:0}: Error finding container a1fd198886ffa5f7ff4df117b30542f1b5240cc9ecc8867e3c384073a51a64dc: Status 404 returned error can't find the container with id a1fd198886ffa5f7ff4df117b30542f1b5240cc9ecc8867e3c384073a51a64dc
Sep 09 21:01:26.889375 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:26.889336    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:01:26.889476 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:26.889439    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:01:26.889545 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:26.889486    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:01:27.695858 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:27.695809    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm" event={"ID":"cbb20d4d-d468-4972-ac78-1678e69b548b","Type":"ContainerStarted","Data":"79fb0b9def7e823767e221081fbd8459b5d4ff1c75a1a06f0007ccaaf91d766a"}
Sep 09 21:01:27.696387 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:27.695906    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm" event={"ID":"cbb20d4d-d468-4972-ac78-1678e69b548b","Type":"ContainerStarted","Data":"a1fd198886ffa5f7ff4df117b30542f1b5240cc9ecc8867e3c384073a51a64dc"}
Sep 09 21:01:27.724422 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:27.724369    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm" podStartSLOduration=1.724354506 podStartE2EDuration="1.724354506s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:01:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:01:27.724042356 +0000 UTC m=+1249.351670228" watchObservedRunningTime="2026-09-09 21:01:27.724354506 +0000 UTC m=+1249.351982380"
Sep 09 21:01:28.598299 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:28.598259    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm"
Sep 09 21:01:28.603646 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:28.603621    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm"
Sep 09 21:01:28.701489 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:28.701466    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm"
Sep 09 21:01:28.702507 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:28.702489    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm"
Sep 09 21:01:38.960132 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:38.960023    2581 scope.go:117] "RemoveContainer" containerID="393c6d339f5d5fa3bc5594b995f9eee753ae7519dfd71db61ba56015e233d104"
Sep 09 21:01:38.968957 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:38.968939    2581 scope.go:117] "RemoveContainer" containerID="8d7a1f3a11e913073da54e7a9e9af623746a44689be2946383df04fa3550373c"
Sep 09 21:01:56.877608 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:56.877512    2581 generic.go:356] "Generic (PLEG): container finished" podID="cbb20d4d-d468-4972-ac78-1678e69b548b" containerID="79fb0b9def7e823767e221081fbd8459b5d4ff1c75a1a06f0007ccaaf91d766a" exitCode=0
Sep 09 21:01:56.877608 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:56.877583    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm" event={"ID":"cbb20d4d-d468-4972-ac78-1678e69b548b","Type":"ContainerDied","Data":"79fb0b9def7e823767e221081fbd8459b5d4ff1c75a1a06f0007ccaaf91d766a"}
Sep 09 21:01:56.878035 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:56.878008    2581 scope.go:117] "RemoveContainer" containerID="79fb0b9def7e823767e221081fbd8459b5d4ff1c75a1a06f0007ccaaf91d766a"
Sep 09 21:01:56.879402 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:56.879377    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:01:56.879469 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:56.879441    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:01:56.879506 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:56.879469    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:01:57.885454 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:57.885419    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm" event={"ID":"cbb20d4d-d468-4972-ac78-1678e69b548b","Type":"ContainerStarted","Data":"8207eabdd1ce6afe9ecae8cc6e59a88004fcfa397853ff490438a0f74008e0b7"}
Sep 09 21:01:58.598685 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:58.598651    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm"
Sep 09 21:01:58.599833 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:58.599806    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Sep 09 21:01:58.599962 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:58.599857    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm" podUID="cbb20d4d-d468-4972-ac78-1678e69b548b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Sep 09 21:01:58.704538 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:58.704497    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm"
Sep 09 21:01:59.598964 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:59.598930    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Sep 09 21:01:59.599365 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:01:59.598985    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm" podUID="cbb20d4d-d468-4972-ac78-1678e69b548b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Sep 09 21:02:00.598746 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:02:00.598708    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Sep 09 21:02:00.598909 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:02:00.598777    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm" podUID="cbb20d4d-d468-4972-ac78-1678e69b548b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Sep 09 21:02:01.598715 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:02:01.598680    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Sep 09 21:02:01.599140 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:02:01.598748    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm" podUID="cbb20d4d-d468-4972-ac78-1678e69b548b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Sep 09 21:02:02.598960 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:02:02.598923    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Sep 09 21:02:02.599398 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:02:02.598983    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm" podUID="cbb20d4d-d468-4972-ac78-1678e69b548b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Sep 09 21:02:03.598625 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:02:03.598587    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Sep 09 21:02:03.598808 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:02:03.598643    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm" podUID="cbb20d4d-d468-4972-ac78-1678e69b548b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Sep 09 21:02:04.602894 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:02:04.602865    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm"
Sep 09 21:02:04.603793 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:02:04.603773    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm"
Sep 09 21:02:39.007535 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:02:39.007504    2581 scope.go:117] "RemoveContainer" containerID="40b8ebe399738336eeab361350b9c0309da4812e47dc34d76ce4f331db1c0eeb"
Sep 09 21:02:39.016609 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:02:39.016587    2581 scope.go:117] "RemoveContainer" containerID="3e0a5fdf6784f512a1a43db048b72e287684327b50ccd46867a4069055e1ddc0"
Sep 09 21:02:39.025310 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:02:39.025247    2581 scope.go:117] "RemoveContainer" containerID="2908f3e5f8d3ca8e6a6ccab8c2ec1fe223da36abc4d27796c932b7bc8e3da531"
Sep 09 21:04:03.596954 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:04:03.596917    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-12f2a92f-xhjnp"]
Sep 09 21:04:03.600713 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:04:03.600696    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-12f2a92f-xhjnp"
Sep 09 21:04:03.603414 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:04:03.603390    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-glz44\""
Sep 09 21:04:03.608605 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:04:03.608556    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-12f2a92f-xhjnp"]
Sep 09 21:04:03.741258 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:04:03.741218    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b807efc-699a-4736-b0ab-8d2ac0f8bd81-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-12f2a92f-xhjnp\" (UID: \"3b807efc-699a-4736-b0ab-8d2ac0f8bd81\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-12f2a92f-xhjnp"
Sep 09 21:04:03.842718 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:04:03.842680    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b807efc-699a-4736-b0ab-8d2ac0f8bd81-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-12f2a92f-xhjnp\" (UID: \"3b807efc-699a-4736-b0ab-8d2ac0f8bd81\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-12f2a92f-xhjnp"
Sep 09 21:04:03.843358 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:04:03.843335    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b807efc-699a-4736-b0ab-8d2ac0f8bd81-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-12f2a92f-xhjnp\" (UID: \"3b807efc-699a-4736-b0ab-8d2ac0f8bd81\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-12f2a92f-xhjnp"
Sep 09 21:04:03.913593 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:04:03.913531    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-12f2a92f-xhjnp"
Sep 09 21:04:04.189585 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:04:04.189560    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-12f2a92f-xhjnp"]
Sep 09 21:04:04.191870 ip-10-0-129-99 kubenswrapper[2581]: W0909 21:04:04.191839    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b807efc_699a_4736_b0ab_8d2ac0f8bd81.slice/crio-d299de685fb50c9b0b2491b41c1eec5fd023d7cc921ec83791495d392ad95e49 WatchSource:0}: Error finding container d299de685fb50c9b0b2491b41c1eec5fd023d7cc921ec83791495d392ad95e49: Status 404 returned error can't find the container with id d299de685fb50c9b0b2491b41c1eec5fd023d7cc921ec83791495d392ad95e49
Sep 09 21:04:04.659912 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:04:04.659876    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-12f2a92f-xhjnp" event={"ID":"3b807efc-699a-4736-b0ab-8d2ac0f8bd81","Type":"ContainerStarted","Data":"394138d2723d9325f201d18fce2abfa53b78289b2ad76d61a908a9b9c3d8ce4f"}
Sep 09 21:04:04.660318 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:04:04.659930    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-12f2a92f-xhjnp" event={"ID":"3b807efc-699a-4736-b0ab-8d2ac0f8bd81","Type":"ContainerStarted","Data":"d299de685fb50c9b0b2491b41c1eec5fd023d7cc921ec83791495d392ad95e49"}
Sep 09 21:04:04.676049 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:04:04.676000    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-12f2a92f-xhjnp" podStartSLOduration=1.675984879 podStartE2EDuration="1.675984879s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:04:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:04:04.675356238 +0000 UTC m=+1406.302984112" watchObservedRunningTime="2026-09-09 21:04:04.675984879 +0000 UTC m=+1406.303612751"
Sep 09 21:04:05.667447 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:04:05.667411    2581 generic.go:356] "Generic (PLEG): container finished" podID="3b807efc-699a-4736-b0ab-8d2ac0f8bd81" containerID="394138d2723d9325f201d18fce2abfa53b78289b2ad76d61a908a9b9c3d8ce4f" exitCode=0
Sep 09 21:04:05.667887 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:04:05.667494    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-12f2a92f-xhjnp" event={"ID":"3b807efc-699a-4736-b0ab-8d2ac0f8bd81","Type":"ContainerDied","Data":"394138d2723d9325f201d18fce2abfa53b78289b2ad76d61a908a9b9c3d8ce4f"}
Sep 09 21:04:06.812042 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:04:06.812016    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-12f2a92f-xhjnp"
Sep 09 21:04:06.875390 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:04:06.875358    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b807efc-699a-4736-b0ab-8d2ac0f8bd81-maas-api-service-ca\") pod \"3b807efc-699a-4736-b0ab-8d2ac0f8bd81\" (UID: \"3b807efc-699a-4736-b0ab-8d2ac0f8bd81\") "
Sep 09 21:04:06.875738 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:04:06.875712    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b807efc-699a-4736-b0ab-8d2ac0f8bd81-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3b807efc-699a-4736-b0ab-8d2ac0f8bd81" (UID: "3b807efc-699a-4736-b0ab-8d2ac0f8bd81"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:04:06.976913 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:04:06.976830    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b807efc-699a-4736-b0ab-8d2ac0f8bd81-maas-api-service-ca\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:04:07.679822 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:04:07.679793    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-12f2a92f-xhjnp"
Sep 09 21:04:07.680006 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:04:07.679800    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-12f2a92f-xhjnp" event={"ID":"3b807efc-699a-4736-b0ab-8d2ac0f8bd81","Type":"ContainerDied","Data":"d299de685fb50c9b0b2491b41c1eec5fd023d7cc921ec83791495d392ad95e49"}
Sep 09 21:04:07.680006 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:04:07.679914    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d299de685fb50c9b0b2491b41c1eec5fd023d7cc921ec83791495d392ad95e49"
Sep 09 21:04:19.660677 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:04:19.660646    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-12f2a92f-xhjnp"]
Sep 09 21:04:19.663927 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:04:19.663907    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-12f2a92f-xhjnp"]
Sep 09 21:04:20.821361 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:04:20.821323    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b807efc-699a-4736-b0ab-8d2ac0f8bd81" path="/var/lib/kubelet/pods/3b807efc-699a-4736-b0ab-8d2ac0f8bd81/volumes"
Sep 09 21:04:39.064837 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:04:39.064764    2581 scope.go:117] "RemoveContainer" containerID="e00aff975e91660a54658df827bfac09babb55cd4052aa302607638554bae993"
Sep 09 21:05:38.916162 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:05:38.916133    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-wsv4v_2b1f9a6e-7a79-4e89-ac3f-4658d6d91550/console-operator/1.log"
Sep 09 21:05:38.918280 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:05:38.918258    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-wsv4v_2b1f9a6e-7a79-4e89-ac3f-4658d6d91550/console-operator/1.log"
Sep 09 21:05:38.922608 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:05:38.922591    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-75wj4_55b4fbae-29dc-41bf-a907-b1412b852cef/ovn-acl-logging/0.log"
Sep 09 21:05:38.925079 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:05:38.925062    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-75wj4_55b4fbae-29dc-41bf-a907-b1412b852cef/ovn-acl-logging/0.log"
Sep 09 21:05:39.086632 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:05:39.086599    2581 scope.go:117] "RemoveContainer" containerID="bb134a497ea012b3402dab57c753a990f8a215d8ff209b5442d7ae9d4652f437"
Sep 09 21:06:20.924580 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:20.924536    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc"]
Sep 09 21:06:20.925314 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:20.925291    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b807efc-699a-4736-b0ab-8d2ac0f8bd81" containerName="revoke-keys"
Sep 09 21:06:20.925486 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:20.925315    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b807efc-699a-4736-b0ab-8d2ac0f8bd81" containerName="revoke-keys"
Sep 09 21:06:20.925486 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:20.925466    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b807efc-699a-4736-b0ab-8d2ac0f8bd81" containerName="revoke-keys"
Sep 09 21:06:20.928711 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:20.928679    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc"
Sep 09 21:06:20.933431 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:20.933408    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-1e8973-openshift-default-dockercfg-tnwk5\""
Sep 09 21:06:20.948157 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:20.948130    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc"]
Sep 09 21:06:20.995674 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:20.995650    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6070b4c-f954-4cea-9dc8-32de78b13f0b-istio-envoy\") pod \"e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc\" (UID: \"d6070b4c-f954-4cea-9dc8-32de78b13f0b\") " pod="openshift-ingress/e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc"
Sep 09 21:06:20.995816 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:20.995693    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6070b4c-f954-4cea-9dc8-32de78b13f0b-istio-token\") pod \"e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc\" (UID: \"d6070b4c-f954-4cea-9dc8-32de78b13f0b\") " pod="openshift-ingress/e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc"
Sep 09 21:06:20.995816 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:20.995714    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnct9\" (UniqueName: \"kubernetes.io/projected/d6070b4c-f954-4cea-9dc8-32de78b13f0b-kube-api-access-wnct9\") pod \"e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc\" (UID: \"d6070b4c-f954-4cea-9dc8-32de78b13f0b\") " pod="openshift-ingress/e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc"
Sep 09 21:06:20.995816 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:20.995767    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6070b4c-f954-4cea-9dc8-32de78b13f0b-workload-certs\") pod \"e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc\" (UID: \"d6070b4c-f954-4cea-9dc8-32de78b13f0b\") " pod="openshift-ingress/e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc"
Sep 09 21:06:20.995816 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:20.995811    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6070b4c-f954-4cea-9dc8-32de78b13f0b-istio-podinfo\") pod \"e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc\" (UID: \"d6070b4c-f954-4cea-9dc8-32de78b13f0b\") " pod="openshift-ingress/e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc"
Sep 09 21:06:20.995979 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:20.995894    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6070b4c-f954-4cea-9dc8-32de78b13f0b-workload-socket\") pod \"e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc\" (UID: \"d6070b4c-f954-4cea-9dc8-32de78b13f0b\") " pod="openshift-ingress/e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc"
Sep 09 21:06:20.995979 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:20.995917    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6070b4c-f954-4cea-9dc8-32de78b13f0b-credential-socket\") pod \"e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc\" (UID: \"d6070b4c-f954-4cea-9dc8-32de78b13f0b\") " pod="openshift-ingress/e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc"
Sep 09 21:06:20.995979 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:20.995964    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6070b4c-f954-4cea-9dc8-32de78b13f0b-istiod-ca-cert\") pod \"e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc\" (UID: \"d6070b4c-f954-4cea-9dc8-32de78b13f0b\") " pod="openshift-ingress/e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc"
Sep 09 21:06:20.996082 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:20.996005    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6070b4c-f954-4cea-9dc8-32de78b13f0b-istio-data\") pod \"e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc\" (UID: \"d6070b4c-f954-4cea-9dc8-32de78b13f0b\") " pod="openshift-ingress/e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc"
Sep 09 21:06:21.097156 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:21.097120    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6070b4c-f954-4cea-9dc8-32de78b13f0b-workload-socket\") pod \"e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc\" (UID: \"d6070b4c-f954-4cea-9dc8-32de78b13f0b\") " pod="openshift-ingress/e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc"
Sep 09 21:06:21.097344 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:21.097167    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6070b4c-f954-4cea-9dc8-32de78b13f0b-credential-socket\") pod \"e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc\" (UID: \"d6070b4c-f954-4cea-9dc8-32de78b13f0b\") " pod="openshift-ingress/e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc"
Sep 09 21:06:21.097344 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:21.097226    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6070b4c-f954-4cea-9dc8-32de78b13f0b-istiod-ca-cert\") pod \"e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc\" (UID: \"d6070b4c-f954-4cea-9dc8-32de78b13f0b\") " pod="openshift-ingress/e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc"
Sep 09 21:06:21.097344 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:21.097260    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6070b4c-f954-4cea-9dc8-32de78b13f0b-istio-data\") pod \"e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc\" (UID: \"d6070b4c-f954-4cea-9dc8-32de78b13f0b\") " pod="openshift-ingress/e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc"
Sep 09 21:06:21.097344 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:21.097300    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6070b4c-f954-4cea-9dc8-32de78b13f0b-istio-envoy\") pod \"e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc\" (UID: \"d6070b4c-f954-4cea-9dc8-32de78b13f0b\") " pod="openshift-ingress/e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc"
Sep 09 21:06:21.097574 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:21.097433    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6070b4c-f954-4cea-9dc8-32de78b13f0b-istio-token\") pod \"e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc\" (UID: \"d6070b4c-f954-4cea-9dc8-32de78b13f0b\") " pod="openshift-ingress/e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc"
Sep 09 21:06:21.097574 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:21.097486    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnct9\" (UniqueName: \"kubernetes.io/projected/d6070b4c-f954-4cea-9dc8-32de78b13f0b-kube-api-access-wnct9\") pod \"e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc\" (UID: \"d6070b4c-f954-4cea-9dc8-32de78b13f0b\") " pod="openshift-ingress/e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc"
Sep 09 21:06:21.097574 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:21.097505    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6070b4c-f954-4cea-9dc8-32de78b13f0b-workload-socket\") pod \"e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc\" (UID: \"d6070b4c-f954-4cea-9dc8-32de78b13f0b\") " pod="openshift-ingress/e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc"
Sep 09 21:06:21.097574 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:21.097536    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6070b4c-f954-4cea-9dc8-32de78b13f0b-workload-certs\") pod \"e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc\" (UID: \"d6070b4c-f954-4cea-9dc8-32de78b13f0b\") " pod="openshift-ingress/e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc"
Sep 09 21:06:21.097574 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:21.097539    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6070b4c-f954-4cea-9dc8-32de78b13f0b-credential-socket\") pod \"e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc\" (UID: \"d6070b4c-f954-4cea-9dc8-32de78b13f0b\") " pod="openshift-ingress/e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc"
Sep 09 21:06:21.097842 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:21.097672    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6070b4c-f954-4cea-9dc8-32de78b13f0b-istio-podinfo\") pod \"e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc\" (UID: \"d6070b4c-f954-4cea-9dc8-32de78b13f0b\") " pod="openshift-ingress/e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc"
Sep 09 21:06:21.097842 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:21.097721    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6070b4c-f954-4cea-9dc8-32de78b13f0b-istio-data\") pod \"e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc\" (UID: \"d6070b4c-f954-4cea-9dc8-32de78b13f0b\") " pod="openshift-ingress/e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc"
Sep 09 21:06:21.097842 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:21.097809    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6070b4c-f954-4cea-9dc8-32de78b13f0b-workload-certs\") pod \"e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc\" (UID: \"d6070b4c-f954-4cea-9dc8-32de78b13f0b\") " pod="openshift-ingress/e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc"
Sep 09 21:06:21.098040 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:21.098017    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6070b4c-f954-4cea-9dc8-32de78b13f0b-istiod-ca-cert\") pod \"e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc\" (UID: \"d6070b4c-f954-4cea-9dc8-32de78b13f0b\") " pod="openshift-ingress/e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc"
Sep 09 21:06:21.099837 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:21.099809    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6070b4c-f954-4cea-9dc8-32de78b13f0b-istio-envoy\") pod \"e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc\" (UID: \"d6070b4c-f954-4cea-9dc8-32de78b13f0b\") " pod="openshift-ingress/e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc"
Sep 09 21:06:21.100378 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:21.100358    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6070b4c-f954-4cea-9dc8-32de78b13f0b-istio-podinfo\") pod \"e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc\" (UID: \"d6070b4c-f954-4cea-9dc8-32de78b13f0b\") " pod="openshift-ingress/e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc"
Sep 09 21:06:21.106288 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:21.106259    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6070b4c-f954-4cea-9dc8-32de78b13f0b-istio-token\") pod \"e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc\" (UID: \"d6070b4c-f954-4cea-9dc8-32de78b13f0b\") " pod="openshift-ingress/e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc"
Sep 09 21:06:21.106936 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:21.106910    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnct9\" (UniqueName: \"kubernetes.io/projected/d6070b4c-f954-4cea-9dc8-32de78b13f0b-kube-api-access-wnct9\") pod \"e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc\" (UID: \"d6070b4c-f954-4cea-9dc8-32de78b13f0b\") " pod="openshift-ingress/e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc"
Sep 09 21:06:21.239554 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:21.239524    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc"
Sep 09 21:06:21.522427 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:21.522396    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc"]
Sep 09 21:06:21.524064 ip-10-0-129-99 kubenswrapper[2581]: W0909 21:06:21.524035    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6070b4c_f954_4cea_9dc8_32de78b13f0b.slice/crio-a72e9cfb39e7ea34b1a767fc7ad91a6ad7c9d4dec024a9293dd15d743ef42bd7 WatchSource:0}: Error finding container a72e9cfb39e7ea34b1a767fc7ad91a6ad7c9d4dec024a9293dd15d743ef42bd7: Status 404 returned error can't find the container with id a72e9cfb39e7ea34b1a767fc7ad91a6ad7c9d4dec024a9293dd15d743ef42bd7
Sep 09 21:06:21.526414 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:21.526370    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:06:21.526531 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:21.526446    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:06:21.526531 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:21.526483    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:06:22.486341 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:22.486300    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc" event={"ID":"d6070b4c-f954-4cea-9dc8-32de78b13f0b","Type":"ContainerStarted","Data":"acf7ac9a58b1be54ab70933cc306b870c62aac305c38e2733f4a3a0e7f28eccf"}
Sep 09 21:06:22.486795 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:22.486354    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc" event={"ID":"d6070b4c-f954-4cea-9dc8-32de78b13f0b","Type":"ContainerStarted","Data":"a72e9cfb39e7ea34b1a767fc7ad91a6ad7c9d4dec024a9293dd15d743ef42bd7"}
Sep 09 21:06:22.515814 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:22.515765    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc" podStartSLOduration=2.515749885 podStartE2EDuration="2.515749885s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:06:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:06:22.514266695 +0000 UTC m=+1544.141894568" watchObservedRunningTime="2026-09-09 21:06:22.515749885 +0000 UTC m=+1544.143377758"
Sep 09 21:06:23.239889 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:23.239853    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc"
Sep 09 21:06:23.245596 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:23.245570    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc"
Sep 09 21:06:23.492473 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:23.492375    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc"
Sep 09 21:06:23.493332 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:23.493313    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc"
Sep 09 21:06:26.377322 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:26.377286    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-1e8973-6d5d9c9f66-w58rq"]
Sep 09 21:06:26.381096 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:26.381079    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1e8973-6d5d9c9f66-w58rq"
Sep 09 21:06:26.384354 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:26.384330    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-1e8973\""
Sep 09 21:06:26.393669 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:26.393631    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-1e8973-6d5d9c9f66-w58rq"]
Sep 09 21:06:26.409423 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:26.409401    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-1e8973-f988f6d57-wgkwt"]
Sep 09 21:06:26.413404 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:26.413383    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-1e8973-f988f6d57-wgkwt"
Sep 09 21:06:26.416809 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:26.416786    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-1e8973-dockercfg-ns7b8\""
Sep 09 21:06:26.429696 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:26.429672    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-1e8973-f988f6d57-wgkwt"]
Sep 09 21:06:26.455817 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:26.455785    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32334605-b6c1-47c4-b36a-c8eda7e5bd62-plugins-config-volume\") pod \"payload-processing-e2e-api-a-1e8973-f988f6d57-wgkwt\" (UID: \"32334605-b6c1-47c4-b36a-c8eda7e5bd62\") " pod="openshift-ingress/payload-processing-e2e-api-a-1e8973-f988f6d57-wgkwt"
Sep 09 21:06:26.456011 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:26.455896    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65z7d\" (UniqueName: \"kubernetes.io/projected/32334605-b6c1-47c4-b36a-c8eda7e5bd62-kube-api-access-65z7d\") pod \"payload-processing-e2e-api-a-1e8973-f988f6d57-wgkwt\" (UID: \"32334605-b6c1-47c4-b36a-c8eda7e5bd62\") " pod="openshift-ingress/payload-processing-e2e-api-a-1e8973-f988f6d57-wgkwt"
Sep 09 21:06:26.456011 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:26.455918    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/81af25c4-2a3d-4bc1-af8f-facfffb6a554-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-1e8973-6d5d9c9f66-w58rq\" (UID: \"81af25c4-2a3d-4bc1-af8f-facfffb6a554\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-1e8973-6d5d9c9f66-w58rq"
Sep 09 21:06:26.456011 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:26.455964    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vd4bx\" (UniqueName: \"kubernetes.io/projected/81af25c4-2a3d-4bc1-af8f-facfffb6a554-kube-api-access-vd4bx\") pod \"payload-pre-processing-e2e-api-a-1e8973-6d5d9c9f66-w58rq\" (UID: \"81af25c4-2a3d-4bc1-af8f-facfffb6a554\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-1e8973-6d5d9c9f66-w58rq"
Sep 09 21:06:26.557840 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:26.557703    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65z7d\" (UniqueName: \"kubernetes.io/projected/32334605-b6c1-47c4-b36a-c8eda7e5bd62-kube-api-access-65z7d\") pod \"payload-processing-e2e-api-a-1e8973-f988f6d57-wgkwt\" (UID: \"32334605-b6c1-47c4-b36a-c8eda7e5bd62\") " pod="openshift-ingress/payload-processing-e2e-api-a-1e8973-f988f6d57-wgkwt"
Sep 09 21:06:26.557840 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:26.557755    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/81af25c4-2a3d-4bc1-af8f-facfffb6a554-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-1e8973-6d5d9c9f66-w58rq\" (UID: \"81af25c4-2a3d-4bc1-af8f-facfffb6a554\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-1e8973-6d5d9c9f66-w58rq"
Sep 09 21:06:26.557840 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:26.557840    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vd4bx\" (UniqueName: \"kubernetes.io/projected/81af25c4-2a3d-4bc1-af8f-facfffb6a554-kube-api-access-vd4bx\") pod \"payload-pre-processing-e2e-api-a-1e8973-6d5d9c9f66-w58rq\" (UID: \"81af25c4-2a3d-4bc1-af8f-facfffb6a554\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-1e8973-6d5d9c9f66-w58rq"
Sep 09 21:06:26.558201 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:26.557949    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32334605-b6c1-47c4-b36a-c8eda7e5bd62-plugins-config-volume\") pod \"payload-processing-e2e-api-a-1e8973-f988f6d57-wgkwt\" (UID: \"32334605-b6c1-47c4-b36a-c8eda7e5bd62\") " pod="openshift-ingress/payload-processing-e2e-api-a-1e8973-f988f6d57-wgkwt"
Sep 09 21:06:26.559020 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:26.558996    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32334605-b6c1-47c4-b36a-c8eda7e5bd62-plugins-config-volume\") pod \"payload-processing-e2e-api-a-1e8973-f988f6d57-wgkwt\" (UID: \"32334605-b6c1-47c4-b36a-c8eda7e5bd62\") " pod="openshift-ingress/payload-processing-e2e-api-a-1e8973-f988f6d57-wgkwt"
Sep 09 21:06:26.559215 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:26.559173    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/81af25c4-2a3d-4bc1-af8f-facfffb6a554-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-1e8973-6d5d9c9f66-w58rq\" (UID: \"81af25c4-2a3d-4bc1-af8f-facfffb6a554\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-1e8973-6d5d9c9f66-w58rq"
Sep 09 21:06:26.570405 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:26.570380    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vd4bx\" (UniqueName: \"kubernetes.io/projected/81af25c4-2a3d-4bc1-af8f-facfffb6a554-kube-api-access-vd4bx\") pod \"payload-pre-processing-e2e-api-a-1e8973-6d5d9c9f66-w58rq\" (UID: \"81af25c4-2a3d-4bc1-af8f-facfffb6a554\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-1e8973-6d5d9c9f66-w58rq"
Sep 09 21:06:26.570620 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:26.570598    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-65z7d\" (UniqueName: \"kubernetes.io/projected/32334605-b6c1-47c4-b36a-c8eda7e5bd62-kube-api-access-65z7d\") pod \"payload-processing-e2e-api-a-1e8973-f988f6d57-wgkwt\" (UID: \"32334605-b6c1-47c4-b36a-c8eda7e5bd62\") " pod="openshift-ingress/payload-processing-e2e-api-a-1e8973-f988f6d57-wgkwt"
Sep 09 21:06:26.692832 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:26.692755    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1e8973-6d5d9c9f66-w58rq"
Sep 09 21:06:26.724377 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:26.724343    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-1e8973-f988f6d57-wgkwt"
Sep 09 21:06:26.979169 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:26.979069    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-1e8973-6d5d9c9f66-w58rq"]
Sep 09 21:06:26.985518 ip-10-0-129-99 kubenswrapper[2581]: W0909 21:06:26.985490    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81af25c4_2a3d_4bc1_af8f_facfffb6a554.slice/crio-567202b723904f261bc2b1c9d84ca84a060caa475a7399a7d784d88b975868d9 WatchSource:0}: Error finding container 567202b723904f261bc2b1c9d84ca84a060caa475a7399a7d784d88b975868d9: Status 404 returned error can't find the container with id 567202b723904f261bc2b1c9d84ca84a060caa475a7399a7d784d88b975868d9
Sep 09 21:06:27.015824 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:27.015800    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-1e8973-f988f6d57-wgkwt"]
Sep 09 21:06:27.022366 ip-10-0-129-99 kubenswrapper[2581]: W0909 21:06:27.022340    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32334605_b6c1_47c4_b36a_c8eda7e5bd62.slice/crio-4f884dfc082c310025a0ecdcb876c64312a0d9b5996ad99c5d0f698c008d5a37 WatchSource:0}: Error finding container 4f884dfc082c310025a0ecdcb876c64312a0d9b5996ad99c5d0f698c008d5a37: Status 404 returned error can't find the container with id 4f884dfc082c310025a0ecdcb876c64312a0d9b5996ad99c5d0f698c008d5a37
Sep 09 21:06:27.537331 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:27.537295    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1e8973-6d5d9c9f66-w58rq" event={"ID":"81af25c4-2a3d-4bc1-af8f-facfffb6a554","Type":"ContainerStarted","Data":"e063e27086e6510ccdb806fe47470e29b0cdfd6f6810d6b3a46fc0f461b614ef"}
Sep 09 21:06:27.537805 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:27.537356    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1e8973-6d5d9c9f66-w58rq" event={"ID":"81af25c4-2a3d-4bc1-af8f-facfffb6a554","Type":"ContainerStarted","Data":"567202b723904f261bc2b1c9d84ca84a060caa475a7399a7d784d88b975868d9"}
Sep 09 21:06:27.537805 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:27.537476    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1e8973-6d5d9c9f66-w58rq"
Sep 09 21:06:27.538942 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:27.538911    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-1e8973-f988f6d57-wgkwt" event={"ID":"32334605-b6c1-47c4-b36a-c8eda7e5bd62","Type":"ContainerStarted","Data":"b62605ee125e7f758b1bbdf25af1380f16d2fc5c6fe39a6e50c4793b831897a7"}
Sep 09 21:06:27.539067 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:27.538953    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-1e8973-f988f6d57-wgkwt" event={"ID":"32334605-b6c1-47c4-b36a-c8eda7e5bd62","Type":"ContainerStarted","Data":"4f884dfc082c310025a0ecdcb876c64312a0d9b5996ad99c5d0f698c008d5a37"}
Sep 09 21:06:27.539067 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:27.539020    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-1e8973-f988f6d57-wgkwt"
Sep 09 21:06:27.581176 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:27.581126    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1e8973-6d5d9c9f66-w58rq" podStartSLOduration=1.581085042 podStartE2EDuration="1.581085042s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:06:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:06:27.562363373 +0000 UTC m=+1549.189991248" watchObservedRunningTime="2026-09-09 21:06:27.581085042 +0000 UTC m=+1549.208712915"
Sep 09 21:06:38.545828 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:38.545797    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-1e8973-f988f6d57-wgkwt"
Sep 09 21:06:38.546346 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:38.545882    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1e8973-6d5d9c9f66-w58rq"
Sep 09 21:06:38.565904 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:38.565848    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-1e8973-f988f6d57-wgkwt" podStartSLOduration=12.565831433 podStartE2EDuration="12.565831433s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:06:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:06:27.580808378 +0000 UTC m=+1549.208436253" watchObservedRunningTime="2026-09-09 21:06:38.565831433 +0000 UTC m=+1560.193459325"
Sep 09 21:06:39.108137 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:39.108089    2581 scope.go:117] "RemoveContainer" containerID="ff8eb15373989afc543474758a7341b5edd7dab6c281b76b9d740e39e590c9bc"
Sep 09 21:06:39.116649 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:39.116629    2581 scope.go:117] "RemoveContainer" containerID="b5137f94e95cdbc14ef40cbb1c3bba0f72e837216f9b4b67a0cbd6b82613287a"
Sep 09 21:06:39.125324 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:39.125304    2581 scope.go:117] "RemoveContainer" containerID="c40ac317c2654cf70dc170cf164d33213b81a778d74b604e3bd230f24594cee8"
Sep 09 21:06:45.289190 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:45.289153    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-d36b91-cdrdg"]
Sep 09 21:06:45.292960 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:45.292937    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-d36b91-cdrdg"
Sep 09 21:06:45.296335 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:45.296310    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-glz44\""
Sep 09 21:06:45.302124 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:45.302067    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-d36b91-cdrdg"]
Sep 09 21:06:45.466283 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:45.466250    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a92bc038-afaf-4fe2-851c-a4caca3cca49-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-d36b91-cdrdg\" (UID: \"a92bc038-afaf-4fe2-851c-a4caca3cca49\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-d36b91-cdrdg"
Sep 09 21:06:45.568119 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:45.568015    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a92bc038-afaf-4fe2-851c-a4caca3cca49-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-d36b91-cdrdg\" (UID: \"a92bc038-afaf-4fe2-851c-a4caca3cca49\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-d36b91-cdrdg"
Sep 09 21:06:45.568720 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:45.568699    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a92bc038-afaf-4fe2-851c-a4caca3cca49-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-d36b91-cdrdg\" (UID: \"a92bc038-afaf-4fe2-851c-a4caca3cca49\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-d36b91-cdrdg"
Sep 09 21:06:45.605428 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:45.605397    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-d36b91-cdrdg"
Sep 09 21:06:45.883747 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:45.883718    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-d36b91-cdrdg"]
Sep 09 21:06:45.885165 ip-10-0-129-99 kubenswrapper[2581]: W0909 21:06:45.885138    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda92bc038_afaf_4fe2_851c_a4caca3cca49.slice/crio-b2ef2ceda867b6751d30b439f0cc557994bf2e006230a160cc158284df194831 WatchSource:0}: Error finding container b2ef2ceda867b6751d30b439f0cc557994bf2e006230a160cc158284df194831: Status 404 returned error can't find the container with id b2ef2ceda867b6751d30b439f0cc557994bf2e006230a160cc158284df194831
Sep 09 21:06:46.662804 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:46.662770    2581 generic.go:356] "Generic (PLEG): container finished" podID="a92bc038-afaf-4fe2-851c-a4caca3cca49" containerID="2c4fedf4f5641df78029439b24cc34f405418b3d2c62420b0fe3ed2f8598e743" exitCode=0
Sep 09 21:06:46.663235 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:46.662866    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-d36b91-cdrdg" event={"ID":"a92bc038-afaf-4fe2-851c-a4caca3cca49","Type":"ContainerDied","Data":"2c4fedf4f5641df78029439b24cc34f405418b3d2c62420b0fe3ed2f8598e743"}
Sep 09 21:06:46.663235 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:46.662934    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-d36b91-cdrdg" event={"ID":"a92bc038-afaf-4fe2-851c-a4caca3cca49","Type":"ContainerStarted","Data":"b2ef2ceda867b6751d30b439f0cc557994bf2e006230a160cc158284df194831"}
Sep 09 21:06:47.811482 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:47.811456    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-d36b91-cdrdg"
Sep 09 21:06:47.994864 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:47.994836    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a92bc038-afaf-4fe2-851c-a4caca3cca49-maas-api-service-ca\") pod \"a92bc038-afaf-4fe2-851c-a4caca3cca49\" (UID: \"a92bc038-afaf-4fe2-851c-a4caca3cca49\") "
Sep 09 21:06:47.995271 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:47.995243    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a92bc038-afaf-4fe2-851c-a4caca3cca49-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a92bc038-afaf-4fe2-851c-a4caca3cca49" (UID: "a92bc038-afaf-4fe2-851c-a4caca3cca49"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:06:48.096963 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:48.096928    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a92bc038-afaf-4fe2-851c-a4caca3cca49-maas-api-service-ca\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:06:48.676278 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:48.676254    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-d36b91-cdrdg"
Sep 09 21:06:48.676473 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:48.676252    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-d36b91-cdrdg" event={"ID":"a92bc038-afaf-4fe2-851c-a4caca3cca49","Type":"ContainerDied","Data":"b2ef2ceda867b6751d30b439f0cc557994bf2e006230a160cc158284df194831"}
Sep 09 21:06:48.676473 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:48.676380    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b2ef2ceda867b6751d30b439f0cc557994bf2e006230a160cc158284df194831"
Sep 09 21:06:55.799399 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:55.799358    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-0622e2-7f4cc4664b-2jft4"]
Sep 09 21:06:55.800064 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:55.800041    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a92bc038-afaf-4fe2-851c-a4caca3cca49" containerName="revoke-keys"
Sep 09 21:06:55.800064 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:55.800066    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a92bc038-afaf-4fe2-851c-a4caca3cca49" containerName="revoke-keys"
Sep 09 21:06:55.800224 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:55.800211    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a92bc038-afaf-4fe2-851c-a4caca3cca49" containerName="revoke-keys"
Sep 09 21:06:55.804632 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:55.804611    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0622e2-7f4cc4664b-2jft4"
Sep 09 21:06:55.807872 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:55.807854    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-0622e2-dockercfg-bg7jg\""
Sep 09 21:06:55.808159 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:55.808142    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-0622e2\""
Sep 09 21:06:55.819733 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:55.819711    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-0622e2-7f4cc4664b-2jft4"]
Sep 09 21:06:55.897516 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:55.897476    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hsz9\" (UniqueName: \"kubernetes.io/projected/a461f916-8fba-4ea4-a2b6-c7b1694970bd-kube-api-access-5hsz9\") pod \"payload-pre-processing-e2e-api-b-0622e2-7f4cc4664b-2jft4\" (UID: \"a461f916-8fba-4ea4-a2b6-c7b1694970bd\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-0622e2-7f4cc4664b-2jft4"
Sep 09 21:06:55.898211 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:55.898184    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a461f916-8fba-4ea4-a2b6-c7b1694970bd-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-0622e2-7f4cc4664b-2jft4\" (UID: \"a461f916-8fba-4ea4-a2b6-c7b1694970bd\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-0622e2-7f4cc4664b-2jft4"
Sep 09 21:06:55.999365 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:55.999333    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hsz9\" (UniqueName: \"kubernetes.io/projected/a461f916-8fba-4ea4-a2b6-c7b1694970bd-kube-api-access-5hsz9\") pod \"payload-pre-processing-e2e-api-b-0622e2-7f4cc4664b-2jft4\" (UID: \"a461f916-8fba-4ea4-a2b6-c7b1694970bd\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-0622e2-7f4cc4664b-2jft4"
Sep 09 21:06:55.999537 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:55.999379    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a461f916-8fba-4ea4-a2b6-c7b1694970bd-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-0622e2-7f4cc4664b-2jft4\" (UID: \"a461f916-8fba-4ea4-a2b6-c7b1694970bd\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-0622e2-7f4cc4664b-2jft4"
Sep 09 21:06:56.000064 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:56.000044    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a461f916-8fba-4ea4-a2b6-c7b1694970bd-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-0622e2-7f4cc4664b-2jft4\" (UID: \"a461f916-8fba-4ea4-a2b6-c7b1694970bd\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-0622e2-7f4cc4664b-2jft4"
Sep 09 21:06:56.007993 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:56.007967    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hsz9\" (UniqueName: \"kubernetes.io/projected/a461f916-8fba-4ea4-a2b6-c7b1694970bd-kube-api-access-5hsz9\") pod \"payload-pre-processing-e2e-api-b-0622e2-7f4cc4664b-2jft4\" (UID: \"a461f916-8fba-4ea4-a2b6-c7b1694970bd\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-0622e2-7f4cc4664b-2jft4"
Sep 09 21:06:56.115289 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:56.115198    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0622e2-7f4cc4664b-2jft4"
Sep 09 21:06:56.393173 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:56.393147    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-0622e2-7f4cc4664b-2jft4"]
Sep 09 21:06:56.394552 ip-10-0-129-99 kubenswrapper[2581]: W0909 21:06:56.394522    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda461f916_8fba_4ea4_a2b6_c7b1694970bd.slice/crio-dd3b484c11f39aea1411448f546163a85f008b77480d6b2b85f5f659ddb339a1 WatchSource:0}: Error finding container dd3b484c11f39aea1411448f546163a85f008b77480d6b2b85f5f659ddb339a1: Status 404 returned error can't find the container with id dd3b484c11f39aea1411448f546163a85f008b77480d6b2b85f5f659ddb339a1
Sep 09 21:06:56.725587 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:56.725412    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0622e2-7f4cc4664b-2jft4" event={"ID":"a461f916-8fba-4ea4-a2b6-c7b1694970bd","Type":"ContainerStarted","Data":"99d5ab8c241c10fe8e9c56a2d8e898cc5aa16a33a6265ee842def55683da76ec"}
Sep 09 21:06:56.725587 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:56.725474    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0622e2-7f4cc4664b-2jft4" event={"ID":"a461f916-8fba-4ea4-a2b6-c7b1694970bd","Type":"ContainerStarted","Data":"dd3b484c11f39aea1411448f546163a85f008b77480d6b2b85f5f659ddb339a1"}
Sep 09 21:06:56.744828 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:56.725644    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0622e2-7f4cc4664b-2jft4"
Sep 09 21:06:56.752556 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:06:56.752502    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0622e2-7f4cc4664b-2jft4" podStartSLOduration=1.752487773 podStartE2EDuration="1.752487773s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:06:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:06:56.750918129 +0000 UTC m=+1578.378546003" watchObservedRunningTime="2026-09-09 21:06:56.752487773 +0000 UTC m=+1578.380115645"
Sep 09 21:07:01.479126 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:01.479066    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-d36b91-cdrdg"]
Sep 09 21:07:01.484177 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:01.484152    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-d36b91-cdrdg"]
Sep 09 21:07:02.821172 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:02.821140    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a92bc038-afaf-4fe2-851c-a4caca3cca49" path="/var/lib/kubelet/pods/a92bc038-afaf-4fe2-851c-a4caca3cca49/volumes"
Sep 09 21:07:07.733614 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:07.733583    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0622e2-7f4cc4664b-2jft4"
Sep 09 21:07:08.541022 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:08.540985    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd"]
Sep 09 21:07:08.541263 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:08.541238    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd" podUID="3dff37dd-436f-470c-9b0e-71ce7cd1d9f2" containerName="istio-proxy" containerID="cri-o://c5452b45a75f014bf445627945b7a155cc7df5e54c8f348d672d239af406a0c8" gracePeriod=30
Sep 09 21:07:13.837735 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:13.837660    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd"
Sep 09 21:07:13.845622 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:13.845590    2581 generic.go:356] "Generic (PLEG): container finished" podID="3dff37dd-436f-470c-9b0e-71ce7cd1d9f2" containerID="c5452b45a75f014bf445627945b7a155cc7df5e54c8f348d672d239af406a0c8" exitCode=0
Sep 09 21:07:13.845770 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:13.845632    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd"
Sep 09 21:07:13.845770 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:13.845676    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd" event={"ID":"3dff37dd-436f-470c-9b0e-71ce7cd1d9f2","Type":"ContainerDied","Data":"c5452b45a75f014bf445627945b7a155cc7df5e54c8f348d672d239af406a0c8"}
Sep 09 21:07:13.845770 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:13.845754    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd" event={"ID":"3dff37dd-436f-470c-9b0e-71ce7cd1d9f2","Type":"ContainerDied","Data":"708a43949ce14176947444d6165891ecb544ec4d74903a9119b0e67d52ef4cb3"}
Sep 09 21:07:13.845880 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:13.845779    2581 scope.go:117] "RemoveContainer" containerID="c5452b45a75f014bf445627945b7a155cc7df5e54c8f348d672d239af406a0c8"
Sep 09 21:07:13.855619 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:13.855598    2581 scope.go:117] "RemoveContainer" containerID="cce5895e33621c95c8291dab949ab8ace88c6c68e91bbea855fe67bfb2ec24a4"
Sep 09 21:07:13.866538 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:13.866509    2581 scope.go:117] "RemoveContainer" containerID="c5452b45a75f014bf445627945b7a155cc7df5e54c8f348d672d239af406a0c8"
Sep 09 21:07:13.867030 ip-10-0-129-99 kubenswrapper[2581]: E0909 21:07:13.866886    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5452b45a75f014bf445627945b7a155cc7df5e54c8f348d672d239af406a0c8\": container with ID starting with c5452b45a75f014bf445627945b7a155cc7df5e54c8f348d672d239af406a0c8 not found: ID does not exist" containerID="c5452b45a75f014bf445627945b7a155cc7df5e54c8f348d672d239af406a0c8"
Sep 09 21:07:13.867030 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:13.866925    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5452b45a75f014bf445627945b7a155cc7df5e54c8f348d672d239af406a0c8"} err="failed to get container status \"c5452b45a75f014bf445627945b7a155cc7df5e54c8f348d672d239af406a0c8\": rpc error: code = NotFound desc = could not find container \"c5452b45a75f014bf445627945b7a155cc7df5e54c8f348d672d239af406a0c8\": container with ID starting with c5452b45a75f014bf445627945b7a155cc7df5e54c8f348d672d239af406a0c8 not found: ID does not exist"
Sep 09 21:07:13.867030 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:13.866949    2581 scope.go:117] "RemoveContainer" containerID="cce5895e33621c95c8291dab949ab8ace88c6c68e91bbea855fe67bfb2ec24a4"
Sep 09 21:07:13.867536 ip-10-0-129-99 kubenswrapper[2581]: E0909 21:07:13.867504    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cce5895e33621c95c8291dab949ab8ace88c6c68e91bbea855fe67bfb2ec24a4\": container with ID starting with cce5895e33621c95c8291dab949ab8ace88c6c68e91bbea855fe67bfb2ec24a4 not found: ID does not exist" containerID="cce5895e33621c95c8291dab949ab8ace88c6c68e91bbea855fe67bfb2ec24a4"
Sep 09 21:07:13.867642 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:13.867540    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cce5895e33621c95c8291dab949ab8ace88c6c68e91bbea855fe67bfb2ec24a4"} err="failed to get container status \"cce5895e33621c95c8291dab949ab8ace88c6c68e91bbea855fe67bfb2ec24a4\": rpc error: code = NotFound desc = could not find container \"cce5895e33621c95c8291dab949ab8ace88c6c68e91bbea855fe67bfb2ec24a4\": container with ID starting with cce5895e33621c95c8291dab949ab8ace88c6c68e91bbea855fe67bfb2ec24a4 not found: ID does not exist"
Sep 09 21:07:13.930047 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:13.930014    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3dff37dd-436f-470c-9b0e-71ce7cd1d9f2-credential-socket\") pod \"3dff37dd-436f-470c-9b0e-71ce7cd1d9f2\" (UID: \"3dff37dd-436f-470c-9b0e-71ce7cd1d9f2\") "
Sep 09 21:07:13.930266 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:13.930087    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8jxvb\" (UniqueName: \"kubernetes.io/projected/3dff37dd-436f-470c-9b0e-71ce7cd1d9f2-kube-api-access-8jxvb\") pod \"3dff37dd-436f-470c-9b0e-71ce7cd1d9f2\" (UID: \"3dff37dd-436f-470c-9b0e-71ce7cd1d9f2\") "
Sep 09 21:07:13.930266 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:13.930154    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3dff37dd-436f-470c-9b0e-71ce7cd1d9f2-istio-data\") pod \"3dff37dd-436f-470c-9b0e-71ce7cd1d9f2\" (UID: \"3dff37dd-436f-470c-9b0e-71ce7cd1d9f2\") "
Sep 09 21:07:13.930266 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:13.930175    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3dff37dd-436f-470c-9b0e-71ce7cd1d9f2-istio-envoy\") pod \"3dff37dd-436f-470c-9b0e-71ce7cd1d9f2\" (UID: \"3dff37dd-436f-470c-9b0e-71ce7cd1d9f2\") "
Sep 09 21:07:13.930266 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:13.930214    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3dff37dd-436f-470c-9b0e-71ce7cd1d9f2-istiod-ca-cert\") pod \"3dff37dd-436f-470c-9b0e-71ce7cd1d9f2\" (UID: \"3dff37dd-436f-470c-9b0e-71ce7cd1d9f2\") "
Sep 09 21:07:13.930498 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:13.930271    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3dff37dd-436f-470c-9b0e-71ce7cd1d9f2-workload-certs\") pod \"3dff37dd-436f-470c-9b0e-71ce7cd1d9f2\" (UID: \"3dff37dd-436f-470c-9b0e-71ce7cd1d9f2\") "
Sep 09 21:07:13.930498 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:13.930293    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3dff37dd-436f-470c-9b0e-71ce7cd1d9f2-istio-token\") pod \"3dff37dd-436f-470c-9b0e-71ce7cd1d9f2\" (UID: \"3dff37dd-436f-470c-9b0e-71ce7cd1d9f2\") "
Sep 09 21:07:13.930498 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:13.930308    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3dff37dd-436f-470c-9b0e-71ce7cd1d9f2-workload-socket\") pod \"3dff37dd-436f-470c-9b0e-71ce7cd1d9f2\" (UID: \"3dff37dd-436f-470c-9b0e-71ce7cd1d9f2\") "
Sep 09 21:07:13.930498 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:13.930328    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3dff37dd-436f-470c-9b0e-71ce7cd1d9f2-istio-podinfo\") pod \"3dff37dd-436f-470c-9b0e-71ce7cd1d9f2\" (UID: \"3dff37dd-436f-470c-9b0e-71ce7cd1d9f2\") "
Sep 09 21:07:13.930498 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:13.930385    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3dff37dd-436f-470c-9b0e-71ce7cd1d9f2-istio-data" (OuterVolumeSpecName: "istio-data") pod "3dff37dd-436f-470c-9b0e-71ce7cd1d9f2" (UID: "3dff37dd-436f-470c-9b0e-71ce7cd1d9f2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:07:13.930498 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:13.930382    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3dff37dd-436f-470c-9b0e-71ce7cd1d9f2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3dff37dd-436f-470c-9b0e-71ce7cd1d9f2" (UID: "3dff37dd-436f-470c-9b0e-71ce7cd1d9f2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:07:13.930808 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:13.930565    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3dff37dd-436f-470c-9b0e-71ce7cd1d9f2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3dff37dd-436f-470c-9b0e-71ce7cd1d9f2" (UID: "3dff37dd-436f-470c-9b0e-71ce7cd1d9f2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:07:13.930808 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:13.930631    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3dff37dd-436f-470c-9b0e-71ce7cd1d9f2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3dff37dd-436f-470c-9b0e-71ce7cd1d9f2" (UID: "3dff37dd-436f-470c-9b0e-71ce7cd1d9f2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:07:13.930808 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:13.930689    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3dff37dd-436f-470c-9b0e-71ce7cd1d9f2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3dff37dd-436f-470c-9b0e-71ce7cd1d9f2" (UID: "3dff37dd-436f-470c-9b0e-71ce7cd1d9f2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:07:13.930976 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:13.930850    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3dff37dd-436f-470c-9b0e-71ce7cd1d9f2-credential-socket\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:07:13.930976 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:13.930870    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3dff37dd-436f-470c-9b0e-71ce7cd1d9f2-istio-data\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:07:13.930976 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:13.930883    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3dff37dd-436f-470c-9b0e-71ce7cd1d9f2-istiod-ca-cert\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:07:13.930976 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:13.930894    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3dff37dd-436f-470c-9b0e-71ce7cd1d9f2-workload-certs\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:07:13.930976 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:13.930905    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3dff37dd-436f-470c-9b0e-71ce7cd1d9f2-workload-socket\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:07:13.932679 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:13.932646    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3dff37dd-436f-470c-9b0e-71ce7cd1d9f2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3dff37dd-436f-470c-9b0e-71ce7cd1d9f2" (UID: "3dff37dd-436f-470c-9b0e-71ce7cd1d9f2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:07:13.932813 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:13.932737    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3dff37dd-436f-470c-9b0e-71ce7cd1d9f2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3dff37dd-436f-470c-9b0e-71ce7cd1d9f2" (UID: "3dff37dd-436f-470c-9b0e-71ce7cd1d9f2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 21:07:13.932894 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:13.932877    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3dff37dd-436f-470c-9b0e-71ce7cd1d9f2-kube-api-access-8jxvb" (OuterVolumeSpecName: "kube-api-access-8jxvb") pod "3dff37dd-436f-470c-9b0e-71ce7cd1d9f2" (UID: "3dff37dd-436f-470c-9b0e-71ce7cd1d9f2"). InnerVolumeSpecName "kube-api-access-8jxvb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:07:13.932894 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:13.932888    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3dff37dd-436f-470c-9b0e-71ce7cd1d9f2-istio-token" (OuterVolumeSpecName: "istio-token") pod "3dff37dd-436f-470c-9b0e-71ce7cd1d9f2" (UID: "3dff37dd-436f-470c-9b0e-71ce7cd1d9f2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:07:14.031776 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:14.031741    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8jxvb\" (UniqueName: \"kubernetes.io/projected/3dff37dd-436f-470c-9b0e-71ce7cd1d9f2-kube-api-access-8jxvb\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:07:14.031776 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:14.031775    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3dff37dd-436f-470c-9b0e-71ce7cd1d9f2-istio-envoy\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:07:14.031776 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:14.031784    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3dff37dd-436f-470c-9b0e-71ce7cd1d9f2-istio-token\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:07:14.032020 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:14.031792    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3dff37dd-436f-470c-9b0e-71ce7cd1d9f2-istio-podinfo\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:07:14.170884 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:14.170850    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd"]
Sep 09 21:07:14.174740 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:14.174711    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-d36b91-openshift-default-75b96fb6c4-tcgvd"]
Sep 09 21:07:14.821506 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:14.821466    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3dff37dd-436f-470c-9b0e-71ce7cd1d9f2" path="/var/lib/kubelet/pods/3dff37dd-436f-470c-9b0e-71ce7cd1d9f2/volumes"
Sep 09 21:07:17.871632 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:17.871539    2581 generic.go:356] "Generic (PLEG): container finished" podID="d6070b4c-f954-4cea-9dc8-32de78b13f0b" containerID="acf7ac9a58b1be54ab70933cc306b870c62aac305c38e2733f4a3a0e7f28eccf" exitCode=0
Sep 09 21:07:17.871632 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:17.871615    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc" event={"ID":"d6070b4c-f954-4cea-9dc8-32de78b13f0b","Type":"ContainerDied","Data":"acf7ac9a58b1be54ab70933cc306b870c62aac305c38e2733f4a3a0e7f28eccf"}
Sep 09 21:07:17.872240 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:17.872220    2581 scope.go:117] "RemoveContainer" containerID="acf7ac9a58b1be54ab70933cc306b870c62aac305c38e2733f4a3a0e7f28eccf"
Sep 09 21:07:17.873464 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:17.873432    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:07:17.873565 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:17.873502    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:07:17.873565 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:17.873534    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:07:18.878733 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:18.878696    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc" event={"ID":"d6070b4c-f954-4cea-9dc8-32de78b13f0b","Type":"ContainerStarted","Data":"56bed93fe6781b3c2225b57143598b8db2b151c97732b1479b92cc9079d772dc"}
Sep 09 21:07:19.240428 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:19.240387    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc"
Sep 09 21:07:19.241671 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:19.241647    2581 patch_prober.go:28] interesting pod/e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Sep 09 21:07:19.241734 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:19.241711    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc" podUID="d6070b4c-f954-4cea-9dc8-32de78b13f0b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Sep 09 21:07:20.240526 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:20.240479    2581 patch_prober.go:28] interesting pod/e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Sep 09 21:07:20.240934 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:20.240548    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc" podUID="d6070b4c-f954-4cea-9dc8-32de78b13f0b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Sep 09 21:07:21.191385 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:21.191353    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-1e8973-khxff"]
Sep 09 21:07:21.191824 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:21.191812    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3dff37dd-436f-470c-9b0e-71ce7cd1d9f2" containerName="istio-proxy"
Sep 09 21:07:21.191879 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:21.191827    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="3dff37dd-436f-470c-9b0e-71ce7cd1d9f2" containerName="istio-proxy"
Sep 09 21:07:21.191879 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:21.191842    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3dff37dd-436f-470c-9b0e-71ce7cd1d9f2" containerName="istio-proxy"
Sep 09 21:07:21.191879 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:21.191849    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="3dff37dd-436f-470c-9b0e-71ce7cd1d9f2" containerName="istio-proxy"
Sep 09 21:07:21.191985 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:21.191927    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="3dff37dd-436f-470c-9b0e-71ce7cd1d9f2" containerName="istio-proxy"
Sep 09 21:07:21.191985 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:21.191937    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="3dff37dd-436f-470c-9b0e-71ce7cd1d9f2" containerName="istio-proxy"
Sep 09 21:07:21.195551 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:21.195533    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-1e8973-khxff"
Sep 09 21:07:21.200703 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:21.200678    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-glz44\""
Sep 09 21:07:21.208339 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:21.208310    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-1e8973-khxff"]
Sep 09 21:07:21.240775 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:21.240736    2581 patch_prober.go:28] interesting pod/e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Sep 09 21:07:21.241205 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:21.240805    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc" podUID="d6070b4c-f954-4cea-9dc8-32de78b13f0b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Sep 09 21:07:21.314122 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:21.313881    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4bbc01b-6825-440e-8642-b5b9e4e1c0fc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-1e8973-khxff\" (UID: \"b4bbc01b-6825-440e-8642-b5b9e4e1c0fc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-1e8973-khxff"
Sep 09 21:07:21.414981 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:21.414938    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4bbc01b-6825-440e-8642-b5b9e4e1c0fc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-1e8973-khxff\" (UID: \"b4bbc01b-6825-440e-8642-b5b9e4e1c0fc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-1e8973-khxff"
Sep 09 21:07:21.415959 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:21.415925    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4bbc01b-6825-440e-8642-b5b9e4e1c0fc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-1e8973-khxff\" (UID: \"b4bbc01b-6825-440e-8642-b5b9e4e1c0fc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-1e8973-khxff"
Sep 09 21:07:21.509504 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:21.509470    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-1e8973-khxff"
Sep 09 21:07:21.792500 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:21.792476    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-1e8973-khxff"]
Sep 09 21:07:21.794201 ip-10-0-129-99 kubenswrapper[2581]: W0909 21:07:21.794172    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4bbc01b_6825_440e_8642_b5b9e4e1c0fc.slice/crio-7fc77859f2905dff3c6371defe48c78145dfa979eff8232f87bf025b16ff2e3d WatchSource:0}: Error finding container 7fc77859f2905dff3c6371defe48c78145dfa979eff8232f87bf025b16ff2e3d: Status 404 returned error can't find the container with id 7fc77859f2905dff3c6371defe48c78145dfa979eff8232f87bf025b16ff2e3d
Sep 09 21:07:21.896488 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:21.896455    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-1e8973-khxff" event={"ID":"b4bbc01b-6825-440e-8642-b5b9e4e1c0fc","Type":"ContainerStarted","Data":"7fc77859f2905dff3c6371defe48c78145dfa979eff8232f87bf025b16ff2e3d"}
Sep 09 21:07:22.240901 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:22.240858    2581 patch_prober.go:28] interesting pod/e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Sep 09 21:07:22.241355 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:22.240920    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc" podUID="d6070b4c-f954-4cea-9dc8-32de78b13f0b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Sep 09 21:07:22.906536 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:22.905920    2581 generic.go:356] "Generic (PLEG): container finished" podID="b4bbc01b-6825-440e-8642-b5b9e4e1c0fc" containerID="7918ea17671346e5b9bc4c398d3e9599fe1a1f23de9fea2a72efc4265bd3bbe6" exitCode=0
Sep 09 21:07:22.906536 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:22.906023    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-1e8973-khxff" event={"ID":"b4bbc01b-6825-440e-8642-b5b9e4e1c0fc","Type":"ContainerDied","Data":"7918ea17671346e5b9bc4c398d3e9599fe1a1f23de9fea2a72efc4265bd3bbe6"}
Sep 09 21:07:23.240397 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:23.240360    2581 patch_prober.go:28] interesting pod/e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Sep 09 21:07:23.240548 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:23.240424    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc" podUID="d6070b4c-f954-4cea-9dc8-32de78b13f0b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Sep 09 21:07:23.492671 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:23.492570    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc"
Sep 09 21:07:24.070636 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:24.070608    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-1e8973-khxff"
Sep 09 21:07:24.152559 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:24.152519    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4bbc01b-6825-440e-8642-b5b9e4e1c0fc-maas-api-service-ca\") pod \"b4bbc01b-6825-440e-8642-b5b9e4e1c0fc\" (UID: \"b4bbc01b-6825-440e-8642-b5b9e4e1c0fc\") "
Sep 09 21:07:24.152930 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:24.152901    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4bbc01b-6825-440e-8642-b5b9e4e1c0fc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b4bbc01b-6825-440e-8642-b5b9e4e1c0fc" (UID: "b4bbc01b-6825-440e-8642-b5b9e4e1c0fc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:07:24.153026 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:24.152996    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4bbc01b-6825-440e-8642-b5b9e4e1c0fc-maas-api-service-ca\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:07:24.244877 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:24.244846    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc"
Sep 09 21:07:24.245894 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:24.245873    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc"
Sep 09 21:07:24.923046 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:24.923002    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-1e8973-khxff" event={"ID":"b4bbc01b-6825-440e-8642-b5b9e4e1c0fc","Type":"ContainerDied","Data":"7fc77859f2905dff3c6371defe48c78145dfa979eff8232f87bf025b16ff2e3d"}
Sep 09 21:07:24.923046 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:24.923032    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-1e8973-khxff"
Sep 09 21:07:24.923526 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:24.923085    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7fc77859f2905dff3c6371defe48c78145dfa979eff8232f87bf025b16ff2e3d"
Sep 09 21:07:36.841287 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:36.841208    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-1e8973-f988f6d57-wgkwt"]
Sep 09 21:07:36.841810 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:36.841419    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-1e8973-f988f6d57-wgkwt" podUID="32334605-b6c1-47c4-b36a-c8eda7e5bd62" containerName="payload-processing" containerID="cri-o://b62605ee125e7f758b1bbdf25af1380f16d2fc5c6fe39a6e50c4793b831897a7" gracePeriod=30
Sep 09 21:07:36.852129 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:36.852084    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-1e8973-6d5d9c9f66-w58rq"]
Sep 09 21:07:36.853004 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:36.852310    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1e8973-6d5d9c9f66-w58rq" podUID="81af25c4-2a3d-4bc1-af8f-facfffb6a554" containerName="payload-pre-processing" containerID="cri-o://e063e27086e6510ccdb806fe47470e29b0cdfd6f6810d6b3a46fc0f461b614ef" gracePeriod=30
Sep 09 21:07:37.002320 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:37.002290    2581 generic.go:356] "Generic (PLEG): container finished" podID="81af25c4-2a3d-4bc1-af8f-facfffb6a554" containerID="e063e27086e6510ccdb806fe47470e29b0cdfd6f6810d6b3a46fc0f461b614ef" exitCode=0
Sep 09 21:07:37.002474 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:37.002361    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1e8973-6d5d9c9f66-w58rq" event={"ID":"81af25c4-2a3d-4bc1-af8f-facfffb6a554","Type":"ContainerDied","Data":"e063e27086e6510ccdb806fe47470e29b0cdfd6f6810d6b3a46fc0f461b614ef"}
Sep 09 21:07:37.004026 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:37.003974    2581 generic.go:356] "Generic (PLEG): container finished" podID="32334605-b6c1-47c4-b36a-c8eda7e5bd62" containerID="b62605ee125e7f758b1bbdf25af1380f16d2fc5c6fe39a6e50c4793b831897a7" exitCode=0
Sep 09 21:07:37.004026 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:37.004008    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-1e8973-f988f6d57-wgkwt" event={"ID":"32334605-b6c1-47c4-b36a-c8eda7e5bd62","Type":"ContainerDied","Data":"b62605ee125e7f758b1bbdf25af1380f16d2fc5c6fe39a6e50c4793b831897a7"}
Sep 09 21:07:37.115219 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:37.115190    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-1e8973-khxff"]
Sep 09 21:07:37.122407 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:37.122377    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-1e8973-khxff"]
Sep 09 21:07:37.151901 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:37.151874    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-1e8973-f988f6d57-wgkwt"
Sep 09 21:07:37.153952 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:37.153935    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1e8973-6d5d9c9f66-w58rq"
Sep 09 21:07:37.294637 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:37.294599    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32334605-b6c1-47c4-b36a-c8eda7e5bd62-plugins-config-volume\") pod \"32334605-b6c1-47c4-b36a-c8eda7e5bd62\" (UID: \"32334605-b6c1-47c4-b36a-c8eda7e5bd62\") "
Sep 09 21:07:37.294820 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:37.294650    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-65z7d\" (UniqueName: \"kubernetes.io/projected/32334605-b6c1-47c4-b36a-c8eda7e5bd62-kube-api-access-65z7d\") pod \"32334605-b6c1-47c4-b36a-c8eda7e5bd62\" (UID: \"32334605-b6c1-47c4-b36a-c8eda7e5bd62\") "
Sep 09 21:07:37.294820 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:37.294668    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/81af25c4-2a3d-4bc1-af8f-facfffb6a554-plugins-config-volume\") pod \"81af25c4-2a3d-4bc1-af8f-facfffb6a554\" (UID: \"81af25c4-2a3d-4bc1-af8f-facfffb6a554\") "
Sep 09 21:07:37.294820 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:37.294763    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vd4bx\" (UniqueName: \"kubernetes.io/projected/81af25c4-2a3d-4bc1-af8f-facfffb6a554-kube-api-access-vd4bx\") pod \"81af25c4-2a3d-4bc1-af8f-facfffb6a554\" (UID: \"81af25c4-2a3d-4bc1-af8f-facfffb6a554\") "
Sep 09 21:07:37.295009 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:37.294982    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81af25c4-2a3d-4bc1-af8f-facfffb6a554-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "81af25c4-2a3d-4bc1-af8f-facfffb6a554" (UID: "81af25c4-2a3d-4bc1-af8f-facfffb6a554"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:07:37.295069 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:37.295034    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32334605-b6c1-47c4-b36a-c8eda7e5bd62-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "32334605-b6c1-47c4-b36a-c8eda7e5bd62" (UID: "32334605-b6c1-47c4-b36a-c8eda7e5bd62"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:07:37.296927 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:37.296902    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81af25c4-2a3d-4bc1-af8f-facfffb6a554-kube-api-access-vd4bx" (OuterVolumeSpecName: "kube-api-access-vd4bx") pod "81af25c4-2a3d-4bc1-af8f-facfffb6a554" (UID: "81af25c4-2a3d-4bc1-af8f-facfffb6a554"). InnerVolumeSpecName "kube-api-access-vd4bx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:07:37.296927 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:37.296923    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32334605-b6c1-47c4-b36a-c8eda7e5bd62-kube-api-access-65z7d" (OuterVolumeSpecName: "kube-api-access-65z7d") pod "32334605-b6c1-47c4-b36a-c8eda7e5bd62" (UID: "32334605-b6c1-47c4-b36a-c8eda7e5bd62"). InnerVolumeSpecName "kube-api-access-65z7d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:07:37.396374 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:37.396294    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32334605-b6c1-47c4-b36a-c8eda7e5bd62-plugins-config-volume\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:07:37.396374 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:37.396324    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-65z7d\" (UniqueName: \"kubernetes.io/projected/32334605-b6c1-47c4-b36a-c8eda7e5bd62-kube-api-access-65z7d\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:07:37.396374 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:37.396334    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/81af25c4-2a3d-4bc1-af8f-facfffb6a554-plugins-config-volume\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:07:37.396374 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:37.396344    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vd4bx\" (UniqueName: \"kubernetes.io/projected/81af25c4-2a3d-4bc1-af8f-facfffb6a554-kube-api-access-vd4bx\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:07:38.012318 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:38.012291    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1e8973-6d5d9c9f66-w58rq"
Sep 09 21:07:38.012807 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:38.012330    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1e8973-6d5d9c9f66-w58rq" event={"ID":"81af25c4-2a3d-4bc1-af8f-facfffb6a554","Type":"ContainerDied","Data":"567202b723904f261bc2b1c9d84ca84a060caa475a7399a7d784d88b975868d9"}
Sep 09 21:07:38.012807 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:38.012402    2581 scope.go:117] "RemoveContainer" containerID="e063e27086e6510ccdb806fe47470e29b0cdfd6f6810d6b3a46fc0f461b614ef"
Sep 09 21:07:38.013793 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:38.013776    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-1e8973-f988f6d57-wgkwt"
Sep 09 21:07:38.013871 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:38.013816    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-1e8973-f988f6d57-wgkwt" event={"ID":"32334605-b6c1-47c4-b36a-c8eda7e5bd62","Type":"ContainerDied","Data":"4f884dfc082c310025a0ecdcb876c64312a0d9b5996ad99c5d0f698c008d5a37"}
Sep 09 21:07:38.021920 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:38.021901    2581 scope.go:117] "RemoveContainer" containerID="b62605ee125e7f758b1bbdf25af1380f16d2fc5c6fe39a6e50c4793b831897a7"
Sep 09 21:07:38.039638 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:38.039524    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-1e8973-6d5d9c9f66-w58rq"]
Sep 09 21:07:38.042431 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:38.042396    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-1e8973-6d5d9c9f66-w58rq"]
Sep 09 21:07:38.055268 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:38.055233    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-1e8973-f988f6d57-wgkwt"]
Sep 09 21:07:38.061004 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:38.060979    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-1e8973-f988f6d57-wgkwt"]
Sep 09 21:07:38.821136 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:38.821091    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="32334605-b6c1-47c4-b36a-c8eda7e5bd62" path="/var/lib/kubelet/pods/32334605-b6c1-47c4-b36a-c8eda7e5bd62/volumes"
Sep 09 21:07:38.821497 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:38.821484    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81af25c4-2a3d-4bc1-af8f-facfffb6a554" path="/var/lib/kubelet/pods/81af25c4-2a3d-4bc1-af8f-facfffb6a554/volumes"
Sep 09 21:07:38.821801 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:38.821790    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4bbc01b-6825-440e-8642-b5b9e4e1c0fc" path="/var/lib/kubelet/pods/b4bbc01b-6825-440e-8642-b5b9e4e1c0fc/volumes"
Sep 09 21:07:39.161428 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:39.161344    2581 scope.go:117] "RemoveContainer" containerID="3b74befa341add30d924710aa0c0104cf089b1eb850a37f3f303a0879d60e26a"
Sep 09 21:07:44.594483 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:44.594436    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc"]
Sep 09 21:07:44.594894 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:44.594678    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc" podUID="d6070b4c-f954-4cea-9dc8-32de78b13f0b" containerName="istio-proxy" containerID="cri-o://56bed93fe6781b3c2225b57143598b8db2b151c97732b1479b92cc9079d772dc" gracePeriod=30
Sep 09 21:07:44.859706 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:44.859625    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0622e2-vkjbt"]
Sep 09 21:07:44.860122 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:44.860089    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81af25c4-2a3d-4bc1-af8f-facfffb6a554" containerName="payload-pre-processing"
Sep 09 21:07:44.860122 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:44.860117    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="81af25c4-2a3d-4bc1-af8f-facfffb6a554" containerName="payload-pre-processing"
Sep 09 21:07:44.860206 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:44.860133    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32334605-b6c1-47c4-b36a-c8eda7e5bd62" containerName="payload-processing"
Sep 09 21:07:44.860206 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:44.860139    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="32334605-b6c1-47c4-b36a-c8eda7e5bd62" containerName="payload-processing"
Sep 09 21:07:44.860206 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:44.860152    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4bbc01b-6825-440e-8642-b5b9e4e1c0fc" containerName="revoke-keys"
Sep 09 21:07:44.860206 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:44.860157    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4bbc01b-6825-440e-8642-b5b9e4e1c0fc" containerName="revoke-keys"
Sep 09 21:07:44.860340 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:44.860219    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="32334605-b6c1-47c4-b36a-c8eda7e5bd62" containerName="payload-processing"
Sep 09 21:07:44.860340 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:44.860227    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4bbc01b-6825-440e-8642-b5b9e4e1c0fc" containerName="revoke-keys"
Sep 09 21:07:44.860340 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:44.860236    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="81af25c4-2a3d-4bc1-af8f-facfffb6a554" containerName="payload-pre-processing"
Sep 09 21:07:44.863401 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:44.863386    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0622e2-vkjbt"
Sep 09 21:07:44.868447 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:44.868423    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-glz44\""
Sep 09 21:07:44.872845 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:44.872822    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0622e2-vkjbt"]
Sep 09 21:07:44.979636 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:44.979602    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ed82e94-537f-4181-bf6b-796d8148c244-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-0622e2-vkjbt\" (UID: \"8ed82e94-537f-4181-bf6b-796d8148c244\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0622e2-vkjbt"
Sep 09 21:07:45.080550 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:45.080520    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ed82e94-537f-4181-bf6b-796d8148c244-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-0622e2-vkjbt\" (UID: \"8ed82e94-537f-4181-bf6b-796d8148c244\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0622e2-vkjbt"
Sep 09 21:07:45.081084 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:45.081064    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ed82e94-537f-4181-bf6b-796d8148c244-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-0622e2-vkjbt\" (UID: \"8ed82e94-537f-4181-bf6b-796d8148c244\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0622e2-vkjbt"
Sep 09 21:07:45.175945 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:45.175858    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0622e2-vkjbt"
Sep 09 21:07:45.483146 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:45.483121    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0622e2-vkjbt"]
Sep 09 21:07:45.484847 ip-10-0-129-99 kubenswrapper[2581]: W0909 21:07:45.484821    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ed82e94_537f_4181_bf6b_796d8148c244.slice/crio-1a2ced7cdc337208fab9b1bb64a3e247dec3c4334bc1573de930aa29af5a3ee9 WatchSource:0}: Error finding container 1a2ced7cdc337208fab9b1bb64a3e247dec3c4334bc1573de930aa29af5a3ee9: Status 404 returned error can't find the container with id 1a2ced7cdc337208fab9b1bb64a3e247dec3c4334bc1573de930aa29af5a3ee9
Sep 09 21:07:46.067771 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:46.067679    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0622e2-vkjbt" event={"ID":"8ed82e94-537f-4181-bf6b-796d8148c244","Type":"ContainerStarted","Data":"bc2b81e3e7ec16af20bd3db59ffbdd3c248c3f725dfe602b3c13ab87e37f05ba"}
Sep 09 21:07:46.067771 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:46.067745    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0622e2-vkjbt" event={"ID":"8ed82e94-537f-4181-bf6b-796d8148c244","Type":"ContainerStarted","Data":"1a2ced7cdc337208fab9b1bb64a3e247dec3c4334bc1573de930aa29af5a3ee9"}
Sep 09 21:07:46.086748 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:46.086699    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0622e2-vkjbt" podStartSLOduration=2.08668443 podStartE2EDuration="2.08668443s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:07:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:07:46.085654919 +0000 UTC m=+1627.713282793" watchObservedRunningTime="2026-09-09 21:07:46.08668443 +0000 UTC m=+1627.714312304"
Sep 09 21:07:47.074513 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:47.074476    2581 generic.go:356] "Generic (PLEG): container finished" podID="8ed82e94-537f-4181-bf6b-796d8148c244" containerID="bc2b81e3e7ec16af20bd3db59ffbdd3c248c3f725dfe602b3c13ab87e37f05ba" exitCode=0
Sep 09 21:07:47.074910 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:47.074538    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0622e2-vkjbt" event={"ID":"8ed82e94-537f-4181-bf6b-796d8148c244","Type":"ContainerDied","Data":"bc2b81e3e7ec16af20bd3db59ffbdd3c248c3f725dfe602b3c13ab87e37f05ba"}
Sep 09 21:07:48.219235 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:48.219208    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0622e2-vkjbt"
Sep 09 21:07:48.414876 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:48.414790    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ed82e94-537f-4181-bf6b-796d8148c244-maas-api-service-ca\") pod \"8ed82e94-537f-4181-bf6b-796d8148c244\" (UID: \"8ed82e94-537f-4181-bf6b-796d8148c244\") "
Sep 09 21:07:48.415134 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:48.415088    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ed82e94-537f-4181-bf6b-796d8148c244-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8ed82e94-537f-4181-bf6b-796d8148c244" (UID: "8ed82e94-537f-4181-bf6b-796d8148c244"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:07:48.516267 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:48.516232    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ed82e94-537f-4181-bf6b-796d8148c244-maas-api-service-ca\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:07:49.088042 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:49.087949    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0622e2-vkjbt"
Sep 09 21:07:49.088215 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:49.087949    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0622e2-vkjbt" event={"ID":"8ed82e94-537f-4181-bf6b-796d8148c244","Type":"ContainerDied","Data":"1a2ced7cdc337208fab9b1bb64a3e247dec3c4334bc1573de930aa29af5a3ee9"}
Sep 09 21:07:49.088215 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:49.088151    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1a2ced7cdc337208fab9b1bb64a3e247dec3c4334bc1573de930aa29af5a3ee9"
Sep 09 21:07:49.860049 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:49.859992    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc"
Sep 09 21:07:49.931357 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:49.931327    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6070b4c-f954-4cea-9dc8-32de78b13f0b-credential-socket\") pod \"d6070b4c-f954-4cea-9dc8-32de78b13f0b\" (UID: \"d6070b4c-f954-4cea-9dc8-32de78b13f0b\") "
Sep 09 21:07:49.931514 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:49.931363    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6070b4c-f954-4cea-9dc8-32de78b13f0b-istiod-ca-cert\") pod \"d6070b4c-f954-4cea-9dc8-32de78b13f0b\" (UID: \"d6070b4c-f954-4cea-9dc8-32de78b13f0b\") "
Sep 09 21:07:49.931514 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:49.931393    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6070b4c-f954-4cea-9dc8-32de78b13f0b-workload-certs\") pod \"d6070b4c-f954-4cea-9dc8-32de78b13f0b\" (UID: \"d6070b4c-f954-4cea-9dc8-32de78b13f0b\") "
Sep 09 21:07:49.931514 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:49.931428    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6070b4c-f954-4cea-9dc8-32de78b13f0b-istio-podinfo\") pod \"d6070b4c-f954-4cea-9dc8-32de78b13f0b\" (UID: \"d6070b4c-f954-4cea-9dc8-32de78b13f0b\") "
Sep 09 21:07:49.931514 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:49.931479    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6070b4c-f954-4cea-9dc8-32de78b13f0b-istio-token\") pod \"d6070b4c-f954-4cea-9dc8-32de78b13f0b\" (UID: \"d6070b4c-f954-4cea-9dc8-32de78b13f0b\") "
Sep 09 21:07:49.931715 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:49.931540    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wnct9\" (UniqueName: \"kubernetes.io/projected/d6070b4c-f954-4cea-9dc8-32de78b13f0b-kube-api-access-wnct9\") pod \"d6070b4c-f954-4cea-9dc8-32de78b13f0b\" (UID: \"d6070b4c-f954-4cea-9dc8-32de78b13f0b\") "
Sep 09 21:07:49.931715 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:49.931570    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6070b4c-f954-4cea-9dc8-32de78b13f0b-istio-envoy\") pod \"d6070b4c-f954-4cea-9dc8-32de78b13f0b\" (UID: \"d6070b4c-f954-4cea-9dc8-32de78b13f0b\") "
Sep 09 21:07:49.931715 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:49.931622    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6070b4c-f954-4cea-9dc8-32de78b13f0b-workload-socket\") pod \"d6070b4c-f954-4cea-9dc8-32de78b13f0b\" (UID: \"d6070b4c-f954-4cea-9dc8-32de78b13f0b\") "
Sep 09 21:07:49.931715 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:49.931636    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6070b4c-f954-4cea-9dc8-32de78b13f0b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d6070b4c-f954-4cea-9dc8-32de78b13f0b" (UID: "d6070b4c-f954-4cea-9dc8-32de78b13f0b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:07:49.931923 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:49.931784    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6070b4c-f954-4cea-9dc8-32de78b13f0b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d6070b4c-f954-4cea-9dc8-32de78b13f0b" (UID: "d6070b4c-f954-4cea-9dc8-32de78b13f0b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:07:49.931923 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:49.931784    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6070b4c-f954-4cea-9dc8-32de78b13f0b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d6070b4c-f954-4cea-9dc8-32de78b13f0b" (UID: "d6070b4c-f954-4cea-9dc8-32de78b13f0b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:07:49.931923 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:49.931871    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6070b4c-f954-4cea-9dc8-32de78b13f0b-istio-data\") pod \"d6070b4c-f954-4cea-9dc8-32de78b13f0b\" (UID: \"d6070b4c-f954-4cea-9dc8-32de78b13f0b\") "
Sep 09 21:07:49.932114 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:49.931971    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6070b4c-f954-4cea-9dc8-32de78b13f0b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d6070b4c-f954-4cea-9dc8-32de78b13f0b" (UID: "d6070b4c-f954-4cea-9dc8-32de78b13f0b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:07:49.932204 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:49.932179    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6070b4c-f954-4cea-9dc8-32de78b13f0b-istio-data" (OuterVolumeSpecName: "istio-data") pod "d6070b4c-f954-4cea-9dc8-32de78b13f0b" (UID: "d6070b4c-f954-4cea-9dc8-32de78b13f0b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:07:49.932445 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:49.932424    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6070b4c-f954-4cea-9dc8-32de78b13f0b-workload-socket\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:07:49.932516 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:49.932449    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6070b4c-f954-4cea-9dc8-32de78b13f0b-istio-data\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:07:49.932516 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:49.932464    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6070b4c-f954-4cea-9dc8-32de78b13f0b-credential-socket\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:07:49.932516 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:49.932477    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6070b4c-f954-4cea-9dc8-32de78b13f0b-istiod-ca-cert\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:07:49.932516 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:49.932490    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6070b4c-f954-4cea-9dc8-32de78b13f0b-workload-certs\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:07:49.933765 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:49.933718    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6070b4c-f954-4cea-9dc8-32de78b13f0b-istio-token" (OuterVolumeSpecName: "istio-token") pod "d6070b4c-f954-4cea-9dc8-32de78b13f0b" (UID: "d6070b4c-f954-4cea-9dc8-32de78b13f0b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:07:49.933765 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:49.933726    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d6070b4c-f954-4cea-9dc8-32de78b13f0b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d6070b4c-f954-4cea-9dc8-32de78b13f0b" (UID: "d6070b4c-f954-4cea-9dc8-32de78b13f0b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 21:07:49.933890 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:49.933789    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6070b4c-f954-4cea-9dc8-32de78b13f0b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d6070b4c-f954-4cea-9dc8-32de78b13f0b" (UID: "d6070b4c-f954-4cea-9dc8-32de78b13f0b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:07:49.933890 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:49.933812    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6070b4c-f954-4cea-9dc8-32de78b13f0b-kube-api-access-wnct9" (OuterVolumeSpecName: "kube-api-access-wnct9") pod "d6070b4c-f954-4cea-9dc8-32de78b13f0b" (UID: "d6070b4c-f954-4cea-9dc8-32de78b13f0b"). InnerVolumeSpecName "kube-api-access-wnct9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:07:50.034057 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:50.034019    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wnct9\" (UniqueName: \"kubernetes.io/projected/d6070b4c-f954-4cea-9dc8-32de78b13f0b-kube-api-access-wnct9\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:07:50.034057 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:50.034052    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6070b4c-f954-4cea-9dc8-32de78b13f0b-istio-envoy\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:07:50.034057 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:50.034063    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6070b4c-f954-4cea-9dc8-32de78b13f0b-istio-podinfo\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:07:50.034326 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:50.034079    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6070b4c-f954-4cea-9dc8-32de78b13f0b-istio-token\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:07:50.095573 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:50.095536    2581 generic.go:356] "Generic (PLEG): container finished" podID="d6070b4c-f954-4cea-9dc8-32de78b13f0b" containerID="56bed93fe6781b3c2225b57143598b8db2b151c97732b1479b92cc9079d772dc" exitCode=0
Sep 09 21:07:50.095729 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:50.095588    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc"
Sep 09 21:07:50.095729 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:50.095606    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc" event={"ID":"d6070b4c-f954-4cea-9dc8-32de78b13f0b","Type":"ContainerDied","Data":"56bed93fe6781b3c2225b57143598b8db2b151c97732b1479b92cc9079d772dc"}
Sep 09 21:07:50.095729 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:50.095691    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc" event={"ID":"d6070b4c-f954-4cea-9dc8-32de78b13f0b","Type":"ContainerDied","Data":"a72e9cfb39e7ea34b1a767fc7ad91a6ad7c9d4dec024a9293dd15d743ef42bd7"}
Sep 09 21:07:50.095729 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:50.095719    2581 scope.go:117] "RemoveContainer" containerID="56bed93fe6781b3c2225b57143598b8db2b151c97732b1479b92cc9079d772dc"
Sep 09 21:07:50.105500 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:50.105483    2581 scope.go:117] "RemoveContainer" containerID="acf7ac9a58b1be54ab70933cc306b870c62aac305c38e2733f4a3a0e7f28eccf"
Sep 09 21:07:50.114163 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:50.114144    2581 scope.go:117] "RemoveContainer" containerID="56bed93fe6781b3c2225b57143598b8db2b151c97732b1479b92cc9079d772dc"
Sep 09 21:07:50.114425 ip-10-0-129-99 kubenswrapper[2581]: E0909 21:07:50.114409    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56bed93fe6781b3c2225b57143598b8db2b151c97732b1479b92cc9079d772dc\": container with ID starting with 56bed93fe6781b3c2225b57143598b8db2b151c97732b1479b92cc9079d772dc not found: ID does not exist" containerID="56bed93fe6781b3c2225b57143598b8db2b151c97732b1479b92cc9079d772dc"
Sep 09 21:07:50.114475 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:50.114432    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56bed93fe6781b3c2225b57143598b8db2b151c97732b1479b92cc9079d772dc"} err="failed to get container status \"56bed93fe6781b3c2225b57143598b8db2b151c97732b1479b92cc9079d772dc\": rpc error: code = NotFound desc = could not find container \"56bed93fe6781b3c2225b57143598b8db2b151c97732b1479b92cc9079d772dc\": container with ID starting with 56bed93fe6781b3c2225b57143598b8db2b151c97732b1479b92cc9079d772dc not found: ID does not exist"
Sep 09 21:07:50.114475 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:50.114448    2581 scope.go:117] "RemoveContainer" containerID="acf7ac9a58b1be54ab70933cc306b870c62aac305c38e2733f4a3a0e7f28eccf"
Sep 09 21:07:50.114691 ip-10-0-129-99 kubenswrapper[2581]: E0909 21:07:50.114672    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"acf7ac9a58b1be54ab70933cc306b870c62aac305c38e2733f4a3a0e7f28eccf\": container with ID starting with acf7ac9a58b1be54ab70933cc306b870c62aac305c38e2733f4a3a0e7f28eccf not found: ID does not exist" containerID="acf7ac9a58b1be54ab70933cc306b870c62aac305c38e2733f4a3a0e7f28eccf"
Sep 09 21:07:50.114731 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:50.114698    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"acf7ac9a58b1be54ab70933cc306b870c62aac305c38e2733f4a3a0e7f28eccf"} err="failed to get container status \"acf7ac9a58b1be54ab70933cc306b870c62aac305c38e2733f4a3a0e7f28eccf\": rpc error: code = NotFound desc = could not find container \"acf7ac9a58b1be54ab70933cc306b870c62aac305c38e2733f4a3a0e7f28eccf\": container with ID starting with acf7ac9a58b1be54ab70933cc306b870c62aac305c38e2733f4a3a0e7f28eccf not found: ID does not exist"
Sep 09 21:07:50.119839 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:50.119814    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc"]
Sep 09 21:07:50.124279 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:50.124258    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-1e8973-openshift-default-7f5f4b4894-chwkc"]
Sep 09 21:07:50.821568 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:07:50.821534    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6070b4c-f954-4cea-9dc8-32de78b13f0b" path="/var/lib/kubelet/pods/d6070b4c-f954-4cea-9dc8-32de78b13f0b/volumes"
Sep 09 21:08:00.489025 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:00.488987    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-0622e2-7f4cc4664b-2jft4"]
Sep 09 21:08:00.489537 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:00.489271    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0622e2-7f4cc4664b-2jft4" podUID="a461f916-8fba-4ea4-a2b6-c7b1694970bd" containerName="payload-pre-processing" containerID="cri-o://99d5ab8c241c10fe8e9c56a2d8e898cc5aa16a33a6265ee842def55683da76ec" gracePeriod=30
Sep 09 21:08:00.777785 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:00.777758    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0622e2-7f4cc4664b-2jft4"
Sep 09 21:08:00.848157 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:00.848114    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0622e2-vkjbt"]
Sep 09 21:08:00.848616 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:00.848590    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a461f916-8fba-4ea4-a2b6-c7b1694970bd-plugins-config-volume\") pod \"a461f916-8fba-4ea4-a2b6-c7b1694970bd\" (UID: \"a461f916-8fba-4ea4-a2b6-c7b1694970bd\") "
Sep 09 21:08:00.848710 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:00.848661    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5hsz9\" (UniqueName: \"kubernetes.io/projected/a461f916-8fba-4ea4-a2b6-c7b1694970bd-kube-api-access-5hsz9\") pod \"a461f916-8fba-4ea4-a2b6-c7b1694970bd\" (UID: \"a461f916-8fba-4ea4-a2b6-c7b1694970bd\") "
Sep 09 21:08:00.849896 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:00.849711    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a461f916-8fba-4ea4-a2b6-c7b1694970bd-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a461f916-8fba-4ea4-a2b6-c7b1694970bd" (UID: "a461f916-8fba-4ea4-a2b6-c7b1694970bd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:08:00.851671 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:00.851644    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a461f916-8fba-4ea4-a2b6-c7b1694970bd-kube-api-access-5hsz9" (OuterVolumeSpecName: "kube-api-access-5hsz9") pod "a461f916-8fba-4ea4-a2b6-c7b1694970bd" (UID: "a461f916-8fba-4ea4-a2b6-c7b1694970bd"). InnerVolumeSpecName "kube-api-access-5hsz9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:08:00.858495 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:00.858469    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0622e2-vkjbt"]
Sep 09 21:08:00.950534 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:00.950501    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a461f916-8fba-4ea4-a2b6-c7b1694970bd-plugins-config-volume\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:08:00.950534 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:00.950531    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5hsz9\" (UniqueName: \"kubernetes.io/projected/a461f916-8fba-4ea4-a2b6-c7b1694970bd-kube-api-access-5hsz9\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:08:01.160632 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:01.160528    2581 generic.go:356] "Generic (PLEG): container finished" podID="a461f916-8fba-4ea4-a2b6-c7b1694970bd" containerID="99d5ab8c241c10fe8e9c56a2d8e898cc5aa16a33a6265ee842def55683da76ec" exitCode=0
Sep 09 21:08:01.160632 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:01.160592    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0622e2-7f4cc4664b-2jft4"
Sep 09 21:08:01.160632 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:01.160619    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0622e2-7f4cc4664b-2jft4" event={"ID":"a461f916-8fba-4ea4-a2b6-c7b1694970bd","Type":"ContainerDied","Data":"99d5ab8c241c10fe8e9c56a2d8e898cc5aa16a33a6265ee842def55683da76ec"}
Sep 09 21:08:01.160915 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:01.160700    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0622e2-7f4cc4664b-2jft4" event={"ID":"a461f916-8fba-4ea4-a2b6-c7b1694970bd","Type":"ContainerDied","Data":"dd3b484c11f39aea1411448f546163a85f008b77480d6b2b85f5f659ddb339a1"}
Sep 09 21:08:01.160915 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:01.160725    2581 scope.go:117] "RemoveContainer" containerID="99d5ab8c241c10fe8e9c56a2d8e898cc5aa16a33a6265ee842def55683da76ec"
Sep 09 21:08:01.169900 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:01.169884    2581 scope.go:117] "RemoveContainer" containerID="99d5ab8c241c10fe8e9c56a2d8e898cc5aa16a33a6265ee842def55683da76ec"
Sep 09 21:08:01.170216 ip-10-0-129-99 kubenswrapper[2581]: E0909 21:08:01.170196    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99d5ab8c241c10fe8e9c56a2d8e898cc5aa16a33a6265ee842def55683da76ec\": container with ID starting with 99d5ab8c241c10fe8e9c56a2d8e898cc5aa16a33a6265ee842def55683da76ec not found: ID does not exist" containerID="99d5ab8c241c10fe8e9c56a2d8e898cc5aa16a33a6265ee842def55683da76ec"
Sep 09 21:08:01.170281 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:01.170224    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99d5ab8c241c10fe8e9c56a2d8e898cc5aa16a33a6265ee842def55683da76ec"} err="failed to get container status \"99d5ab8c241c10fe8e9c56a2d8e898cc5aa16a33a6265ee842def55683da76ec\": rpc error: code = NotFound desc = could not find container \"99d5ab8c241c10fe8e9c56a2d8e898cc5aa16a33a6265ee842def55683da76ec\": container with ID starting with 99d5ab8c241c10fe8e9c56a2d8e898cc5aa16a33a6265ee842def55683da76ec not found: ID does not exist"
Sep 09 21:08:01.186201 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:01.186177    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-0622e2-7f4cc4664b-2jft4"]
Sep 09 21:08:01.190612 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:01.190588    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-0622e2-7f4cc4664b-2jft4"]
Sep 09 21:08:02.823341 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:02.823305    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ed82e94-537f-4181-bf6b-796d8148c244" path="/var/lib/kubelet/pods/8ed82e94-537f-4181-bf6b-796d8148c244/volumes"
Sep 09 21:08:02.823892 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:02.823870    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a461f916-8fba-4ea4-a2b6-c7b1694970bd" path="/var/lib/kubelet/pods/a461f916-8fba-4ea4-a2b6-c7b1694970bd/volumes"
Sep 09 21:08:08.183870 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:08.183829    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e73aab-v5pb8"]
Sep 09 21:08:08.184327 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:08.184304    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ed82e94-537f-4181-bf6b-796d8148c244" containerName="revoke-keys"
Sep 09 21:08:08.184327 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:08.184317    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ed82e94-537f-4181-bf6b-796d8148c244" containerName="revoke-keys"
Sep 09 21:08:08.184410 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:08.184332    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a461f916-8fba-4ea4-a2b6-c7b1694970bd" containerName="payload-pre-processing"
Sep 09 21:08:08.184410 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:08.184338    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a461f916-8fba-4ea4-a2b6-c7b1694970bd" containerName="payload-pre-processing"
Sep 09 21:08:08.184410 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:08.184352    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6070b4c-f954-4cea-9dc8-32de78b13f0b" containerName="istio-proxy"
Sep 09 21:08:08.184410 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:08.184359    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6070b4c-f954-4cea-9dc8-32de78b13f0b" containerName="istio-proxy"
Sep 09 21:08:08.184410 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:08.184370    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6070b4c-f954-4cea-9dc8-32de78b13f0b" containerName="istio-proxy"
Sep 09 21:08:08.184410 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:08.184375    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6070b4c-f954-4cea-9dc8-32de78b13f0b" containerName="istio-proxy"
Sep 09 21:08:08.184594 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:08.184441    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ed82e94-537f-4181-bf6b-796d8148c244" containerName="revoke-keys"
Sep 09 21:08:08.184594 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:08.184451    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6070b4c-f954-4cea-9dc8-32de78b13f0b" containerName="istio-proxy"
Sep 09 21:08:08.184594 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:08.184457    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a461f916-8fba-4ea4-a2b6-c7b1694970bd" containerName="payload-pre-processing"
Sep 09 21:08:08.184594 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:08.184467    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6070b4c-f954-4cea-9dc8-32de78b13f0b" containerName="istio-proxy"
Sep 09 21:08:08.189193 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:08.189176    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e73aab-v5pb8"
Sep 09 21:08:08.192716 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:08.192690    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-glz44\""
Sep 09 21:08:08.204846 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:08.204822    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e73aab-v5pb8"]
Sep 09 21:08:08.331872 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:08.331834    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd985671-74d9-4116-8fa2-c754c8d19fbb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-e73aab-v5pb8\" (UID: \"dd985671-74d9-4116-8fa2-c754c8d19fbb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e73aab-v5pb8"
Sep 09 21:08:08.432598 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:08.432561    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd985671-74d9-4116-8fa2-c754c8d19fbb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-e73aab-v5pb8\" (UID: \"dd985671-74d9-4116-8fa2-c754c8d19fbb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e73aab-v5pb8"
Sep 09 21:08:08.433219 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:08.433200    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd985671-74d9-4116-8fa2-c754c8d19fbb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-e73aab-v5pb8\" (UID: \"dd985671-74d9-4116-8fa2-c754c8d19fbb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e73aab-v5pb8"
Sep 09 21:08:08.499053 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:08.499024    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e73aab-v5pb8"
Sep 09 21:08:08.770479 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:08.770453    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e73aab-v5pb8"]
Sep 09 21:08:08.772232 ip-10-0-129-99 kubenswrapper[2581]: W0909 21:08:08.772206    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd985671_74d9_4116_8fa2_c754c8d19fbb.slice/crio-e4503a046b013daa0d32c40614b74bd48f2943cd83b57d6c70379104a31b5e0f WatchSource:0}: Error finding container e4503a046b013daa0d32c40614b74bd48f2943cd83b57d6c70379104a31b5e0f: Status 404 returned error can't find the container with id e4503a046b013daa0d32c40614b74bd48f2943cd83b57d6c70379104a31b5e0f
Sep 09 21:08:09.210711 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:09.210672    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e73aab-v5pb8" event={"ID":"dd985671-74d9-4116-8fa2-c754c8d19fbb","Type":"ContainerStarted","Data":"8f67311221fc43072e4a2b4c73402d25e878b77a2b8ae4ac13214de2a944efaa"}
Sep 09 21:08:09.210711 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:09.210729    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e73aab-v5pb8" event={"ID":"dd985671-74d9-4116-8fa2-c754c8d19fbb","Type":"ContainerStarted","Data":"e4503a046b013daa0d32c40614b74bd48f2943cd83b57d6c70379104a31b5e0f"}
Sep 09 21:08:09.235702 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:09.235647    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e73aab-v5pb8" podStartSLOduration=1.235628204 podStartE2EDuration="1.235628204s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:08:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:08:09.232778563 +0000 UTC m=+1650.860406436" watchObservedRunningTime="2026-09-09 21:08:09.235628204 +0000 UTC m=+1650.863256077"
Sep 09 21:08:10.218278 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:10.218242    2581 generic.go:356] "Generic (PLEG): container finished" podID="dd985671-74d9-4116-8fa2-c754c8d19fbb" containerID="8f67311221fc43072e4a2b4c73402d25e878b77a2b8ae4ac13214de2a944efaa" exitCode=0
Sep 09 21:08:10.218691 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:10.218314    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e73aab-v5pb8" event={"ID":"dd985671-74d9-4116-8fa2-c754c8d19fbb","Type":"ContainerDied","Data":"8f67311221fc43072e4a2b4c73402d25e878b77a2b8ae4ac13214de2a944efaa"}
Sep 09 21:08:11.363259 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:11.363229    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e73aab-v5pb8"
Sep 09 21:08:11.466381 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:11.466348    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd985671-74d9-4116-8fa2-c754c8d19fbb-maas-api-service-ca\") pod \"dd985671-74d9-4116-8fa2-c754c8d19fbb\" (UID: \"dd985671-74d9-4116-8fa2-c754c8d19fbb\") "
Sep 09 21:08:11.466692 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:11.466666    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd985671-74d9-4116-8fa2-c754c8d19fbb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "dd985671-74d9-4116-8fa2-c754c8d19fbb" (UID: "dd985671-74d9-4116-8fa2-c754c8d19fbb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:08:11.567689 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:11.567657    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd985671-74d9-4116-8fa2-c754c8d19fbb-maas-api-service-ca\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:08:12.231168 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:12.231130    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e73aab-v5pb8" event={"ID":"dd985671-74d9-4116-8fa2-c754c8d19fbb","Type":"ContainerDied","Data":"e4503a046b013daa0d32c40614b74bd48f2943cd83b57d6c70379104a31b5e0f"}
Sep 09 21:08:12.231168 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:12.231161    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e73aab-v5pb8"
Sep 09 21:08:12.231392 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:12.231192    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e4503a046b013daa0d32c40614b74bd48f2943cd83b57d6c70379104a31b5e0f"
Sep 09 21:08:18.938789 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:18.938749    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-e73aab-29816460-z97p9"]
Sep 09 21:08:18.941140 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:18.941123    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-e73aab-29816460-z97p9"]
Sep 09 21:08:18.992091 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:18.992060    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-e73aab-6678d9b6cf-6rqh2"]
Sep 09 21:08:18.992423 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:18.992357    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e73aab-6678d9b6cf-6rqh2" podUID="5f8d283b-a4b9-4e8c-8b3d-e793c73e7f71" containerName="payload-processing" containerID="cri-o://2160abc018e3bdf52afc07805dfa83c7b4ce86afcce864d39ac57234a23723fb" gracePeriod=30
Sep 09 21:08:19.016073 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:19.016050    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e73aab-7f86bb8d78-5q7mc"]
Sep 09 21:08:19.016307 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:19.016270    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e73aab-7f86bb8d78-5q7mc" podUID="254f0e4f-9e77-4b6c-82e4-fe37777ad5bd" containerName="payload-pre-processing" containerID="cri-o://d5c3a5ed4d386757887a2022958b92564ea57735aeaf1c1e3b55189f5ed86f5a" gracePeriod=30
Sep 09 21:08:19.280307 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:19.280269    2581 generic.go:356] "Generic (PLEG): container finished" podID="254f0e4f-9e77-4b6c-82e4-fe37777ad5bd" containerID="d5c3a5ed4d386757887a2022958b92564ea57735aeaf1c1e3b55189f5ed86f5a" exitCode=0
Sep 09 21:08:19.280432 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:19.280348    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e73aab-7f86bb8d78-5q7mc" event={"ID":"254f0e4f-9e77-4b6c-82e4-fe37777ad5bd","Type":"ContainerDied","Data":"d5c3a5ed4d386757887a2022958b92564ea57735aeaf1c1e3b55189f5ed86f5a"}
Sep 09 21:08:19.282004 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:19.281979    2581 generic.go:356] "Generic (PLEG): container finished" podID="5f8d283b-a4b9-4e8c-8b3d-e793c73e7f71" containerID="2160abc018e3bdf52afc07805dfa83c7b4ce86afcce864d39ac57234a23723fb" exitCode=0
Sep 09 21:08:19.282166 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:19.282073    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e73aab-6678d9b6cf-6rqh2" event={"ID":"5f8d283b-a4b9-4e8c-8b3d-e793c73e7f71","Type":"ContainerDied","Data":"2160abc018e3bdf52afc07805dfa83c7b4ce86afcce864d39ac57234a23723fb"}
Sep 09 21:08:19.282166 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:19.282134    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e73aab-6678d9b6cf-6rqh2" event={"ID":"5f8d283b-a4b9-4e8c-8b3d-e793c73e7f71","Type":"ContainerDied","Data":"9c604c8007c619c3270942e0b2eb6f13dfe187cae43af9185cb1b085f4f4f3de"}
Sep 09 21:08:19.282166 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:19.282160    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9c604c8007c619c3270942e0b2eb6f13dfe187cae43af9185cb1b085f4f4f3de"
Sep 09 21:08:19.284347 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:19.284327    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e73aab-6678d9b6cf-6rqh2"
Sep 09 21:08:19.308558 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:19.308541    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e73aab-7f86bb8d78-5q7mc"
Sep 09 21:08:19.341738 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:19.341714    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e73aab-v5pb8"]
Sep 09 21:08:19.344794 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:19.344765    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e73aab-v5pb8"]
Sep 09 21:08:19.446110 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:19.446052    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xc69v\" (UniqueName: \"kubernetes.io/projected/254f0e4f-9e77-4b6c-82e4-fe37777ad5bd-kube-api-access-xc69v\") pod \"254f0e4f-9e77-4b6c-82e4-fe37777ad5bd\" (UID: \"254f0e4f-9e77-4b6c-82e4-fe37777ad5bd\") "
Sep 09 21:08:19.446281 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:19.446153    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f8d283b-a4b9-4e8c-8b3d-e793c73e7f71-plugins-config-volume\") pod \"5f8d283b-a4b9-4e8c-8b3d-e793c73e7f71\" (UID: \"5f8d283b-a4b9-4e8c-8b3d-e793c73e7f71\") "
Sep 09 21:08:19.446281 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:19.446198    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/254f0e4f-9e77-4b6c-82e4-fe37777ad5bd-plugins-config-volume\") pod \"254f0e4f-9e77-4b6c-82e4-fe37777ad5bd\" (UID: \"254f0e4f-9e77-4b6c-82e4-fe37777ad5bd\") "
Sep 09 21:08:19.446281 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:19.446261    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fqzzv\" (UniqueName: \"kubernetes.io/projected/5f8d283b-a4b9-4e8c-8b3d-e793c73e7f71-kube-api-access-fqzzv\") pod \"5f8d283b-a4b9-4e8c-8b3d-e793c73e7f71\" (UID: \"5f8d283b-a4b9-4e8c-8b3d-e793c73e7f71\") "
Sep 09 21:08:19.446590 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:19.446562    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/254f0e4f-9e77-4b6c-82e4-fe37777ad5bd-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "254f0e4f-9e77-4b6c-82e4-fe37777ad5bd" (UID: "254f0e4f-9e77-4b6c-82e4-fe37777ad5bd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:08:19.446660 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:19.446561    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f8d283b-a4b9-4e8c-8b3d-e793c73e7f71-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5f8d283b-a4b9-4e8c-8b3d-e793c73e7f71" (UID: "5f8d283b-a4b9-4e8c-8b3d-e793c73e7f71"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:08:19.446710 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:19.446659    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f8d283b-a4b9-4e8c-8b3d-e793c73e7f71-plugins-config-volume\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:08:19.446710 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:19.446676    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/254f0e4f-9e77-4b6c-82e4-fe37777ad5bd-plugins-config-volume\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:08:19.448387 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:19.448363    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/254f0e4f-9e77-4b6c-82e4-fe37777ad5bd-kube-api-access-xc69v" (OuterVolumeSpecName: "kube-api-access-xc69v") pod "254f0e4f-9e77-4b6c-82e4-fe37777ad5bd" (UID: "254f0e4f-9e77-4b6c-82e4-fe37777ad5bd"). InnerVolumeSpecName "kube-api-access-xc69v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:08:19.448507 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:19.448498    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f8d283b-a4b9-4e8c-8b3d-e793c73e7f71-kube-api-access-fqzzv" (OuterVolumeSpecName: "kube-api-access-fqzzv") pod "5f8d283b-a4b9-4e8c-8b3d-e793c73e7f71" (UID: "5f8d283b-a4b9-4e8c-8b3d-e793c73e7f71"). InnerVolumeSpecName "kube-api-access-fqzzv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:08:19.548064 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:19.548028    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fqzzv\" (UniqueName: \"kubernetes.io/projected/5f8d283b-a4b9-4e8c-8b3d-e793c73e7f71-kube-api-access-fqzzv\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:08:19.548064 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:19.548065    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xc69v\" (UniqueName: \"kubernetes.io/projected/254f0e4f-9e77-4b6c-82e4-fe37777ad5bd-kube-api-access-xc69v\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:08:20.289806 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:20.289713    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e73aab-7f86bb8d78-5q7mc"
Sep 09 21:08:20.289806 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:20.289737    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e73aab-6678d9b6cf-6rqh2"
Sep 09 21:08:20.289806 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:20.289744    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e73aab-7f86bb8d78-5q7mc" event={"ID":"254f0e4f-9e77-4b6c-82e4-fe37777ad5bd","Type":"ContainerDied","Data":"a7ca85c6ede25fd1deb3a9ca5c308d35c9f66918d88cd27828ee0fdadf58cef3"}
Sep 09 21:08:20.290362 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:20.289808    2581 scope.go:117] "RemoveContainer" containerID="d5c3a5ed4d386757887a2022958b92564ea57735aeaf1c1e3b55189f5ed86f5a"
Sep 09 21:08:20.321856 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:20.321829    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e73aab-7f86bb8d78-5q7mc"]
Sep 09 21:08:20.326680 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:20.326651    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e73aab-7f86bb8d78-5q7mc"]
Sep 09 21:08:20.340487 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:20.340466    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-e73aab-6678d9b6cf-6rqh2"]
Sep 09 21:08:20.345603 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:20.345581    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-e73aab-6678d9b6cf-6rqh2"]
Sep 09 21:08:20.820761 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:20.820725    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="254f0e4f-9e77-4b6c-82e4-fe37777ad5bd" path="/var/lib/kubelet/pods/254f0e4f-9e77-4b6c-82e4-fe37777ad5bd/volumes"
Sep 09 21:08:20.821077 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:20.821065    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f8d283b-a4b9-4e8c-8b3d-e793c73e7f71" path="/var/lib/kubelet/pods/5f8d283b-a4b9-4e8c-8b3d-e793c73e7f71/volumes"
Sep 09 21:08:20.821352 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:20.821339    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6653cb7-c4bf-4703-87b7-40ea28390c44" path="/var/lib/kubelet/pods/b6653cb7-c4bf-4703-87b7-40ea28390c44/volumes"
Sep 09 21:08:20.821595 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:20.821583    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd985671-74d9-4116-8fa2-c754c8d19fbb" path="/var/lib/kubelet/pods/dd985671-74d9-4116-8fa2-c754c8d19fbb/volumes"
Sep 09 21:08:26.129266 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:26.129234    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9"]
Sep 09 21:08:26.129657 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:26.129445    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9" podUID="4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb" containerName="istio-proxy" containerID="cri-o://4189c01731023c93ec96e6dca32e3e8710cfd22b16f3e76fc4c0505ce87ee97c" gracePeriod=30
Sep 09 21:08:26.401010 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:26.400913    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-06e1ff-x7hzp"]
Sep 09 21:08:26.401471 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:26.401451    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd985671-74d9-4116-8fa2-c754c8d19fbb" containerName="revoke-keys"
Sep 09 21:08:26.401573 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:26.401473    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd985671-74d9-4116-8fa2-c754c8d19fbb" containerName="revoke-keys"
Sep 09 21:08:26.401573 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:26.401501    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="254f0e4f-9e77-4b6c-82e4-fe37777ad5bd" containerName="payload-pre-processing"
Sep 09 21:08:26.401573 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:26.401511    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="254f0e4f-9e77-4b6c-82e4-fe37777ad5bd" containerName="payload-pre-processing"
Sep 09 21:08:26.401573 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:26.401528    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f8d283b-a4b9-4e8c-8b3d-e793c73e7f71" containerName="payload-processing"
Sep 09 21:08:26.401573 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:26.401537    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f8d283b-a4b9-4e8c-8b3d-e793c73e7f71" containerName="payload-processing"
Sep 09 21:08:26.401820 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:26.401648    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f8d283b-a4b9-4e8c-8b3d-e793c73e7f71" containerName="payload-processing"
Sep 09 21:08:26.401820 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:26.401664    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="254f0e4f-9e77-4b6c-82e4-fe37777ad5bd" containerName="payload-pre-processing"
Sep 09 21:08:26.401820 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:26.401676    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd985671-74d9-4116-8fa2-c754c8d19fbb" containerName="revoke-keys"
Sep 09 21:08:26.406491 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:26.406470    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-06e1ff-x7hzp"
Sep 09 21:08:26.416523 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:26.416308    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-glz44\""
Sep 09 21:08:26.418227 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:26.418203    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-06e1ff-x7hzp"]
Sep 09 21:08:26.517046 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:26.517012    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/042e92a3-e611-48b3-976b-0ba3c771fb4c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-06e1ff-x7hzp\" (UID: \"042e92a3-e611-48b3-976b-0ba3c771fb4c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-06e1ff-x7hzp"
Sep 09 21:08:26.617625 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:26.617587    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/042e92a3-e611-48b3-976b-0ba3c771fb4c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-06e1ff-x7hzp\" (UID: \"042e92a3-e611-48b3-976b-0ba3c771fb4c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-06e1ff-x7hzp"
Sep 09 21:08:26.618346 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:26.618309    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/042e92a3-e611-48b3-976b-0ba3c771fb4c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-06e1ff-x7hzp\" (UID: \"042e92a3-e611-48b3-976b-0ba3c771fb4c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-06e1ff-x7hzp"
Sep 09 21:08:26.717596 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:26.717509    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-06e1ff-x7hzp"
Sep 09 21:08:26.994372 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:26.990584    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-06e1ff-x7hzp"]
Sep 09 21:08:26.999181 ip-10-0-129-99 kubenswrapper[2581]: W0909 21:08:26.999147    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod042e92a3_e611_48b3_976b_0ba3c771fb4c.slice/crio-cdcd381086c84056b4017c3e104b0d60814a16101f87d59d29c805b74ee18ae5 WatchSource:0}: Error finding container cdcd381086c84056b4017c3e104b0d60814a16101f87d59d29c805b74ee18ae5: Status 404 returned error can't find the container with id cdcd381086c84056b4017c3e104b0d60814a16101f87d59d29c805b74ee18ae5
Sep 09 21:08:27.333507 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:27.333474    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-06e1ff-x7hzp" event={"ID":"042e92a3-e611-48b3-976b-0ba3c771fb4c","Type":"ContainerStarted","Data":"75412e3e2952004f4d9bb7e120ddb61a01eecd3b6dd5cda98d66d25ed476ae14"}
Sep 09 21:08:27.333866 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:27.333533    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-06e1ff-x7hzp" event={"ID":"042e92a3-e611-48b3-976b-0ba3c771fb4c","Type":"ContainerStarted","Data":"cdcd381086c84056b4017c3e104b0d60814a16101f87d59d29c805b74ee18ae5"}
Sep 09 21:08:27.355823 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:27.355770    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-06e1ff-x7hzp" podStartSLOduration=1.355754619 podStartE2EDuration="1.355754619s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:08:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:08:27.353642705 +0000 UTC m=+1668.981270577" watchObservedRunningTime="2026-09-09 21:08:27.355754619 +0000 UTC m=+1668.983382491"
Sep 09 21:08:28.340609 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:28.340573    2581 generic.go:356] "Generic (PLEG): container finished" podID="042e92a3-e611-48b3-976b-0ba3c771fb4c" containerID="75412e3e2952004f4d9bb7e120ddb61a01eecd3b6dd5cda98d66d25ed476ae14" exitCode=0
Sep 09 21:08:28.340991 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:28.340659    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-06e1ff-x7hzp" event={"ID":"042e92a3-e611-48b3-976b-0ba3c771fb4c","Type":"ContainerDied","Data":"75412e3e2952004f4d9bb7e120ddb61a01eecd3b6dd5cda98d66d25ed476ae14"}
Sep 09 21:08:29.488033 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:29.488005    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-06e1ff-x7hzp"
Sep 09 21:08:29.545906 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:29.545871    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/042e92a3-e611-48b3-976b-0ba3c771fb4c-maas-api-service-ca\") pod \"042e92a3-e611-48b3-976b-0ba3c771fb4c\" (UID: \"042e92a3-e611-48b3-976b-0ba3c771fb4c\") "
Sep 09 21:08:29.546283 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:29.546259    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/042e92a3-e611-48b3-976b-0ba3c771fb4c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "042e92a3-e611-48b3-976b-0ba3c771fb4c" (UID: "042e92a3-e611-48b3-976b-0ba3c771fb4c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:08:29.646814 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:29.646718    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/042e92a3-e611-48b3-976b-0ba3c771fb4c-maas-api-service-ca\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:08:30.352728 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:30.352701    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-06e1ff-x7hzp"
Sep 09 21:08:30.352910 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:30.352734    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-06e1ff-x7hzp" event={"ID":"042e92a3-e611-48b3-976b-0ba3c771fb4c","Type":"ContainerDied","Data":"cdcd381086c84056b4017c3e104b0d60814a16101f87d59d29c805b74ee18ae5"}
Sep 09 21:08:30.352910 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:30.352776    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cdcd381086c84056b4017c3e104b0d60814a16101f87d59d29c805b74ee18ae5"
Sep 09 21:08:30.446988 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:30.446957    2581 patch_prober.go:28] interesting pod/e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 09 21:08:30.447150 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:30.447010    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9" podUID="4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:08:31.360386 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:31.360356    2581 generic.go:356] "Generic (PLEG): container finished" podID="4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb" containerID="4189c01731023c93ec96e6dca32e3e8710cfd22b16f3e76fc4c0505ce87ee97c" exitCode=0
Sep 09 21:08:31.360759 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:31.360424    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9" event={"ID":"4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb","Type":"ContainerDied","Data":"4189c01731023c93ec96e6dca32e3e8710cfd22b16f3e76fc4c0505ce87ee97c"}
Sep 09 21:08:31.360759 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:31.360485    2581 scope.go:117] "RemoveContainer" containerID="5d436b45429bb66f9650eecb0af1270bffdc7659eb46d6c5efa5887817d51b04"
Sep 09 21:08:31.396995 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:31.396943    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9"
Sep 09 21:08:31.464998 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:31.464965    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb-workload-certs\") pod \"4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb\" (UID: \"4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb\") "
Sep 09 21:08:31.465190 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:31.465035    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb-istio-token\") pod \"4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb\" (UID: \"4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb\") "
Sep 09 21:08:31.465190 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:31.465070    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb-credential-socket\") pod \"4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb\" (UID: \"4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb\") "
Sep 09 21:08:31.465190 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:31.465117    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb-istio-envoy\") pod \"4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb\" (UID: \"4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb\") "
Sep 09 21:08:31.465190 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:31.465163    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb-workload-socket\") pod \"4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb\" (UID: \"4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb\") "
Sep 09 21:08:31.465424 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:31.465215    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb-istiod-ca-cert\") pod \"4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb\" (UID: \"4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb\") "
Sep 09 21:08:31.465424 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:31.465286    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb-istio-data\") pod \"4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb\" (UID: \"4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb\") "
Sep 09 21:08:31.465424 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:31.465310    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb-istio-podinfo\") pod \"4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb\" (UID: \"4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb\") "
Sep 09 21:08:31.465424 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:31.465312    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb" (UID: "4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:08:31.465424 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:31.465338    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wkf8k\" (UniqueName: \"kubernetes.io/projected/4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb-kube-api-access-wkf8k\") pod \"4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb\" (UID: \"4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb\") "
Sep 09 21:08:31.465674 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:31.465427    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb" (UID: "4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:08:31.465674 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:31.465498    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb" (UID: "4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:08:31.465674 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:31.465553    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb-istio-data" (OuterVolumeSpecName: "istio-data") pod "4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb" (UID: "4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:08:31.465845 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:31.465735    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb-workload-certs\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:08:31.465845 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:31.465774    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb-credential-socket\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:08:31.465845 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:31.465801    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb-workload-socket\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:08:31.465845 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:31.465813    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb-istio-data\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:08:31.466023 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:31.465921    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb" (UID: "4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:08:31.467418 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:31.467394    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb-istio-token" (OuterVolumeSpecName: "istio-token") pod "4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb" (UID: "4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:08:31.467548 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:31.467533    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb-kube-api-access-wkf8k" (OuterVolumeSpecName: "kube-api-access-wkf8k") pod "4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb" (UID: "4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb"). InnerVolumeSpecName "kube-api-access-wkf8k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:08:31.467757 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:31.467734    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb" (UID: "4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:08:31.467757 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:31.467742    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb" (UID: "4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 21:08:31.567164 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:31.567091    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb-istio-token\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:08:31.567164 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:31.567156    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb-istio-envoy\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:08:31.567164 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:31.567165    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb-istiod-ca-cert\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:08:31.567164 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:31.567173    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb-istio-podinfo\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:08:31.567436 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:31.567181    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wkf8k\" (UniqueName: \"kubernetes.io/projected/4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb-kube-api-access-wkf8k\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:08:32.367398 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:32.367366    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9"
Sep 09 21:08:32.367770 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:32.367391    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9" event={"ID":"4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb","Type":"ContainerDied","Data":"2e33616cc2a7c16dbd49c0e45b95e76dfed4a5818200673279d9afdb16e0cb36"}
Sep 09 21:08:32.367770 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:32.367473    2581 scope.go:117] "RemoveContainer" containerID="4189c01731023c93ec96e6dca32e3e8710cfd22b16f3e76fc4c0505ce87ee97c"
Sep 09 21:08:32.392405 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:32.392381    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9"]
Sep 09 21:08:32.404608 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:32.404584    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-e73aab-openshift-default-66c44f9cb8-t44b9"]
Sep 09 21:08:32.820973 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:32.820939    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb" path="/var/lib/kubelet/pods/4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb/volumes"
Sep 09 21:08:37.300227 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:37.300196    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-06e1ff-768dd8d94c-8t6lm"]
Sep 09 21:08:37.300707 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:37.300404    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-06e1ff-768dd8d94c-8t6lm" podUID="ea95b990-0e24-4d67-b761-c811612e7a78" containerName="payload-processing" containerID="cri-o://38a89293a00f29712331c3a618b6e2d1558c30b37961f6ca6f4346d096129b01" gracePeriod=30
Sep 09 21:08:37.562791 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:37.562760    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-06e1ff-x7hzp"]
Sep 09 21:08:37.569550 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:37.569526    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-06e1ff-x7hzp"]
Sep 09 21:08:37.583495 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:37.583475    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-06e1ff-768dd8d94c-8t6lm"
Sep 09 21:08:37.631126 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:37.631068    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea95b990-0e24-4d67-b761-c811612e7a78-plugins-config-volume\") pod \"ea95b990-0e24-4d67-b761-c811612e7a78\" (UID: \"ea95b990-0e24-4d67-b761-c811612e7a78\") "
Sep 09 21:08:37.631302 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:37.631161    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cwmnp\" (UniqueName: \"kubernetes.io/projected/ea95b990-0e24-4d67-b761-c811612e7a78-kube-api-access-cwmnp\") pod \"ea95b990-0e24-4d67-b761-c811612e7a78\" (UID: \"ea95b990-0e24-4d67-b761-c811612e7a78\") "
Sep 09 21:08:37.631607 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:37.631576    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea95b990-0e24-4d67-b761-c811612e7a78-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ea95b990-0e24-4d67-b761-c811612e7a78" (UID: "ea95b990-0e24-4d67-b761-c811612e7a78"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:08:37.633687 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:37.633650    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea95b990-0e24-4d67-b761-c811612e7a78-kube-api-access-cwmnp" (OuterVolumeSpecName: "kube-api-access-cwmnp") pod "ea95b990-0e24-4d67-b761-c811612e7a78" (UID: "ea95b990-0e24-4d67-b761-c811612e7a78"). InnerVolumeSpecName "kube-api-access-cwmnp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:08:37.732932 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:37.732897    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea95b990-0e24-4d67-b761-c811612e7a78-plugins-config-volume\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:08:37.732932 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:37.732926    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cwmnp\" (UniqueName: \"kubernetes.io/projected/ea95b990-0e24-4d67-b761-c811612e7a78-kube-api-access-cwmnp\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:08:38.402561 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:38.402526    2581 generic.go:356] "Generic (PLEG): container finished" podID="ea95b990-0e24-4d67-b761-c811612e7a78" containerID="38a89293a00f29712331c3a618b6e2d1558c30b37961f6ca6f4346d096129b01" exitCode=0
Sep 09 21:08:38.403000 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:38.402570    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-06e1ff-768dd8d94c-8t6lm"
Sep 09 21:08:38.403000 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:38.402622    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-06e1ff-768dd8d94c-8t6lm" event={"ID":"ea95b990-0e24-4d67-b761-c811612e7a78","Type":"ContainerDied","Data":"38a89293a00f29712331c3a618b6e2d1558c30b37961f6ca6f4346d096129b01"}
Sep 09 21:08:38.403000 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:38.402670    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-06e1ff-768dd8d94c-8t6lm" event={"ID":"ea95b990-0e24-4d67-b761-c811612e7a78","Type":"ContainerDied","Data":"a1eedc906e724554918d70db442c7e920b0819951c555cac939ab5b9a17e96f9"}
Sep 09 21:08:38.403000 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:38.402693    2581 scope.go:117] "RemoveContainer" containerID="38a89293a00f29712331c3a618b6e2d1558c30b37961f6ca6f4346d096129b01"
Sep 09 21:08:38.414757 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:38.414738    2581 scope.go:117] "RemoveContainer" containerID="38a89293a00f29712331c3a618b6e2d1558c30b37961f6ca6f4346d096129b01"
Sep 09 21:08:38.415010 ip-10-0-129-99 kubenswrapper[2581]: E0909 21:08:38.414993    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38a89293a00f29712331c3a618b6e2d1558c30b37961f6ca6f4346d096129b01\": container with ID starting with 38a89293a00f29712331c3a618b6e2d1558c30b37961f6ca6f4346d096129b01 not found: ID does not exist" containerID="38a89293a00f29712331c3a618b6e2d1558c30b37961f6ca6f4346d096129b01"
Sep 09 21:08:38.415085 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:38.415017    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38a89293a00f29712331c3a618b6e2d1558c30b37961f6ca6f4346d096129b01"} err="failed to get container status \"38a89293a00f29712331c3a618b6e2d1558c30b37961f6ca6f4346d096129b01\": rpc error: code = NotFound desc = could not find container \"38a89293a00f29712331c3a618b6e2d1558c30b37961f6ca6f4346d096129b01\": container with ID starting with 38a89293a00f29712331c3a618b6e2d1558c30b37961f6ca6f4346d096129b01 not found: ID does not exist"
Sep 09 21:08:38.443265 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:38.443219    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-06e1ff-768dd8d94c-8t6lm"]
Sep 09 21:08:38.445375 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:38.445348    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-06e1ff-768dd8d94c-8t6lm"]
Sep 09 21:08:38.820947 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:38.820918    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="042e92a3-e611-48b3-976b-0ba3c771fb4c" path="/var/lib/kubelet/pods/042e92a3-e611-48b3-976b-0ba3c771fb4c/volumes"
Sep 09 21:08:38.821346 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:38.821332    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea95b990-0e24-4d67-b761-c811612e7a78" path="/var/lib/kubelet/pods/ea95b990-0e24-4d67-b761-c811612e7a78/volumes"
Sep 09 21:08:39.195836 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:39.195751    2581 scope.go:117] "RemoveContainer" containerID="ebedddd971fc496b4f091052890d0e42645433022a85b977df0ea55668ac1de3"
Sep 09 21:08:39.204562 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:39.204543    2581 scope.go:117] "RemoveContainer" containerID="2160abc018e3bdf52afc07805dfa83c7b4ce86afcce864d39ac57234a23723fb"
Sep 09 21:08:44.800389 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:44.800355    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d"]
Sep 09 21:08:44.800885 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:44.800581    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d" podUID="0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19" containerName="istio-proxy" containerID="cri-o://846efa2d5a2296e2eff4f2973251fa588abe8b374146e456001b3f2cb1b1b8f6" gracePeriod=30
Sep 09 21:08:48.766254 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:48.766211    2581 patch_prober.go:28] interesting pod/e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 09 21:08:48.766730 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:48.766284    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d" podUID="0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:08:50.056447 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:50.056422    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d"
Sep 09 21:08:50.160658 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:50.160568    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19-istio-envoy\") pod \"0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19\" (UID: \"0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19\") "
Sep 09 21:08:50.160658 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:50.160602    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19-istio-token\") pod \"0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19\" (UID: \"0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19\") "
Sep 09 21:08:50.160658 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:50.160635    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19-workload-socket\") pod \"0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19\" (UID: \"0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19\") "
Sep 09 21:08:50.160950 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:50.160688    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19-workload-certs\") pod \"0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19\" (UID: \"0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19\") "
Sep 09 21:08:50.160950 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:50.160722    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19-istiod-ca-cert\") pod \"0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19\" (UID: \"0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19\") "
Sep 09 21:08:50.160950 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:50.160744    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19-istio-podinfo\") pod \"0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19\" (UID: \"0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19\") "
Sep 09 21:08:50.160950 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:50.160768    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lqc5g\" (UniqueName: \"kubernetes.io/projected/0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19-kube-api-access-lqc5g\") pod \"0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19\" (UID: \"0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19\") "
Sep 09 21:08:50.160950 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:50.160794    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19-credential-socket\") pod \"0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19\" (UID: \"0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19\") "
Sep 09 21:08:50.160950 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:50.160909    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19-istio-data\") pod \"0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19\" (UID: \"0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19\") "
Sep 09 21:08:50.161312 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:50.160964    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19" (UID: "0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:08:50.161312 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:50.161010    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19" (UID: "0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:08:50.161312 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:50.161205    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19-istio-data" (OuterVolumeSpecName: "istio-data") pod "0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19" (UID: "0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:08:50.161312 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:50.161231    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19" (UID: "0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:08:50.161541 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:50.161328    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19" (UID: "0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:08:50.161541 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:50.161431    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19-istio-data\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:08:50.161541 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:50.161449    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19-workload-socket\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:08:50.161541 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:50.161463    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19-workload-certs\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:08:50.161541 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:50.161475    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19-istiod-ca-cert\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:08:50.161541 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:50.161487    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19-credential-socket\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:08:50.162888 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:50.162860    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19" (UID: "0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:08:50.162888 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:50.162874    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19-istio-token" (OuterVolumeSpecName: "istio-token") pod "0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19" (UID: "0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:08:50.163020 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:50.162990    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19-kube-api-access-lqc5g" (OuterVolumeSpecName: "kube-api-access-lqc5g") pod "0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19" (UID: "0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19"). InnerVolumeSpecName "kube-api-access-lqc5g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:08:50.163078 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:50.163050    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19" (UID: "0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 21:08:50.261982 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:50.261951    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19-istio-podinfo\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:08:50.261982 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:50.261978    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lqc5g\" (UniqueName: \"kubernetes.io/projected/0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19-kube-api-access-lqc5g\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:08:50.261982 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:50.261989    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19-istio-envoy\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:08:50.262239 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:50.261999    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19-istio-token\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:08:50.467160 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:50.467061    2581 generic.go:356] "Generic (PLEG): container finished" podID="0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19" containerID="846efa2d5a2296e2eff4f2973251fa588abe8b374146e456001b3f2cb1b1b8f6" exitCode=0
Sep 09 21:08:50.467160 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:50.467135    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d"
Sep 09 21:08:50.467333 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:50.467129    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d" event={"ID":"0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19","Type":"ContainerDied","Data":"846efa2d5a2296e2eff4f2973251fa588abe8b374146e456001b3f2cb1b1b8f6"}
Sep 09 21:08:50.467333 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:50.467272    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d" event={"ID":"0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19","Type":"ContainerDied","Data":"bc3a225477954c3cfb5f7193d2ccbd02fe3be074e2273e74d2b8e56360fff661"}
Sep 09 21:08:50.467333 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:50.467295    2581 scope.go:117] "RemoveContainer" containerID="846efa2d5a2296e2eff4f2973251fa588abe8b374146e456001b3f2cb1b1b8f6"
Sep 09 21:08:50.477398 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:50.477382    2581 scope.go:117] "RemoveContainer" containerID="f216c804a79fbbecd1c0b8efb736e59ea13ff0e38cbb24279f3032ae76994df3"
Sep 09 21:08:50.485710 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:50.485692    2581 scope.go:117] "RemoveContainer" containerID="846efa2d5a2296e2eff4f2973251fa588abe8b374146e456001b3f2cb1b1b8f6"
Sep 09 21:08:50.485973 ip-10-0-129-99 kubenswrapper[2581]: E0909 21:08:50.485954    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"846efa2d5a2296e2eff4f2973251fa588abe8b374146e456001b3f2cb1b1b8f6\": container with ID starting with 846efa2d5a2296e2eff4f2973251fa588abe8b374146e456001b3f2cb1b1b8f6 not found: ID does not exist" containerID="846efa2d5a2296e2eff4f2973251fa588abe8b374146e456001b3f2cb1b1b8f6"
Sep 09 21:08:50.486034 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:50.485982    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"846efa2d5a2296e2eff4f2973251fa588abe8b374146e456001b3f2cb1b1b8f6"} err="failed to get container status \"846efa2d5a2296e2eff4f2973251fa588abe8b374146e456001b3f2cb1b1b8f6\": rpc error: code = NotFound desc = could not find container \"846efa2d5a2296e2eff4f2973251fa588abe8b374146e456001b3f2cb1b1b8f6\": container with ID starting with 846efa2d5a2296e2eff4f2973251fa588abe8b374146e456001b3f2cb1b1b8f6 not found: ID does not exist"
Sep 09 21:08:50.486034 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:50.485999    2581 scope.go:117] "RemoveContainer" containerID="f216c804a79fbbecd1c0b8efb736e59ea13ff0e38cbb24279f3032ae76994df3"
Sep 09 21:08:50.486306 ip-10-0-129-99 kubenswrapper[2581]: E0909 21:08:50.486286    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f216c804a79fbbecd1c0b8efb736e59ea13ff0e38cbb24279f3032ae76994df3\": container with ID starting with f216c804a79fbbecd1c0b8efb736e59ea13ff0e38cbb24279f3032ae76994df3 not found: ID does not exist" containerID="f216c804a79fbbecd1c0b8efb736e59ea13ff0e38cbb24279f3032ae76994df3"
Sep 09 21:08:50.486367 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:50.486315    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f216c804a79fbbecd1c0b8efb736e59ea13ff0e38cbb24279f3032ae76994df3"} err="failed to get container status \"f216c804a79fbbecd1c0b8efb736e59ea13ff0e38cbb24279f3032ae76994df3\": rpc error: code = NotFound desc = could not find container \"f216c804a79fbbecd1c0b8efb736e59ea13ff0e38cbb24279f3032ae76994df3\": container with ID starting with f216c804a79fbbecd1c0b8efb736e59ea13ff0e38cbb24279f3032ae76994df3 not found: ID does not exist"
Sep 09 21:08:50.490734 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:50.490712    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d"]
Sep 09 21:08:50.495913 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:50.495891    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-06e1ff-openshift-default-655f494d8-htk7d"]
Sep 09 21:08:50.821190 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:08:50.821157    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19" path="/var/lib/kubelet/pods/0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19/volumes"
Sep 09 21:09:06.629062 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:06.628983    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-47e41c-85m9l"]
Sep 09 21:09:06.629423 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:06.629397    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb" containerName="istio-proxy"
Sep 09 21:09:06.629423 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:06.629408    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb" containerName="istio-proxy"
Sep 09 21:09:06.629423 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:06.629425    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19" containerName="istio-proxy"
Sep 09 21:09:06.629532 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:06.629430    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19" containerName="istio-proxy"
Sep 09 21:09:06.629532 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:06.629438    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea95b990-0e24-4d67-b761-c811612e7a78" containerName="payload-processing"
Sep 09 21:09:06.629532 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:06.629443    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea95b990-0e24-4d67-b761-c811612e7a78" containerName="payload-processing"
Sep 09 21:09:06.629532 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:06.629454    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="042e92a3-e611-48b3-976b-0ba3c771fb4c" containerName="revoke-keys"
Sep 09 21:09:06.629532 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:06.629459    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="042e92a3-e611-48b3-976b-0ba3c771fb4c" containerName="revoke-keys"
Sep 09 21:09:06.629532 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:06.629466    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19" containerName="istio-proxy"
Sep 09 21:09:06.629532 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:06.629471    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19" containerName="istio-proxy"
Sep 09 21:09:06.629532 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:06.629479    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb" containerName="istio-proxy"
Sep 09 21:09:06.629532 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:06.629484    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb" containerName="istio-proxy"
Sep 09 21:09:06.629805 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:06.629540    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19" containerName="istio-proxy"
Sep 09 21:09:06.629805 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:06.629549    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb" containerName="istio-proxy"
Sep 09 21:09:06.629805 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:06.629555    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea95b990-0e24-4d67-b761-c811612e7a78" containerName="payload-processing"
Sep 09 21:09:06.629805 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:06.629562    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="0110caa6-0efd-4f2e-8ac4-ca1c09b0cc19" containerName="istio-proxy"
Sep 09 21:09:06.629805 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:06.629570    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="042e92a3-e611-48b3-976b-0ba3c771fb4c" containerName="revoke-keys"
Sep 09 21:09:06.634083 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:06.634058    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-47e41c-85m9l"
Sep 09 21:09:06.638050 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:06.638025    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-glz44\""
Sep 09 21:09:06.647086 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:06.647060    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-47e41c-85m9l"]
Sep 09 21:09:06.731707 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:06.731665    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/211bfc10-b3b0-41ae-baad-d25586d102e5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-47e41c-85m9l\" (UID: \"211bfc10-b3b0-41ae-baad-d25586d102e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-47e41c-85m9l"
Sep 09 21:09:06.833183 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:06.833152    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/211bfc10-b3b0-41ae-baad-d25586d102e5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-47e41c-85m9l\" (UID: \"211bfc10-b3b0-41ae-baad-d25586d102e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-47e41c-85m9l"
Sep 09 21:09:06.833752 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:06.833730    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/211bfc10-b3b0-41ae-baad-d25586d102e5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-47e41c-85m9l\" (UID: \"211bfc10-b3b0-41ae-baad-d25586d102e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-47e41c-85m9l"
Sep 09 21:09:06.945500 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:06.945400    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-47e41c-85m9l"
Sep 09 21:09:07.225038 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:07.224864    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-47e41c-85m9l"]
Sep 09 21:09:07.227442 ip-10-0-129-99 kubenswrapper[2581]: W0909 21:09:07.227414    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod211bfc10_b3b0_41ae_baad_d25586d102e5.slice/crio-547d4ded1691479864c3a39372483ca358835034bfbe447817ce33d0edd63354 WatchSource:0}: Error finding container 547d4ded1691479864c3a39372483ca358835034bfbe447817ce33d0edd63354: Status 404 returned error can't find the container with id 547d4ded1691479864c3a39372483ca358835034bfbe447817ce33d0edd63354
Sep 09 21:09:07.566338 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:07.566303    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-47e41c-85m9l" event={"ID":"211bfc10-b3b0-41ae-baad-d25586d102e5","Type":"ContainerStarted","Data":"1cb1f6c6bd56f9927ca9865c000a44b4770cfb6e3c73c493d3ff557a8e6d7c70"}
Sep 09 21:09:07.566474 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:07.566381    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-47e41c-85m9l" event={"ID":"211bfc10-b3b0-41ae-baad-d25586d102e5","Type":"ContainerStarted","Data":"547d4ded1691479864c3a39372483ca358835034bfbe447817ce33d0edd63354"}
Sep 09 21:09:08.573372 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:08.573335    2581 generic.go:356] "Generic (PLEG): container finished" podID="211bfc10-b3b0-41ae-baad-d25586d102e5" containerID="1cb1f6c6bd56f9927ca9865c000a44b4770cfb6e3c73c493d3ff557a8e6d7c70" exitCode=0
Sep 09 21:09:08.573806 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:08.573424    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-47e41c-85m9l" event={"ID":"211bfc10-b3b0-41ae-baad-d25586d102e5","Type":"ContainerDied","Data":"1cb1f6c6bd56f9927ca9865c000a44b4770cfb6e3c73c493d3ff557a8e6d7c70"}
Sep 09 21:09:09.716687 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:09.716654    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-47e41c-85m9l"
Sep 09 21:09:09.862599 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:09.862508    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/211bfc10-b3b0-41ae-baad-d25586d102e5-maas-api-service-ca\") pod \"211bfc10-b3b0-41ae-baad-d25586d102e5\" (UID: \"211bfc10-b3b0-41ae-baad-d25586d102e5\") "
Sep 09 21:09:09.862831 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:09.862808    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/211bfc10-b3b0-41ae-baad-d25586d102e5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "211bfc10-b3b0-41ae-baad-d25586d102e5" (UID: "211bfc10-b3b0-41ae-baad-d25586d102e5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:09:09.963579 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:09.963543    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/211bfc10-b3b0-41ae-baad-d25586d102e5-maas-api-service-ca\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:09:10.586623 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:10.586586    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-47e41c-85m9l" event={"ID":"211bfc10-b3b0-41ae-baad-d25586d102e5","Type":"ContainerDied","Data":"547d4ded1691479864c3a39372483ca358835034bfbe447817ce33d0edd63354"}
Sep 09 21:09:10.586804 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:10.586648    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="547d4ded1691479864c3a39372483ca358835034bfbe447817ce33d0edd63354"
Sep 09 21:09:10.586804 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:10.586603    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-47e41c-85m9l"
Sep 09 21:09:11.491801 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:11.491769    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-47e41c/distinct-2-w4-47e41c-kserve-67c87c7696-slxnj"]
Sep 09 21:09:11.522596 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:11.522569    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-47e41c/premium-w4-47e41c-kserve-6b47f4769b-lrvrt"]
Sep 09 21:09:22.578193 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:22.578155    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-47e41c-29816460-95t86"]
Sep 09 21:09:22.595330 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:22.595293    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-47e41c-29816460-95t86"]
Sep 09 21:09:22.820970 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:22.820934    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c45a510c-8af8-45c9-8499-c11b11ae4bb2" path="/var/lib/kubelet/pods/c45a510c-8af8-45c9-8499-c11b11ae4bb2/volumes"
Sep 09 21:09:22.896443 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:22.896357    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-47e41c-85m9l"]
Sep 09 21:09:22.899052 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:22.899028    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-47e41c-85m9l"]
Sep 09 21:09:24.822555 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:24.822521    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="211bfc10-b3b0-41ae-baad-d25586d102e5" path="/var/lib/kubelet/pods/211bfc10-b3b0-41ae-baad-d25586d102e5/volumes"
Sep 09 21:09:26.511498 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:26.511458    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-47e41c/distinct-2-w4-47e41c-kserve-67c87c7696-slxnj" podUID="6500905e-d7dd-4d93-bfd9-c92b320c2817" containerName="main" containerID="cri-o://63dad889e0e472d47c31644a35057577053bbbedb89a8601b152f1de881ad359" gracePeriod=45
Sep 09 21:09:26.541613 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:26.541583    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-47e41c/premium-w4-47e41c-kserve-6b47f4769b-lrvrt" podUID="6a2c1169-5aab-447b-90ff-1fbc19876d31" containerName="main" containerID="cri-o://3e2e5f33f279ef8a60d96a8ddfb159d17f0b0a0abc20889262fd84e2150efd54" gracePeriod=45
Sep 09 21:09:26.681204 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:26.681172    2581 generic.go:356] "Generic (PLEG): container finished" podID="6500905e-d7dd-4d93-bfd9-c92b320c2817" containerID="63dad889e0e472d47c31644a35057577053bbbedb89a8601b152f1de881ad359" exitCode=0
Sep 09 21:09:26.681372 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:26.681249    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-47e41c/distinct-2-w4-47e41c-kserve-67c87c7696-slxnj" event={"ID":"6500905e-d7dd-4d93-bfd9-c92b320c2817","Type":"ContainerDied","Data":"63dad889e0e472d47c31644a35057577053bbbedb89a8601b152f1de881ad359"}
Sep 09 21:09:26.683180 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:26.683156    2581 generic.go:356] "Generic (PLEG): container finished" podID="6a2c1169-5aab-447b-90ff-1fbc19876d31" containerID="3e2e5f33f279ef8a60d96a8ddfb159d17f0b0a0abc20889262fd84e2150efd54" exitCode=0
Sep 09 21:09:26.683276 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:26.683239    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-47e41c/premium-w4-47e41c-kserve-6b47f4769b-lrvrt" event={"ID":"6a2c1169-5aab-447b-90ff-1fbc19876d31","Type":"ContainerDied","Data":"3e2e5f33f279ef8a60d96a8ddfb159d17f0b0a0abc20889262fd84e2150efd54"}
Sep 09 21:09:26.814302 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:26.814278    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-47e41c/distinct-2-w4-47e41c-kserve-67c87c7696-slxnj"
Sep 09 21:09:26.817481 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:26.817458    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-47e41c/premium-w4-47e41c-kserve-6b47f4769b-lrvrt"
Sep 09 21:09:26.933838 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:26.933805    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6500905e-d7dd-4d93-bfd9-c92b320c2817-tls-certs\") pod \"6500905e-d7dd-4d93-bfd9-c92b320c2817\" (UID: \"6500905e-d7dd-4d93-bfd9-c92b320c2817\") "
Sep 09 21:09:26.933838 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:26.933840    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6500905e-d7dd-4d93-bfd9-c92b320c2817-model-cache\") pod \"6500905e-d7dd-4d93-bfd9-c92b320c2817\" (UID: \"6500905e-d7dd-4d93-bfd9-c92b320c2817\") "
Sep 09 21:09:26.934085 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:26.933868    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6500905e-d7dd-4d93-bfd9-c92b320c2817-dshm\") pod \"6500905e-d7dd-4d93-bfd9-c92b320c2817\" (UID: \"6500905e-d7dd-4d93-bfd9-c92b320c2817\") "
Sep 09 21:09:26.934085 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:26.933902    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7hqc2\" (UniqueName: \"kubernetes.io/projected/6a2c1169-5aab-447b-90ff-1fbc19876d31-kube-api-access-7hqc2\") pod \"6a2c1169-5aab-447b-90ff-1fbc19876d31\" (UID: \"6a2c1169-5aab-447b-90ff-1fbc19876d31\") "
Sep 09 21:09:26.934085 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:26.933945    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-drwd9\" (UniqueName: \"kubernetes.io/projected/6500905e-d7dd-4d93-bfd9-c92b320c2817-kube-api-access-drwd9\") pod \"6500905e-d7dd-4d93-bfd9-c92b320c2817\" (UID: \"6500905e-d7dd-4d93-bfd9-c92b320c2817\") "
Sep 09 21:09:26.934085 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:26.933981    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6500905e-d7dd-4d93-bfd9-c92b320c2817-home\") pod \"6500905e-d7dd-4d93-bfd9-c92b320c2817\" (UID: \"6500905e-d7dd-4d93-bfd9-c92b320c2817\") "
Sep 09 21:09:26.934085 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:26.934039    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a2c1169-5aab-447b-90ff-1fbc19876d31-tls-certs\") pod \"6a2c1169-5aab-447b-90ff-1fbc19876d31\" (UID: \"6a2c1169-5aab-447b-90ff-1fbc19876d31\") "
Sep 09 21:09:26.934085 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:26.934076    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6a2c1169-5aab-447b-90ff-1fbc19876d31-model-cache\") pod \"6a2c1169-5aab-447b-90ff-1fbc19876d31\" (UID: \"6a2c1169-5aab-447b-90ff-1fbc19876d31\") "
Sep 09 21:09:26.934429 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:26.934166    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6a2c1169-5aab-447b-90ff-1fbc19876d31-dshm\") pod \"6a2c1169-5aab-447b-90ff-1fbc19876d31\" (UID: \"6a2c1169-5aab-447b-90ff-1fbc19876d31\") "
Sep 09 21:09:26.934429 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:26.934206    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a2c1169-5aab-447b-90ff-1fbc19876d31-tmp-dir\") pod \"6a2c1169-5aab-447b-90ff-1fbc19876d31\" (UID: \"6a2c1169-5aab-447b-90ff-1fbc19876d31\") "
Sep 09 21:09:26.934429 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:26.934237    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6a2c1169-5aab-447b-90ff-1fbc19876d31-home\") pod \"6a2c1169-5aab-447b-90ff-1fbc19876d31\" (UID: \"6a2c1169-5aab-447b-90ff-1fbc19876d31\") "
Sep 09 21:09:26.934429 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:26.934287    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6500905e-d7dd-4d93-bfd9-c92b320c2817-tmp-dir\") pod \"6500905e-d7dd-4d93-bfd9-c92b320c2817\" (UID: \"6500905e-d7dd-4d93-bfd9-c92b320c2817\") "
Sep 09 21:09:26.934964 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:26.934162    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6500905e-d7dd-4d93-bfd9-c92b320c2817-model-cache" (OuterVolumeSpecName: "model-cache") pod "6500905e-d7dd-4d93-bfd9-c92b320c2817" (UID: "6500905e-d7dd-4d93-bfd9-c92b320c2817"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:09:26.935508 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:26.935135    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a2c1169-5aab-447b-90ff-1fbc19876d31-model-cache" (OuterVolumeSpecName: "model-cache") pod "6a2c1169-5aab-447b-90ff-1fbc19876d31" (UID: "6a2c1169-5aab-447b-90ff-1fbc19876d31"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:09:26.935508 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:26.934355    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6500905e-d7dd-4d93-bfd9-c92b320c2817-home" (OuterVolumeSpecName: "home") pod "6500905e-d7dd-4d93-bfd9-c92b320c2817" (UID: "6500905e-d7dd-4d93-bfd9-c92b320c2817"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:09:26.935508 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:26.934938    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6500905e-d7dd-4d93-bfd9-c92b320c2817-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6500905e-d7dd-4d93-bfd9-c92b320c2817" (UID: "6500905e-d7dd-4d93-bfd9-c92b320c2817"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:09:26.935508 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:26.935339    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a2c1169-5aab-447b-90ff-1fbc19876d31-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6a2c1169-5aab-447b-90ff-1fbc19876d31" (UID: "6a2c1169-5aab-447b-90ff-1fbc19876d31"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:09:26.935508 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:26.935476    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a2c1169-5aab-447b-90ff-1fbc19876d31-home" (OuterVolumeSpecName: "home") pod "6a2c1169-5aab-447b-90ff-1fbc19876d31" (UID: "6a2c1169-5aab-447b-90ff-1fbc19876d31"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:09:26.936697 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:26.936671    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6500905e-d7dd-4d93-bfd9-c92b320c2817-dshm" (OuterVolumeSpecName: "dshm") pod "6500905e-d7dd-4d93-bfd9-c92b320c2817" (UID: "6500905e-d7dd-4d93-bfd9-c92b320c2817"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:09:26.937138 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:26.937078    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a2c1169-5aab-447b-90ff-1fbc19876d31-dshm" (OuterVolumeSpecName: "dshm") pod "6a2c1169-5aab-447b-90ff-1fbc19876d31" (UID: "6a2c1169-5aab-447b-90ff-1fbc19876d31"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:09:26.937237 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:26.937143    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a2c1169-5aab-447b-90ff-1fbc19876d31-kube-api-access-7hqc2" (OuterVolumeSpecName: "kube-api-access-7hqc2") pod "6a2c1169-5aab-447b-90ff-1fbc19876d31" (UID: "6a2c1169-5aab-447b-90ff-1fbc19876d31"). InnerVolumeSpecName "kube-api-access-7hqc2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:09:26.937386 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:26.937360    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6500905e-d7dd-4d93-bfd9-c92b320c2817-kube-api-access-drwd9" (OuterVolumeSpecName: "kube-api-access-drwd9") pod "6500905e-d7dd-4d93-bfd9-c92b320c2817" (UID: "6500905e-d7dd-4d93-bfd9-c92b320c2817"). InnerVolumeSpecName "kube-api-access-drwd9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:09:26.937500 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:26.937392    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6500905e-d7dd-4d93-bfd9-c92b320c2817-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6500905e-d7dd-4d93-bfd9-c92b320c2817" (UID: "6500905e-d7dd-4d93-bfd9-c92b320c2817"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 21:09:26.937844 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:26.937829    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a2c1169-5aab-447b-90ff-1fbc19876d31-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6a2c1169-5aab-447b-90ff-1fbc19876d31" (UID: "6a2c1169-5aab-447b-90ff-1fbc19876d31"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 21:09:27.035740 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:27.035695    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6a2c1169-5aab-447b-90ff-1fbc19876d31-model-cache\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:09:27.035740 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:27.035741    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6a2c1169-5aab-447b-90ff-1fbc19876d31-dshm\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:09:27.035740 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:27.035758    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a2c1169-5aab-447b-90ff-1fbc19876d31-tmp-dir\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:09:27.035978 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:27.035766    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6a2c1169-5aab-447b-90ff-1fbc19876d31-home\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:09:27.035978 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:27.035776    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6500905e-d7dd-4d93-bfd9-c92b320c2817-tmp-dir\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:09:27.035978 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:27.035783    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6500905e-d7dd-4d93-bfd9-c92b320c2817-tls-certs\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:09:27.035978 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:27.035791    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6500905e-d7dd-4d93-bfd9-c92b320c2817-model-cache\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:09:27.035978 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:27.035800    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6500905e-d7dd-4d93-bfd9-c92b320c2817-dshm\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:09:27.035978 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:27.035807    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7hqc2\" (UniqueName: \"kubernetes.io/projected/6a2c1169-5aab-447b-90ff-1fbc19876d31-kube-api-access-7hqc2\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:09:27.035978 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:27.035816    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-drwd9\" (UniqueName: \"kubernetes.io/projected/6500905e-d7dd-4d93-bfd9-c92b320c2817-kube-api-access-drwd9\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:09:27.035978 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:27.035824    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6500905e-d7dd-4d93-bfd9-c92b320c2817-home\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:09:27.035978 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:27.035831    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a2c1169-5aab-447b-90ff-1fbc19876d31-tls-certs\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:09:27.689251 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:27.689220    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-47e41c/distinct-2-w4-47e41c-kserve-67c87c7696-slxnj"
Sep 09 21:09:27.689251 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:27.689241    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-47e41c/distinct-2-w4-47e41c-kserve-67c87c7696-slxnj" event={"ID":"6500905e-d7dd-4d93-bfd9-c92b320c2817","Type":"ContainerDied","Data":"059f6562a2efad9a8532ac78e9beb2d5533c9df1c23d353e007f0117d17301f7"}
Sep 09 21:09:27.689760 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:27.689297    2581 scope.go:117] "RemoveContainer" containerID="63dad889e0e472d47c31644a35057577053bbbedb89a8601b152f1de881ad359"
Sep 09 21:09:27.690735 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:27.690709    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-47e41c/premium-w4-47e41c-kserve-6b47f4769b-lrvrt" event={"ID":"6a2c1169-5aab-447b-90ff-1fbc19876d31","Type":"ContainerDied","Data":"ffcc97bfa6b8fdf8d4332ce7c2e83b0e168ee7f8fa29f3bd5229b1bbf0c80c5c"}
Sep 09 21:09:27.690838 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:27.690736    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-47e41c/premium-w4-47e41c-kserve-6b47f4769b-lrvrt"
Sep 09 21:09:27.698490 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:27.698467    2581 scope.go:117] "RemoveContainer" containerID="3e2e5f33f279ef8a60d96a8ddfb159d17f0b0a0abc20889262fd84e2150efd54"
Sep 09 21:09:27.719184 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:27.719162    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-47e41c/distinct-2-w4-47e41c-kserve-67c87c7696-slxnj"]
Sep 09 21:09:27.722631 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:27.722613    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-47e41c/distinct-2-w4-47e41c-kserve-67c87c7696-slxnj"]
Sep 09 21:09:27.740699 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:27.740679    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-47e41c/premium-w4-47e41c-kserve-6b47f4769b-lrvrt"]
Sep 09 21:09:27.744050 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:27.744030    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-47e41c/premium-w4-47e41c-kserve-6b47f4769b-lrvrt"]
Sep 09 21:09:28.822035 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:28.822004    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6500905e-d7dd-4d93-bfd9-c92b320c2817" path="/var/lib/kubelet/pods/6500905e-d7dd-4d93-bfd9-c92b320c2817/volumes"
Sep 09 21:09:28.822481 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:28.822465    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a2c1169-5aab-447b-90ff-1fbc19876d31" path="/var/lib/kubelet/pods/6a2c1169-5aab-447b-90ff-1fbc19876d31/volumes"
Sep 09 21:09:30.240187 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:30.240157    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk"]
Sep 09 21:09:30.240614 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:30.240381    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk" podUID="07f6f4a7-5d41-4db5-9c4a-eceaf6e96903" containerName="istio-proxy" containerID="cri-o://6abe8eb1096f66f61cc06337e07f88b9740936d05351dc7f6b9d06a568fc7008" gracePeriod=30
Sep 09 21:09:35.488688 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:35.488664    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk"
Sep 09 21:09:35.516227 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:35.516159    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bkcvr\" (UniqueName: \"kubernetes.io/projected/07f6f4a7-5d41-4db5-9c4a-eceaf6e96903-kube-api-access-bkcvr\") pod \"07f6f4a7-5d41-4db5-9c4a-eceaf6e96903\" (UID: \"07f6f4a7-5d41-4db5-9c4a-eceaf6e96903\") "
Sep 09 21:09:35.516227 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:35.516218    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07f6f4a7-5d41-4db5-9c4a-eceaf6e96903-istio-token\") pod \"07f6f4a7-5d41-4db5-9c4a-eceaf6e96903\" (UID: \"07f6f4a7-5d41-4db5-9c4a-eceaf6e96903\") "
Sep 09 21:09:35.516411 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:35.516267    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07f6f4a7-5d41-4db5-9c4a-eceaf6e96903-workload-certs\") pod \"07f6f4a7-5d41-4db5-9c4a-eceaf6e96903\" (UID: \"07f6f4a7-5d41-4db5-9c4a-eceaf6e96903\") "
Sep 09 21:09:35.516411 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:35.516335    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07f6f4a7-5d41-4db5-9c4a-eceaf6e96903-istio-data\") pod \"07f6f4a7-5d41-4db5-9c4a-eceaf6e96903\" (UID: \"07f6f4a7-5d41-4db5-9c4a-eceaf6e96903\") "
Sep 09 21:09:35.516411 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:35.516378    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07f6f4a7-5d41-4db5-9c4a-eceaf6e96903-workload-socket\") pod \"07f6f4a7-5d41-4db5-9c4a-eceaf6e96903\" (UID: \"07f6f4a7-5d41-4db5-9c4a-eceaf6e96903\") "
Sep 09 21:09:35.516545 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:35.516477    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07f6f4a7-5d41-4db5-9c4a-eceaf6e96903-istiod-ca-cert\") pod \"07f6f4a7-5d41-4db5-9c4a-eceaf6e96903\" (UID: \"07f6f4a7-5d41-4db5-9c4a-eceaf6e96903\") "
Sep 09 21:09:35.516545 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:35.516523    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07f6f4a7-5d41-4db5-9c4a-eceaf6e96903-credential-socket\") pod \"07f6f4a7-5d41-4db5-9c4a-eceaf6e96903\" (UID: \"07f6f4a7-5d41-4db5-9c4a-eceaf6e96903\") "
Sep 09 21:09:35.516633 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:35.516550    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07f6f4a7-5d41-4db5-9c4a-eceaf6e96903-istio-envoy\") pod \"07f6f4a7-5d41-4db5-9c4a-eceaf6e96903\" (UID: \"07f6f4a7-5d41-4db5-9c4a-eceaf6e96903\") "
Sep 09 21:09:35.516633 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:35.516577    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07f6f4a7-5d41-4db5-9c4a-eceaf6e96903-istio-podinfo\") pod \"07f6f4a7-5d41-4db5-9c4a-eceaf6e96903\" (UID: \"07f6f4a7-5d41-4db5-9c4a-eceaf6e96903\") "
Sep 09 21:09:35.516772 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:35.516726    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07f6f4a7-5d41-4db5-9c4a-eceaf6e96903-istio-data" (OuterVolumeSpecName: "istio-data") pod "07f6f4a7-5d41-4db5-9c4a-eceaf6e96903" (UID: "07f6f4a7-5d41-4db5-9c4a-eceaf6e96903"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:09:35.517079 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:35.516943    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07f6f4a7-5d41-4db5-9c4a-eceaf6e96903-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "07f6f4a7-5d41-4db5-9c4a-eceaf6e96903" (UID: "07f6f4a7-5d41-4db5-9c4a-eceaf6e96903"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:09:35.517079 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:35.516978    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07f6f4a7-5d41-4db5-9c4a-eceaf6e96903-istio-data\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:09:35.517369 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:35.517299    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07f6f4a7-5d41-4db5-9c4a-eceaf6e96903-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "07f6f4a7-5d41-4db5-9c4a-eceaf6e96903" (UID: "07f6f4a7-5d41-4db5-9c4a-eceaf6e96903"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:09:35.517512 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:35.517487    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07f6f4a7-5d41-4db5-9c4a-eceaf6e96903-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "07f6f4a7-5d41-4db5-9c4a-eceaf6e96903" (UID: "07f6f4a7-5d41-4db5-9c4a-eceaf6e96903"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:09:35.517589 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:35.517512    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07f6f4a7-5d41-4db5-9c4a-eceaf6e96903-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "07f6f4a7-5d41-4db5-9c4a-eceaf6e96903" (UID: "07f6f4a7-5d41-4db5-9c4a-eceaf6e96903"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:09:35.519006 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:35.518964    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07f6f4a7-5d41-4db5-9c4a-eceaf6e96903-istio-token" (OuterVolumeSpecName: "istio-token") pod "07f6f4a7-5d41-4db5-9c4a-eceaf6e96903" (UID: "07f6f4a7-5d41-4db5-9c4a-eceaf6e96903"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:09:35.519229 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:35.519200    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/07f6f4a7-5d41-4db5-9c4a-eceaf6e96903-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "07f6f4a7-5d41-4db5-9c4a-eceaf6e96903" (UID: "07f6f4a7-5d41-4db5-9c4a-eceaf6e96903"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 21:09:35.519370 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:35.519345    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07f6f4a7-5d41-4db5-9c4a-eceaf6e96903-kube-api-access-bkcvr" (OuterVolumeSpecName: "kube-api-access-bkcvr") pod "07f6f4a7-5d41-4db5-9c4a-eceaf6e96903" (UID: "07f6f4a7-5d41-4db5-9c4a-eceaf6e96903"). InnerVolumeSpecName "kube-api-access-bkcvr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:09:35.519541 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:35.519513    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07f6f4a7-5d41-4db5-9c4a-eceaf6e96903-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "07f6f4a7-5d41-4db5-9c4a-eceaf6e96903" (UID: "07f6f4a7-5d41-4db5-9c4a-eceaf6e96903"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:09:35.617898 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:35.617862    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07f6f4a7-5d41-4db5-9c4a-eceaf6e96903-istiod-ca-cert\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:09:35.617898 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:35.617886    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07f6f4a7-5d41-4db5-9c4a-eceaf6e96903-credential-socket\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:09:35.617898 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:35.617897    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07f6f4a7-5d41-4db5-9c4a-eceaf6e96903-istio-envoy\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:09:35.617898 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:35.617906    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07f6f4a7-5d41-4db5-9c4a-eceaf6e96903-istio-podinfo\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:09:35.618204 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:35.617914    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bkcvr\" (UniqueName: \"kubernetes.io/projected/07f6f4a7-5d41-4db5-9c4a-eceaf6e96903-kube-api-access-bkcvr\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:09:35.618204 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:35.617924    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07f6f4a7-5d41-4db5-9c4a-eceaf6e96903-istio-token\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:09:35.618204 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:35.617932    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07f6f4a7-5d41-4db5-9c4a-eceaf6e96903-workload-certs\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:09:35.618204 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:35.617939    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07f6f4a7-5d41-4db5-9c4a-eceaf6e96903-workload-socket\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:09:35.740087 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:35.740048    2581 generic.go:356] "Generic (PLEG): container finished" podID="07f6f4a7-5d41-4db5-9c4a-eceaf6e96903" containerID="6abe8eb1096f66f61cc06337e07f88b9740936d05351dc7f6b9d06a568fc7008" exitCode=0
Sep 09 21:09:35.740087 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:35.740094    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk"
Sep 09 21:09:35.740335 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:35.740132    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk" event={"ID":"07f6f4a7-5d41-4db5-9c4a-eceaf6e96903","Type":"ContainerDied","Data":"6abe8eb1096f66f61cc06337e07f88b9740936d05351dc7f6b9d06a568fc7008"}
Sep 09 21:09:35.740335 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:35.740183    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk" event={"ID":"07f6f4a7-5d41-4db5-9c4a-eceaf6e96903","Type":"ContainerDied","Data":"9a03ad214b0007b089a670bb18ea5b8f3551336e6bc46388de39014779ab95ed"}
Sep 09 21:09:35.740335 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:35.740204    2581 scope.go:117] "RemoveContainer" containerID="6abe8eb1096f66f61cc06337e07f88b9740936d05351dc7f6b9d06a568fc7008"
Sep 09 21:09:35.749915 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:35.749900    2581 scope.go:117] "RemoveContainer" containerID="3387a184712e50dcad798672842de44a6fef191ddf2ece416e7abcb1cc6d7cc7"
Sep 09 21:09:35.758382 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:35.758362    2581 scope.go:117] "RemoveContainer" containerID="6abe8eb1096f66f61cc06337e07f88b9740936d05351dc7f6b9d06a568fc7008"
Sep 09 21:09:35.758633 ip-10-0-129-99 kubenswrapper[2581]: E0909 21:09:35.758614    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6abe8eb1096f66f61cc06337e07f88b9740936d05351dc7f6b9d06a568fc7008\": container with ID starting with 6abe8eb1096f66f61cc06337e07f88b9740936d05351dc7f6b9d06a568fc7008 not found: ID does not exist" containerID="6abe8eb1096f66f61cc06337e07f88b9740936d05351dc7f6b9d06a568fc7008"
Sep 09 21:09:35.758677 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:35.758643    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6abe8eb1096f66f61cc06337e07f88b9740936d05351dc7f6b9d06a568fc7008"} err="failed to get container status \"6abe8eb1096f66f61cc06337e07f88b9740936d05351dc7f6b9d06a568fc7008\": rpc error: code = NotFound desc = could not find container \"6abe8eb1096f66f61cc06337e07f88b9740936d05351dc7f6b9d06a568fc7008\": container with ID starting with 6abe8eb1096f66f61cc06337e07f88b9740936d05351dc7f6b9d06a568fc7008 not found: ID does not exist"
Sep 09 21:09:35.758677 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:35.758659    2581 scope.go:117] "RemoveContainer" containerID="3387a184712e50dcad798672842de44a6fef191ddf2ece416e7abcb1cc6d7cc7"
Sep 09 21:09:35.758902 ip-10-0-129-99 kubenswrapper[2581]: E0909 21:09:35.758885    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3387a184712e50dcad798672842de44a6fef191ddf2ece416e7abcb1cc6d7cc7\": container with ID starting with 3387a184712e50dcad798672842de44a6fef191ddf2ece416e7abcb1cc6d7cc7 not found: ID does not exist" containerID="3387a184712e50dcad798672842de44a6fef191ddf2ece416e7abcb1cc6d7cc7"
Sep 09 21:09:35.758947 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:35.758907    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3387a184712e50dcad798672842de44a6fef191ddf2ece416e7abcb1cc6d7cc7"} err="failed to get container status \"3387a184712e50dcad798672842de44a6fef191ddf2ece416e7abcb1cc6d7cc7\": rpc error: code = NotFound desc = could not find container \"3387a184712e50dcad798672842de44a6fef191ddf2ece416e7abcb1cc6d7cc7\": container with ID starting with 3387a184712e50dcad798672842de44a6fef191ddf2ece416e7abcb1cc6d7cc7 not found: ID does not exist"
Sep 09 21:09:35.766404 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:35.766342    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk"]
Sep 09 21:09:35.769902 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:35.769870    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-47e41c-openshift-default-7b648cdf56-g2ffk"]
Sep 09 21:09:36.820220 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:36.820187    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07f6f4a7-5d41-4db5-9c4a-eceaf6e96903" path="/var/lib/kubelet/pods/07f6f4a7-5d41-4db5-9c4a-eceaf6e96903/volumes"
Sep 09 21:09:39.248446 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:09:39.248418    2581 scope.go:117] "RemoveContainer" containerID="09d4c34209a386a6be05b6baea5cba452bd1ddcfb1ecb4bd9b40f65336955d16"
Sep 09 21:10:38.948609 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:10:38.948580    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-wsv4v_2b1f9a6e-7a79-4e89-ac3f-4658d6d91550/console-operator/1.log"
Sep 09 21:10:38.952343 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:10:38.952321    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-wsv4v_2b1f9a6e-7a79-4e89-ac3f-4658d6d91550/console-operator/1.log"
Sep 09 21:10:38.954970 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:10:38.954950    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-75wj4_55b4fbae-29dc-41bf-a907-b1412b852cef/ovn-acl-logging/0.log"
Sep 09 21:10:38.958632 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:10:38.958614    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-75wj4_55b4fbae-29dc-41bf-a907-b1412b852cef/ovn-acl-logging/0.log"
Sep 09 21:10:39.284228 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:10:39.284195    2581 scope.go:117] "RemoveContainer" containerID="394138d2723d9325f201d18fce2abfa53b78289b2ad76d61a908a9b9c3d8ce4f"
Sep 09 21:12:20.447858 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:20.447826    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr"]
Sep 09 21:12:20.450395 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:20.448331    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a2c1169-5aab-447b-90ff-1fbc19876d31" containerName="main"
Sep 09 21:12:20.450395 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:20.448347    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a2c1169-5aab-447b-90ff-1fbc19876d31" containerName="main"
Sep 09 21:12:20.450395 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:20.448357    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="211bfc10-b3b0-41ae-baad-d25586d102e5" containerName="revoke-keys"
Sep 09 21:12:20.450395 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:20.448364    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="211bfc10-b3b0-41ae-baad-d25586d102e5" containerName="revoke-keys"
Sep 09 21:12:20.450395 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:20.448376    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07f6f4a7-5d41-4db5-9c4a-eceaf6e96903" containerName="istio-proxy"
Sep 09 21:12:20.450395 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:20.448381    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="07f6f4a7-5d41-4db5-9c4a-eceaf6e96903" containerName="istio-proxy"
Sep 09 21:12:20.450395 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:20.448388    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07f6f4a7-5d41-4db5-9c4a-eceaf6e96903" containerName="istio-proxy"
Sep 09 21:12:20.450395 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:20.448392    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="07f6f4a7-5d41-4db5-9c4a-eceaf6e96903" containerName="istio-proxy"
Sep 09 21:12:20.450395 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:20.448408    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6500905e-d7dd-4d93-bfd9-c92b320c2817" containerName="main"
Sep 09 21:12:20.450395 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:20.448413    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="6500905e-d7dd-4d93-bfd9-c92b320c2817" containerName="main"
Sep 09 21:12:20.450395 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:20.448466    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a2c1169-5aab-447b-90ff-1fbc19876d31" containerName="main"
Sep 09 21:12:20.450395 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:20.448473    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="07f6f4a7-5d41-4db5-9c4a-eceaf6e96903" containerName="istio-proxy"
Sep 09 21:12:20.450395 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:20.448478    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="6500905e-d7dd-4d93-bfd9-c92b320c2817" containerName="main"
Sep 09 21:12:20.450395 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:20.448485    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="4dc50419-8643-4bd0-8c7a-c3bd6c4d91eb" containerName="istio-proxy"
Sep 09 21:12:20.450395 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:20.448492    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="211bfc10-b3b0-41ae-baad-d25586d102e5" containerName="revoke-keys"
Sep 09 21:12:20.451493 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:20.451474    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr"
Sep 09 21:12:20.455987 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:20.455967    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-7c0011-openshift-default-dockercfg-7nb87\""
Sep 09 21:12:20.477283 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:20.477252    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr"]
Sep 09 21:12:20.570161 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:20.570117    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0-workload-certs\") pod \"e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr\" (UID: \"eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0\") " pod="openshift-ingress/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr"
Sep 09 21:12:20.570332 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:20.570174    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25jvz\" (UniqueName: \"kubernetes.io/projected/eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0-kube-api-access-25jvz\") pod \"e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr\" (UID: \"eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0\") " pod="openshift-ingress/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr"
Sep 09 21:12:20.570332 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:20.570235    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0-credential-socket\") pod \"e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr\" (UID: \"eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0\") " pod="openshift-ingress/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr"
Sep 09 21:12:20.570332 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:20.570283    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0-workload-socket\") pod \"e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr\" (UID: \"eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0\") " pod="openshift-ingress/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr"
Sep 09 21:12:20.570332 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:20.570321    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0-istio-envoy\") pod \"e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr\" (UID: \"eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0\") " pod="openshift-ingress/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr"
Sep 09 21:12:20.570476 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:20.570339    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0-istio-data\") pod \"e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr\" (UID: \"eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0\") " pod="openshift-ingress/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr"
Sep 09 21:12:20.570476 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:20.570367    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0-istio-podinfo\") pod \"e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr\" (UID: \"eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0\") " pod="openshift-ingress/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr"
Sep 09 21:12:20.570476 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:20.570390    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0-istiod-ca-cert\") pod \"e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr\" (UID: \"eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0\") " pod="openshift-ingress/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr"
Sep 09 21:12:20.570476 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:20.570457    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0-istio-token\") pod \"e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr\" (UID: \"eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0\") " pod="openshift-ingress/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr"
Sep 09 21:12:20.671065 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:20.671029    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0-workload-socket\") pod \"e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr\" (UID: \"eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0\") " pod="openshift-ingress/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr"
Sep 09 21:12:20.671257 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:20.671074    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0-istio-envoy\") pod \"e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr\" (UID: \"eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0\") " pod="openshift-ingress/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr"
Sep 09 21:12:20.671257 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:20.671122    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0-istio-data\") pod \"e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr\" (UID: \"eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0\") " pod="openshift-ingress/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr"
Sep 09 21:12:20.671257 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:20.671160    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0-istio-podinfo\") pod \"e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr\" (UID: \"eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0\") " pod="openshift-ingress/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr"
Sep 09 21:12:20.671257 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:20.671176    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0-istiod-ca-cert\") pod \"e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr\" (UID: \"eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0\") " pod="openshift-ingress/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr"
Sep 09 21:12:20.671257 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:20.671197    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0-istio-token\") pod \"e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr\" (UID: \"eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0\") " pod="openshift-ingress/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr"
Sep 09 21:12:20.671518 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:20.671259    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0-workload-certs\") pod \"e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr\" (UID: \"eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0\") " pod="openshift-ingress/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr"
Sep 09 21:12:20.671518 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:20.671417    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25jvz\" (UniqueName: \"kubernetes.io/projected/eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0-kube-api-access-25jvz\") pod \"e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr\" (UID: \"eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0\") " pod="openshift-ingress/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr"
Sep 09 21:12:20.671626 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:20.671515    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0-credential-socket\") pod \"e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr\" (UID: \"eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0\") " pod="openshift-ingress/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr"
Sep 09 21:12:20.671626 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:20.671526    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0-workload-socket\") pod \"e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr\" (UID: \"eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0\") " pod="openshift-ingress/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr"
Sep 09 21:12:20.671626 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:20.671585    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0-istio-data\") pod \"e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr\" (UID: \"eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0\") " pod="openshift-ingress/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr"
Sep 09 21:12:20.671908 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:20.671887    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0-istiod-ca-cert\") pod \"e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr\" (UID: \"eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0\") " pod="openshift-ingress/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr"
Sep 09 21:12:20.672084 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:20.672065    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0-credential-socket\") pod \"e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr\" (UID: \"eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0\") " pod="openshift-ingress/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr"
Sep 09 21:12:20.672177 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:20.672133    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0-workload-certs\") pod \"e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr\" (UID: \"eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0\") " pod="openshift-ingress/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr"
Sep 09 21:12:20.673398 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:20.673380    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0-istio-envoy\") pod \"e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr\" (UID: \"eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0\") " pod="openshift-ingress/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr"
Sep 09 21:12:20.673663 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:20.673647    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0-istio-podinfo\") pod \"e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr\" (UID: \"eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0\") " pod="openshift-ingress/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr"
Sep 09 21:12:20.681949 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:20.681921    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0-istio-token\") pod \"e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr\" (UID: \"eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0\") " pod="openshift-ingress/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr"
Sep 09 21:12:20.682189 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:20.682168    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-25jvz\" (UniqueName: \"kubernetes.io/projected/eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0-kube-api-access-25jvz\") pod \"e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr\" (UID: \"eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0\") " pod="openshift-ingress/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr"
Sep 09 21:12:20.762927 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:20.762896    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr"
Sep 09 21:12:21.042298 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:21.042267    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr"]
Sep 09 21:12:21.043939 ip-10-0-129-99 kubenswrapper[2581]: W0909 21:12:21.043910    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeabfbc4f_b560_4ada_a33e_d6b83e3aa8e0.slice/crio-98552c6ca6d0f5a44b904608ff8ff49c06840b9b29bb4c1951f039d0c86df50e WatchSource:0}: Error finding container 98552c6ca6d0f5a44b904608ff8ff49c06840b9b29bb4c1951f039d0c86df50e: Status 404 returned error can't find the container with id 98552c6ca6d0f5a44b904608ff8ff49c06840b9b29bb4c1951f039d0c86df50e
Sep 09 21:12:21.046168 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:21.046136    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:12:21.046236 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:21.046208    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:12:21.046275 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:21.046238    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:12:21.598160 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:21.598126    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr" event={"ID":"eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0","Type":"ContainerStarted","Data":"86142652d8985c35e8c79266141007dcdfd00a08af81127e77edf5e29f516fd6"}
Sep 09 21:12:21.598560 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:21.598178    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr" event={"ID":"eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0","Type":"ContainerStarted","Data":"98552c6ca6d0f5a44b904608ff8ff49c06840b9b29bb4c1951f039d0c86df50e"}
Sep 09 21:12:21.623483 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:21.623432    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr" podStartSLOduration=1.623415072 podStartE2EDuration="1.623415072s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:12:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:12:21.620713694 +0000 UTC m=+1903.248341572" watchObservedRunningTime="2026-09-09 21:12:21.623415072 +0000 UTC m=+1903.251042944"
Sep 09 21:12:21.763682 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:21.763648    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr"
Sep 09 21:12:22.769394 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:22.769358    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr"
Sep 09 21:12:23.608683 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:23.608652    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr"
Sep 09 21:12:23.609557 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:23.609537    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr"
Sep 09 21:12:25.552196 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:25.552161    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-7c0011-649d4b577c-gfzmv"]
Sep 09 21:12:25.552655 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:25.552639    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="07f6f4a7-5d41-4db5-9c4a-eceaf6e96903" containerName="istio-proxy"
Sep 09 21:12:25.556034 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:25.556017    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-7c0011-649d4b577c-gfzmv"
Sep 09 21:12:25.558604 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:25.558581    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-7c0011-dockercfg-74rwp\""
Sep 09 21:12:25.558750 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:25.558711    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-7c0011\""
Sep 09 21:12:25.573046 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:25.573023    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-7c0011-649d4b577c-gfzmv"]
Sep 09 21:12:25.618587 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:25.618564    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e7471ab4-63da-4678-8437-96c89cca34ba-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-7c0011-649d4b577c-gfzmv\" (UID: \"e7471ab4-63da-4678-8437-96c89cca34ba\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-7c0011-649d4b577c-gfzmv"
Sep 09 21:12:25.618688 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:25.618660    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qtbs\" (UniqueName: \"kubernetes.io/projected/e7471ab4-63da-4678-8437-96c89cca34ba-kube-api-access-2qtbs\") pod \"payload-processing-e2e-ipp-a-7c0011-649d4b577c-gfzmv\" (UID: \"e7471ab4-63da-4678-8437-96c89cca34ba\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-7c0011-649d4b577c-gfzmv"
Sep 09 21:12:25.719468 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:25.719429    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qtbs\" (UniqueName: \"kubernetes.io/projected/e7471ab4-63da-4678-8437-96c89cca34ba-kube-api-access-2qtbs\") pod \"payload-processing-e2e-ipp-a-7c0011-649d4b577c-gfzmv\" (UID: \"e7471ab4-63da-4678-8437-96c89cca34ba\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-7c0011-649d4b577c-gfzmv"
Sep 09 21:12:25.719664 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:25.719503    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e7471ab4-63da-4678-8437-96c89cca34ba-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-7c0011-649d4b577c-gfzmv\" (UID: \"e7471ab4-63da-4678-8437-96c89cca34ba\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-7c0011-649d4b577c-gfzmv"
Sep 09 21:12:25.720258 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:25.720241    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e7471ab4-63da-4678-8437-96c89cca34ba-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-7c0011-649d4b577c-gfzmv\" (UID: \"e7471ab4-63da-4678-8437-96c89cca34ba\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-7c0011-649d4b577c-gfzmv"
Sep 09 21:12:25.728941 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:25.728910    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qtbs\" (UniqueName: \"kubernetes.io/projected/e7471ab4-63da-4678-8437-96c89cca34ba-kube-api-access-2qtbs\") pod \"payload-processing-e2e-ipp-a-7c0011-649d4b577c-gfzmv\" (UID: \"e7471ab4-63da-4678-8437-96c89cca34ba\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-7c0011-649d4b577c-gfzmv"
Sep 09 21:12:25.866335 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:25.866248    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-7c0011-649d4b577c-gfzmv"
Sep 09 21:12:26.141858 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:26.141829    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-7c0011-649d4b577c-gfzmv"]
Sep 09 21:12:26.142681 ip-10-0-129-99 kubenswrapper[2581]: W0909 21:12:26.142647    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7471ab4_63da_4678_8437_96c89cca34ba.slice/crio-2ae30514138f10387378b0724973b32f585d869e5e0cc95fe26401f2662e5ea2 WatchSource:0}: Error finding container 2ae30514138f10387378b0724973b32f585d869e5e0cc95fe26401f2662e5ea2: Status 404 returned error can't find the container with id 2ae30514138f10387378b0724973b32f585d869e5e0cc95fe26401f2662e5ea2
Sep 09 21:12:26.625411 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:26.625378    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-7c0011-649d4b577c-gfzmv" event={"ID":"e7471ab4-63da-4678-8437-96c89cca34ba","Type":"ContainerStarted","Data":"89737067a568f963e93adaff28cf0fe72ac4edd17a72f55ef0978c536b8256b1"}
Sep 09 21:12:26.625823 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:26.625428    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-7c0011-649d4b577c-gfzmv" event={"ID":"e7471ab4-63da-4678-8437-96c89cca34ba","Type":"ContainerStarted","Data":"2ae30514138f10387378b0724973b32f585d869e5e0cc95fe26401f2662e5ea2"}
Sep 09 21:12:26.625823 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:26.625472    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-7c0011-649d4b577c-gfzmv"
Sep 09 21:12:26.655560 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:26.655514    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-7c0011-649d4b577c-gfzmv" podStartSLOduration=1.655500169 podStartE2EDuration="1.655500169s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:12:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:12:26.654699246 +0000 UTC m=+1908.282327121" watchObservedRunningTime="2026-09-09 21:12:26.655500169 +0000 UTC m=+1908.283128042"
Sep 09 21:12:37.632674 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:37.632639    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-7c0011-649d4b577c-gfzmv"
Sep 09 21:12:39.205396 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:39.204751    2581 patch_prober.go:28] interesting pod/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": read tcp 10.133.0.2:57422->10.133.0.87:15021: read: connection reset by peer" start-of-body=
Sep 09 21:12:39.205396 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:39.204819    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr" podUID="eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": read tcp 10.133.0.2:57422->10.133.0.87:15021: read: connection reset by peer"
Sep 09 21:12:39.697865 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:39.697828    2581 generic.go:356] "Generic (PLEG): container finished" podID="eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0" containerID="86142652d8985c35e8c79266141007dcdfd00a08af81127e77edf5e29f516fd6" exitCode=0
Sep 09 21:12:39.698068 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:39.697901    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr" event={"ID":"eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0","Type":"ContainerDied","Data":"86142652d8985c35e8c79266141007dcdfd00a08af81127e77edf5e29f516fd6"}
Sep 09 21:12:39.698385 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:39.698370    2581 scope.go:117] "RemoveContainer" containerID="86142652d8985c35e8c79266141007dcdfd00a08af81127e77edf5e29f516fd6"
Sep 09 21:12:39.699425 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:39.699395    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:12:39.699517 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:39.699461    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:12:39.699517 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:39.699493    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:12:40.707886 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:40.707841    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr" event={"ID":"eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0","Type":"ContainerStarted","Data":"f291ac9faa1cd3e528c8d5bbd5756e755e3de57718e3df2e5da73f7ae37404a3"}
Sep 09 21:12:40.763934 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:40.763902    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr"
Sep 09 21:12:40.764152 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:40.764128    2581 patch_prober.go:28] interesting pod/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Sep 09 21:12:40.764210 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:40.764185    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr" podUID="eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Sep 09 21:12:41.763982 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:41.763949    2581 patch_prober.go:28] interesting pod/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Sep 09 21:12:41.764391 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:41.764018    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr" podUID="eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Sep 09 21:12:42.764267 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:42.764229    2581 patch_prober.go:28] interesting pod/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Sep 09 21:12:42.764657 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:42.764290    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr" podUID="eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Sep 09 21:12:43.763712 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:43.763677    2581 patch_prober.go:28] interesting pod/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Sep 09 21:12:43.763884 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:43.763729    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr" podUID="eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Sep 09 21:12:44.764268 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:44.764234    2581 patch_prober.go:28] interesting pod/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Sep 09 21:12:44.764672 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:44.764298    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr" podUID="eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Sep 09 21:12:45.764176 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:45.764138    2581 patch_prober.go:28] interesting pod/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Sep 09 21:12:45.764354 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:45.764197    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr" podUID="eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Sep 09 21:12:46.767577 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:46.767545    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr"
Sep 09 21:12:46.767949 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:46.767801    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr"
Sep 09 21:12:46.768600 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:46.768577    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr"
Sep 09 21:12:54.576768 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:54.576733    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-7d51a3-6b7c7bd57d-4pjvc"]
Sep 09 21:12:54.580624 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:54.580606    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7d51a3-6b7c7bd57d-4pjvc"
Sep 09 21:12:54.583209 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:54.583188    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-7d51a3-dockercfg-gcdv8\""
Sep 09 21:12:54.583453 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:54.583434    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-7d51a3\""
Sep 09 21:12:54.593783 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:54.593760    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-7d51a3-6b7c7bd57d-4pjvc"]
Sep 09 21:12:54.713656 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:54.713616    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6936836-46e2-4b9b-b33a-f6355c37dfe6-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-7d51a3-6b7c7bd57d-4pjvc\" (UID: \"b6936836-46e2-4b9b-b33a-f6355c37dfe6\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7d51a3-6b7c7bd57d-4pjvc"
Sep 09 21:12:54.713846 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:54.713685    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7z5l\" (UniqueName: \"kubernetes.io/projected/b6936836-46e2-4b9b-b33a-f6355c37dfe6-kube-api-access-p7z5l\") pod \"payload-pre-processing-e2e-ipp-b-7d51a3-6b7c7bd57d-4pjvc\" (UID: \"b6936836-46e2-4b9b-b33a-f6355c37dfe6\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7d51a3-6b7c7bd57d-4pjvc"
Sep 09 21:12:54.814332 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:54.814297    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7z5l\" (UniqueName: \"kubernetes.io/projected/b6936836-46e2-4b9b-b33a-f6355c37dfe6-kube-api-access-p7z5l\") pod \"payload-pre-processing-e2e-ipp-b-7d51a3-6b7c7bd57d-4pjvc\" (UID: \"b6936836-46e2-4b9b-b33a-f6355c37dfe6\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7d51a3-6b7c7bd57d-4pjvc"
Sep 09 21:12:54.814516 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:54.814434    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6936836-46e2-4b9b-b33a-f6355c37dfe6-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-7d51a3-6b7c7bd57d-4pjvc\" (UID: \"b6936836-46e2-4b9b-b33a-f6355c37dfe6\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7d51a3-6b7c7bd57d-4pjvc"
Sep 09 21:12:54.815125 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:54.815090    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6936836-46e2-4b9b-b33a-f6355c37dfe6-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-7d51a3-6b7c7bd57d-4pjvc\" (UID: \"b6936836-46e2-4b9b-b33a-f6355c37dfe6\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7d51a3-6b7c7bd57d-4pjvc"
Sep 09 21:12:54.828936 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:54.828864    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7z5l\" (UniqueName: \"kubernetes.io/projected/b6936836-46e2-4b9b-b33a-f6355c37dfe6-kube-api-access-p7z5l\") pod \"payload-pre-processing-e2e-ipp-b-7d51a3-6b7c7bd57d-4pjvc\" (UID: \"b6936836-46e2-4b9b-b33a-f6355c37dfe6\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7d51a3-6b7c7bd57d-4pjvc"
Sep 09 21:12:54.891462 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:54.891433    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7d51a3-6b7c7bd57d-4pjvc"
Sep 09 21:12:55.181443 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:55.181353    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-7d51a3-6b7c7bd57d-4pjvc"]
Sep 09 21:12:55.188029 ip-10-0-129-99 kubenswrapper[2581]: W0909 21:12:55.188002    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6936836_46e2_4b9b_b33a_f6355c37dfe6.slice/crio-e411e6192b0bc146fb455379db4a1a920c2d5f2a65cabc6997415916a43aa42d WatchSource:0}: Error finding container e411e6192b0bc146fb455379db4a1a920c2d5f2a65cabc6997415916a43aa42d: Status 404 returned error can't find the container with id e411e6192b0bc146fb455379db4a1a920c2d5f2a65cabc6997415916a43aa42d
Sep 09 21:12:55.803349 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:55.803312    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7d51a3-6b7c7bd57d-4pjvc" event={"ID":"b6936836-46e2-4b9b-b33a-f6355c37dfe6","Type":"ContainerStarted","Data":"637183aa6518c6e0522261b98e6aefc139b90be220004e133a9a48a6d810ceb7"}
Sep 09 21:12:55.803765 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:55.803372    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7d51a3-6b7c7bd57d-4pjvc" event={"ID":"b6936836-46e2-4b9b-b33a-f6355c37dfe6","Type":"ContainerStarted","Data":"e411e6192b0bc146fb455379db4a1a920c2d5f2a65cabc6997415916a43aa42d"}
Sep 09 21:12:55.803765 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:55.803491    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7d51a3-6b7c7bd57d-4pjvc"
Sep 09 21:12:55.839116 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:12:55.839054    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7d51a3-6b7c7bd57d-4pjvc" podStartSLOduration=1.839038169 podStartE2EDuration="1.839038169s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:12:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:12:55.837237933 +0000 UTC m=+1937.464865807" watchObservedRunningTime="2026-09-09 21:12:55.839038169 +0000 UTC m=+1937.466666042"
Sep 09 21:13:06.811983 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:13:06.811955    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7d51a3-6b7c7bd57d-4pjvc"
Sep 09 21:13:39.328981 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:13:39.328898    2581 scope.go:117] "RemoveContainer" containerID="7918ea17671346e5b9bc4c398d3e9599fe1a1f23de9fea2a72efc4265bd3bbe6"
Sep 09 21:13:39.337693 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:13:39.337675    2581 scope.go:117] "RemoveContainer" containerID="2c4fedf4f5641df78029439b24cc34f405418b3d2c62420b0fe3ed2f8598e743"
Sep 09 21:13:58.707475 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:13:58.707442    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-7c0011-649d4b577c-gfzmv_e7471ab4-63da-4678-8437-96c89cca34ba/payload-processing/0.log"
Sep 09 21:13:59.529688 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:13:59.529658    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96"]
Sep 09 21:13:59.533291 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:13:59.533273    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96"
Sep 09 21:13:59.536091 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:13:59.536068    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-5fc36f-openshift-default-dockercfg-sb64v\""
Sep 09 21:13:59.647449 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:13:59.647415    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7800a4e-d6a6-44d8-997c-e320cb487a99-credential-socket\") pod \"e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96\" (UID: \"d7800a4e-d6a6-44d8-997c-e320cb487a99\") " pod="openshift-ingress/e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96"
Sep 09 21:13:59.647646 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:13:59.647524    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khjrc\" (UniqueName: \"kubernetes.io/projected/d7800a4e-d6a6-44d8-997c-e320cb487a99-kube-api-access-khjrc\") pod \"e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96\" (UID: \"d7800a4e-d6a6-44d8-997c-e320cb487a99\") " pod="openshift-ingress/e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96"
Sep 09 21:13:59.647646 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:13:59.647631    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7800a4e-d6a6-44d8-997c-e320cb487a99-istio-data\") pod \"e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96\" (UID: \"d7800a4e-d6a6-44d8-997c-e320cb487a99\") " pod="openshift-ingress/e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96"
Sep 09 21:13:59.647777 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:13:59.647652    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96"]
Sep 09 21:13:59.647777 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:13:59.647666    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7800a4e-d6a6-44d8-997c-e320cb487a99-istiod-ca-cert\") pod \"e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96\" (UID: \"d7800a4e-d6a6-44d8-997c-e320cb487a99\") " pod="openshift-ingress/e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96"
Sep 09 21:13:59.647777 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:13:59.647689    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7800a4e-d6a6-44d8-997c-e320cb487a99-istio-envoy\") pod \"e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96\" (UID: \"d7800a4e-d6a6-44d8-997c-e320cb487a99\") " pod="openshift-ingress/e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96"
Sep 09 21:13:59.647777 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:13:59.647769    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7800a4e-d6a6-44d8-997c-e320cb487a99-workload-socket\") pod \"e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96\" (UID: \"d7800a4e-d6a6-44d8-997c-e320cb487a99\") " pod="openshift-ingress/e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96"
Sep 09 21:13:59.647963 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:13:59.647838    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7800a4e-d6a6-44d8-997c-e320cb487a99-istio-podinfo\") pod \"e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96\" (UID: \"d7800a4e-d6a6-44d8-997c-e320cb487a99\") " pod="openshift-ingress/e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96"
Sep 09 21:13:59.647963 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:13:59.647864    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7800a4e-d6a6-44d8-997c-e320cb487a99-workload-certs\") pod \"e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96\" (UID: \"d7800a4e-d6a6-44d8-997c-e320cb487a99\") " pod="openshift-ingress/e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96"
Sep 09 21:13:59.647963 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:13:59.647887    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7800a4e-d6a6-44d8-997c-e320cb487a99-istio-token\") pod \"e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96\" (UID: \"d7800a4e-d6a6-44d8-997c-e320cb487a99\") " pod="openshift-ingress/e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96"
Sep 09 21:13:59.748953 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:13:59.748916    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7800a4e-d6a6-44d8-997c-e320cb487a99-workload-socket\") pod \"e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96\" (UID: \"d7800a4e-d6a6-44d8-997c-e320cb487a99\") " pod="openshift-ingress/e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96"
Sep 09 21:13:59.749465 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:13:59.748979    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7800a4e-d6a6-44d8-997c-e320cb487a99-istio-podinfo\") pod \"e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96\" (UID: \"d7800a4e-d6a6-44d8-997c-e320cb487a99\") " pod="openshift-ingress/e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96"
Sep 09 21:13:59.749465 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:13:59.748999    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7800a4e-d6a6-44d8-997c-e320cb487a99-workload-certs\") pod \"e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96\" (UID: \"d7800a4e-d6a6-44d8-997c-e320cb487a99\") " pod="openshift-ingress/e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96"
Sep 09 21:13:59.749465 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:13:59.749016    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7800a4e-d6a6-44d8-997c-e320cb487a99-istio-token\") pod \"e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96\" (UID: \"d7800a4e-d6a6-44d8-997c-e320cb487a99\") " pod="openshift-ingress/e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96"
Sep 09 21:13:59.749465 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:13:59.749048    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7800a4e-d6a6-44d8-997c-e320cb487a99-credential-socket\") pod \"e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96\" (UID: \"d7800a4e-d6a6-44d8-997c-e320cb487a99\") " pod="openshift-ingress/e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96"
Sep 09 21:13:59.749465 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:13:59.749092    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khjrc\" (UniqueName: \"kubernetes.io/projected/d7800a4e-d6a6-44d8-997c-e320cb487a99-kube-api-access-khjrc\") pod \"e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96\" (UID: \"d7800a4e-d6a6-44d8-997c-e320cb487a99\") " pod="openshift-ingress/e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96"
Sep 09 21:13:59.749465 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:13:59.749166    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7800a4e-d6a6-44d8-997c-e320cb487a99-istio-data\") pod \"e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96\" (UID: \"d7800a4e-d6a6-44d8-997c-e320cb487a99\") " pod="openshift-ingress/e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96"
Sep 09 21:13:59.749465 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:13:59.749196    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7800a4e-d6a6-44d8-997c-e320cb487a99-istiod-ca-cert\") pod \"e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96\" (UID: \"d7800a4e-d6a6-44d8-997c-e320cb487a99\") " pod="openshift-ingress/e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96"
Sep 09 21:13:59.749465 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:13:59.749218    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7800a4e-d6a6-44d8-997c-e320cb487a99-istio-envoy\") pod \"e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96\" (UID: \"d7800a4e-d6a6-44d8-997c-e320cb487a99\") " pod="openshift-ingress/e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96"
Sep 09 21:13:59.749465 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:13:59.749427    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7800a4e-d6a6-44d8-997c-e320cb487a99-workload-socket\") pod \"e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96\" (UID: \"d7800a4e-d6a6-44d8-997c-e320cb487a99\") " pod="openshift-ingress/e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96"
Sep 09 21:13:59.749465 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:13:59.749462    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7800a4e-d6a6-44d8-997c-e320cb487a99-workload-certs\") pod \"e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96\" (UID: \"d7800a4e-d6a6-44d8-997c-e320cb487a99\") " pod="openshift-ingress/e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96"
Sep 09 21:13:59.749826 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:13:59.749544    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7800a4e-d6a6-44d8-997c-e320cb487a99-credential-socket\") pod \"e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96\" (UID: \"d7800a4e-d6a6-44d8-997c-e320cb487a99\") " pod="openshift-ingress/e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96"
Sep 09 21:13:59.749826 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:13:59.749598    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7800a4e-d6a6-44d8-997c-e320cb487a99-istio-data\") pod \"e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96\" (UID: \"d7800a4e-d6a6-44d8-997c-e320cb487a99\") " pod="openshift-ingress/e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96"
Sep 09 21:13:59.749896 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:13:59.749865    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7800a4e-d6a6-44d8-997c-e320cb487a99-istiod-ca-cert\") pod \"e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96\" (UID: \"d7800a4e-d6a6-44d8-997c-e320cb487a99\") " pod="openshift-ingress/e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96"
Sep 09 21:13:59.751493 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:13:59.751468    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7800a4e-d6a6-44d8-997c-e320cb487a99-istio-podinfo\") pod \"e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96\" (UID: \"d7800a4e-d6a6-44d8-997c-e320cb487a99\") " pod="openshift-ingress/e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96"
Sep 09 21:13:59.751678 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:13:59.751654    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7800a4e-d6a6-44d8-997c-e320cb487a99-istio-envoy\") pod \"e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96\" (UID: \"d7800a4e-d6a6-44d8-997c-e320cb487a99\") " pod="openshift-ingress/e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96"
Sep 09 21:13:59.764510 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:13:59.764488    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-khjrc\" (UniqueName: \"kubernetes.io/projected/d7800a4e-d6a6-44d8-997c-e320cb487a99-kube-api-access-khjrc\") pod \"e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96\" (UID: \"d7800a4e-d6a6-44d8-997c-e320cb487a99\") " pod="openshift-ingress/e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96"
Sep 09 21:13:59.767868 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:13:59.767850    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7800a4e-d6a6-44d8-997c-e320cb487a99-istio-token\") pod \"e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96\" (UID: \"d7800a4e-d6a6-44d8-997c-e320cb487a99\") " pod="openshift-ingress/e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96"
Sep 09 21:13:59.843822 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:13:59.843737    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96"
Sep 09 21:14:00.133743 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:00.133711    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96"]
Sep 09 21:14:00.134838 ip-10-0-129-99 kubenswrapper[2581]: W0909 21:14:00.134809    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7800a4e_d6a6_44d8_997c_e320cb487a99.slice/crio-65c214cc6f4623ddad8f6c08d1954ba3bcf0375f796cc5b27f03d0ad49e56a91 WatchSource:0}: Error finding container 65c214cc6f4623ddad8f6c08d1954ba3bcf0375f796cc5b27f03d0ad49e56a91: Status 404 returned error can't find the container with id 65c214cc6f4623ddad8f6c08d1954ba3bcf0375f796cc5b27f03d0ad49e56a91
Sep 09 21:14:00.137066 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:00.137026    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:14:00.137184 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:00.137112    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:14:00.137184 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:00.137150    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:14:01.142415 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:01.142380    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96" event={"ID":"d7800a4e-d6a6-44d8-997c-e320cb487a99","Type":"ContainerStarted","Data":"2c200817b54216bc80176a3fcfb0c935107cedc9e83f31a50af58f591273f9b9"}
Sep 09 21:14:01.142824 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:01.142433    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96" event={"ID":"d7800a4e-d6a6-44d8-997c-e320cb487a99","Type":"ContainerStarted","Data":"65c214cc6f4623ddad8f6c08d1954ba3bcf0375f796cc5b27f03d0ad49e56a91"}
Sep 09 21:14:01.173474 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:01.173424    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96" podStartSLOduration=2.173409707 podStartE2EDuration="2.173409707s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:13:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:14:01.171013741 +0000 UTC m=+2002.798641615" watchObservedRunningTime="2026-09-09 21:14:01.173409707 +0000 UTC m=+2002.801037580"
Sep 09 21:14:01.844185 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:01.844146    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96"
Sep 09 21:14:01.849264 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:01.849237    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96"
Sep 09 21:14:02.148547 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:02.148429    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96"
Sep 09 21:14:02.149272 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:02.149252    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96"
Sep 09 21:14:05.096129 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:05.093539    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-5fc36f-6f857bd9bf-kszjk"]
Sep 09 21:14:05.099445 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:05.099417    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5fc36f-6f857bd9bf-kszjk"
Sep 09 21:14:05.102267 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:05.102247    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-5fc36f\""
Sep 09 21:14:05.111358 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:05.111335    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-5fc36f-6f857bd9bf-kszjk"]
Sep 09 21:14:05.203270 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:05.203235    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qqtt\" (UniqueName: \"kubernetes.io/projected/fc2d7dd3-043c-41aa-9b92-5d2d9e905dba-kube-api-access-8qqtt\") pod \"payload-processing-e2e-ipp-cleanup-5fc36f-6f857bd9bf-kszjk\" (UID: \"fc2d7dd3-043c-41aa-9b92-5d2d9e905dba\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5fc36f-6f857bd9bf-kszjk"
Sep 09 21:14:05.203429 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:05.203319    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc2d7dd3-043c-41aa-9b92-5d2d9e905dba-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-5fc36f-6f857bd9bf-kszjk\" (UID: \"fc2d7dd3-043c-41aa-9b92-5d2d9e905dba\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5fc36f-6f857bd9bf-kszjk"
Sep 09 21:14:05.304840 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:05.304802    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qqtt\" (UniqueName: \"kubernetes.io/projected/fc2d7dd3-043c-41aa-9b92-5d2d9e905dba-kube-api-access-8qqtt\") pod \"payload-processing-e2e-ipp-cleanup-5fc36f-6f857bd9bf-kszjk\" (UID: \"fc2d7dd3-043c-41aa-9b92-5d2d9e905dba\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5fc36f-6f857bd9bf-kszjk"
Sep 09 21:14:05.305020 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:05.304877    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc2d7dd3-043c-41aa-9b92-5d2d9e905dba-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-5fc36f-6f857bd9bf-kszjk\" (UID: \"fc2d7dd3-043c-41aa-9b92-5d2d9e905dba\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5fc36f-6f857bd9bf-kszjk"
Sep 09 21:14:05.305621 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:05.305602    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc2d7dd3-043c-41aa-9b92-5d2d9e905dba-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-5fc36f-6f857bd9bf-kszjk\" (UID: \"fc2d7dd3-043c-41aa-9b92-5d2d9e905dba\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5fc36f-6f857bd9bf-kszjk"
Sep 09 21:14:05.314088 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:05.314059    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qqtt\" (UniqueName: \"kubernetes.io/projected/fc2d7dd3-043c-41aa-9b92-5d2d9e905dba-kube-api-access-8qqtt\") pod \"payload-processing-e2e-ipp-cleanup-5fc36f-6f857bd9bf-kszjk\" (UID: \"fc2d7dd3-043c-41aa-9b92-5d2d9e905dba\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5fc36f-6f857bd9bf-kszjk"
Sep 09 21:14:05.411941 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:05.411845    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5fc36f-6f857bd9bf-kszjk"
Sep 09 21:14:05.695641 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:05.695617    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-5fc36f-6f857bd9bf-kszjk"]
Sep 09 21:14:05.698183 ip-10-0-129-99 kubenswrapper[2581]: W0909 21:14:05.698154    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc2d7dd3_043c_41aa_9b92_5d2d9e905dba.slice/crio-caf00e5261e3656c176deeff4b69e8959170c363028ce7314944fc16de5cacd0 WatchSource:0}: Error finding container caf00e5261e3656c176deeff4b69e8959170c363028ce7314944fc16de5cacd0: Status 404 returned error can't find the container with id caf00e5261e3656c176deeff4b69e8959170c363028ce7314944fc16de5cacd0
Sep 09 21:14:06.170474 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:06.170432    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5fc36f-6f857bd9bf-kszjk" event={"ID":"fc2d7dd3-043c-41aa-9b92-5d2d9e905dba","Type":"ContainerStarted","Data":"b3db766a79d338106cd4c5bcb5fbc56b9969ba541d71a89086af893320ed0b8c"}
Sep 09 21:14:06.170893 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:06.170487    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5fc36f-6f857bd9bf-kszjk" event={"ID":"fc2d7dd3-043c-41aa-9b92-5d2d9e905dba","Type":"ContainerStarted","Data":"caf00e5261e3656c176deeff4b69e8959170c363028ce7314944fc16de5cacd0"}
Sep 09 21:14:06.170893 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:06.170515    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5fc36f-6f857bd9bf-kszjk"
Sep 09 21:14:06.195525 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:06.195478    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5fc36f-6f857bd9bf-kszjk" podStartSLOduration=1.195464169 podStartE2EDuration="1.195464169s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:14:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:14:06.19295741 +0000 UTC m=+2007.820585277" watchObservedRunningTime="2026-09-09 21:14:06.195464169 +0000 UTC m=+2007.823092082"
Sep 09 21:14:17.177282 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:17.177247    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5fc36f-6f857bd9bf-kszjk"
Sep 09 21:14:28.769793 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:28.769754    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-5fc36f-dgjll"]
Sep 09 21:14:28.773526 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:28.773507    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-5fc36f-dgjll"
Sep 09 21:14:28.776868 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:28.776848    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-glz44\""
Sep 09 21:14:28.784376 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:28.784353    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-5fc36f-dgjll"]
Sep 09 21:14:28.846876 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:28.846835    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/30a1c2b6-7bd2-47c9-b692-ff55885fc77e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-5fc36f-dgjll\" (UID: \"30a1c2b6-7bd2-47c9-b692-ff55885fc77e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-5fc36f-dgjll"
Sep 09 21:14:28.947970 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:28.947921    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/30a1c2b6-7bd2-47c9-b692-ff55885fc77e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-5fc36f-dgjll\" (UID: \"30a1c2b6-7bd2-47c9-b692-ff55885fc77e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-5fc36f-dgjll"
Sep 09 21:14:28.948602 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:28.948578    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/30a1c2b6-7bd2-47c9-b692-ff55885fc77e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-5fc36f-dgjll\" (UID: \"30a1c2b6-7bd2-47c9-b692-ff55885fc77e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-5fc36f-dgjll"
Sep 09 21:14:29.084995 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:29.084899    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-5fc36f-dgjll"
Sep 09 21:14:29.406172 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:29.405149    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-5fc36f-dgjll"]
Sep 09 21:14:29.423261 ip-10-0-129-99 kubenswrapper[2581]: W0909 21:14:29.423224    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30a1c2b6_7bd2_47c9_b692_ff55885fc77e.slice/crio-3997c1bd572ffff7ce2c0bfcd06c429830c8f0a567c35f49ec8fa777ca604bd8 WatchSource:0}: Error finding container 3997c1bd572ffff7ce2c0bfcd06c429830c8f0a567c35f49ec8fa777ca604bd8: Status 404 returned error can't find the container with id 3997c1bd572ffff7ce2c0bfcd06c429830c8f0a567c35f49ec8fa777ca604bd8
Sep 09 21:14:30.298174 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:30.298135    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-5fc36f-dgjll" event={"ID":"30a1c2b6-7bd2-47c9-b692-ff55885fc77e","Type":"ContainerStarted","Data":"0fa058e88059d9cfd33ca1c7adb4d09a335be31502a91daad170ef6f7c406942"}
Sep 09 21:14:30.298568 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:30.298196    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-5fc36f-dgjll" event={"ID":"30a1c2b6-7bd2-47c9-b692-ff55885fc77e","Type":"ContainerStarted","Data":"3997c1bd572ffff7ce2c0bfcd06c429830c8f0a567c35f49ec8fa777ca604bd8"}
Sep 09 21:14:31.308309 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:31.308264    2581 generic.go:356] "Generic (PLEG): container finished" podID="30a1c2b6-7bd2-47c9-b692-ff55885fc77e" containerID="0fa058e88059d9cfd33ca1c7adb4d09a335be31502a91daad170ef6f7c406942" exitCode=0
Sep 09 21:14:31.308824 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:31.308395    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-5fc36f-dgjll" event={"ID":"30a1c2b6-7bd2-47c9-b692-ff55885fc77e","Type":"ContainerDied","Data":"0fa058e88059d9cfd33ca1c7adb4d09a335be31502a91daad170ef6f7c406942"}
Sep 09 21:14:32.456738 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:32.456708    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-5fc36f-dgjll"
Sep 09 21:14:32.586885 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:32.586782    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/30a1c2b6-7bd2-47c9-b692-ff55885fc77e-maas-api-service-ca\") pod \"30a1c2b6-7bd2-47c9-b692-ff55885fc77e\" (UID: \"30a1c2b6-7bd2-47c9-b692-ff55885fc77e\") "
Sep 09 21:14:32.587210 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:32.587184    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30a1c2b6-7bd2-47c9-b692-ff55885fc77e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "30a1c2b6-7bd2-47c9-b692-ff55885fc77e" (UID: "30a1c2b6-7bd2-47c9-b692-ff55885fc77e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:14:32.688558 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:32.688516    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/30a1c2b6-7bd2-47c9-b692-ff55885fc77e-maas-api-service-ca\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:14:33.190315 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:33.190270    2581 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.90:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 09 21:14:33.190496 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:33.190351    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96" podUID="d7800a4e-d6a6-44d8-997c-e320cb487a99" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.90:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 09 21:14:33.323337 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:33.323309    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-5fc36f-dgjll"
Sep 09 21:14:33.323501 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:33.323348    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-5fc36f-dgjll" event={"ID":"30a1c2b6-7bd2-47c9-b692-ff55885fc77e","Type":"ContainerDied","Data":"3997c1bd572ffff7ce2c0bfcd06c429830c8f0a567c35f49ec8fa777ca604bd8"}
Sep 09 21:14:33.323501 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:33.323390    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3997c1bd572ffff7ce2c0bfcd06c429830c8f0a567c35f49ec8fa777ca604bd8"
Sep 09 21:14:35.313878 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:35.313846    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-5fc36f-6f857bd9bf-kszjk"]
Sep 09 21:14:35.314390 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:35.314084    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5fc36f-6f857bd9bf-kszjk" podUID="fc2d7dd3-043c-41aa-9b92-5d2d9e905dba" containerName="payload-processing" containerID="cri-o://b3db766a79d338106cd4c5bcb5fbc56b9969ba541d71a89086af893320ed0b8c" gracePeriod=30
Sep 09 21:14:35.352976 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:35.352944    2581 generic.go:356] "Generic (PLEG): container finished" podID="d7800a4e-d6a6-44d8-997c-e320cb487a99" containerID="2c200817b54216bc80176a3fcfb0c935107cedc9e83f31a50af58f591273f9b9" exitCode=0
Sep 09 21:14:35.353136 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:35.353020    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96" event={"ID":"d7800a4e-d6a6-44d8-997c-e320cb487a99","Type":"ContainerDied","Data":"2c200817b54216bc80176a3fcfb0c935107cedc9e83f31a50af58f591273f9b9"}
Sep 09 21:14:35.353546 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:35.353528    2581 scope.go:117] "RemoveContainer" containerID="2c200817b54216bc80176a3fcfb0c935107cedc9e83f31a50af58f591273f9b9"
Sep 09 21:14:35.354803 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:35.354768    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:14:35.354909 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:35.354848    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:14:35.354909 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:35.354896    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:14:35.600240 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:35.600214    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5fc36f-6f857bd9bf-kszjk"
Sep 09 21:14:35.722993 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:35.722946    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8qqtt\" (UniqueName: \"kubernetes.io/projected/fc2d7dd3-043c-41aa-9b92-5d2d9e905dba-kube-api-access-8qqtt\") pod \"fc2d7dd3-043c-41aa-9b92-5d2d9e905dba\" (UID: \"fc2d7dd3-043c-41aa-9b92-5d2d9e905dba\") "
Sep 09 21:14:35.723197 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:35.722999    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc2d7dd3-043c-41aa-9b92-5d2d9e905dba-plugins-config-volume\") pod \"fc2d7dd3-043c-41aa-9b92-5d2d9e905dba\" (UID: \"fc2d7dd3-043c-41aa-9b92-5d2d9e905dba\") "
Sep 09 21:14:35.723724 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:35.723693    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc2d7dd3-043c-41aa-9b92-5d2d9e905dba-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fc2d7dd3-043c-41aa-9b92-5d2d9e905dba" (UID: "fc2d7dd3-043c-41aa-9b92-5d2d9e905dba"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:14:35.725448 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:35.725407    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc2d7dd3-043c-41aa-9b92-5d2d9e905dba-kube-api-access-8qqtt" (OuterVolumeSpecName: "kube-api-access-8qqtt") pod "fc2d7dd3-043c-41aa-9b92-5d2d9e905dba" (UID: "fc2d7dd3-043c-41aa-9b92-5d2d9e905dba"). InnerVolumeSpecName "kube-api-access-8qqtt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:14:35.810646 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:35.810597    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-5fc36f-dgjll"]
Sep 09 21:14:35.822560 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:35.822531    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-5fc36f-dgjll"]
Sep 09 21:14:35.824579 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:35.824557    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8qqtt\" (UniqueName: \"kubernetes.io/projected/fc2d7dd3-043c-41aa-9b92-5d2d9e905dba-kube-api-access-8qqtt\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:14:35.824670 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:35.824581    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc2d7dd3-043c-41aa-9b92-5d2d9e905dba-plugins-config-volume\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:14:36.360182 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:36.360146    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96" event={"ID":"d7800a4e-d6a6-44d8-997c-e320cb487a99","Type":"ContainerStarted","Data":"32d400d933934bb9d996b407d90c361aaf223a425a1dbf43bee1ff4c08a6060e"}
Sep 09 21:14:36.361420 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:36.361392    2581 generic.go:356] "Generic (PLEG): container finished" podID="fc2d7dd3-043c-41aa-9b92-5d2d9e905dba" containerID="b3db766a79d338106cd4c5bcb5fbc56b9969ba541d71a89086af893320ed0b8c" exitCode=0
Sep 09 21:14:36.361538 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:36.361421    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5fc36f-6f857bd9bf-kszjk"
Sep 09 21:14:36.361538 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:36.361466    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5fc36f-6f857bd9bf-kszjk" event={"ID":"fc2d7dd3-043c-41aa-9b92-5d2d9e905dba","Type":"ContainerDied","Data":"b3db766a79d338106cd4c5bcb5fbc56b9969ba541d71a89086af893320ed0b8c"}
Sep 09 21:14:36.361538 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:36.361507    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-5fc36f-6f857bd9bf-kszjk" event={"ID":"fc2d7dd3-043c-41aa-9b92-5d2d9e905dba","Type":"ContainerDied","Data":"caf00e5261e3656c176deeff4b69e8959170c363028ce7314944fc16de5cacd0"}
Sep 09 21:14:36.361538 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:36.361528    2581 scope.go:117] "RemoveContainer" containerID="b3db766a79d338106cd4c5bcb5fbc56b9969ba541d71a89086af893320ed0b8c"
Sep 09 21:14:36.370577 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:36.370560    2581 scope.go:117] "RemoveContainer" containerID="b3db766a79d338106cd4c5bcb5fbc56b9969ba541d71a89086af893320ed0b8c"
Sep 09 21:14:36.370821 ip-10-0-129-99 kubenswrapper[2581]: E0909 21:14:36.370800    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3db766a79d338106cd4c5bcb5fbc56b9969ba541d71a89086af893320ed0b8c\": container with ID starting with b3db766a79d338106cd4c5bcb5fbc56b9969ba541d71a89086af893320ed0b8c not found: ID does not exist" containerID="b3db766a79d338106cd4c5bcb5fbc56b9969ba541d71a89086af893320ed0b8c"
Sep 09 21:14:36.370862 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:36.370829    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3db766a79d338106cd4c5bcb5fbc56b9969ba541d71a89086af893320ed0b8c"} err="failed to get container status \"b3db766a79d338106cd4c5bcb5fbc56b9969ba541d71a89086af893320ed0b8c\": rpc error: code = NotFound desc = could not find container \"b3db766a79d338106cd4c5bcb5fbc56b9969ba541d71a89086af893320ed0b8c\": container with ID starting with b3db766a79d338106cd4c5bcb5fbc56b9969ba541d71a89086af893320ed0b8c not found: ID does not exist"
Sep 09 21:14:36.409607 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:36.409569    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-5fc36f-6f857bd9bf-kszjk"]
Sep 09 21:14:36.412958 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:36.412922    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-5fc36f-6f857bd9bf-kszjk"]
Sep 09 21:14:36.826666 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:36.826627    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30a1c2b6-7bd2-47c9-b692-ff55885fc77e" path="/var/lib/kubelet/pods/30a1c2b6-7bd2-47c9-b692-ff55885fc77e/volumes"
Sep 09 21:14:36.827221 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:36.827202    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc2d7dd3-043c-41aa-9b92-5d2d9e905dba" path="/var/lib/kubelet/pods/fc2d7dd3-043c-41aa-9b92-5d2d9e905dba/volumes"
Sep 09 21:14:36.844827 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:36.844793    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96"
Sep 09 21:14:36.845088 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:36.845062    2581 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused" start-of-body=
Sep 09 21:14:36.845229 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:36.845160    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96" podUID="d7800a4e-d6a6-44d8-997c-e320cb487a99" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused"
Sep 09 21:14:37.844666 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:37.844623    2581 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused" start-of-body=
Sep 09 21:14:37.845193 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:37.844686    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96" podUID="d7800a4e-d6a6-44d8-997c-e320cb487a99" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused"
Sep 09 21:14:38.844920 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:38.844884    2581 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused" start-of-body=
Sep 09 21:14:38.845316 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:38.844949    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96" podUID="d7800a4e-d6a6-44d8-997c-e320cb487a99" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused"
Sep 09 21:14:39.360804 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:39.360773    2581 scope.go:117] "RemoveContainer" containerID="8f67311221fc43072e4a2b4c73402d25e878b77a2b8ae4ac13214de2a944efaa"
Sep 09 21:14:39.369278 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:39.369255    2581 scope.go:117] "RemoveContainer" containerID="75412e3e2952004f4d9bb7e120ddb61a01eecd3b6dd5cda98d66d25ed476ae14"
Sep 09 21:14:39.376765 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:39.376743    2581 scope.go:117] "RemoveContainer" containerID="bc2b81e3e7ec16af20bd3db59ffbdd3c248c3f725dfe602b3c13ab87e37f05ba"
Sep 09 21:14:39.844906 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:39.844870    2581 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused" start-of-body=
Sep 09 21:14:39.845118 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:39.844927    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96" podUID="d7800a4e-d6a6-44d8-997c-e320cb487a99" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused"
Sep 09 21:14:40.844439 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:40.844400    2581 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused" start-of-body=
Sep 09 21:14:40.844611 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:40.844465    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96" podUID="d7800a4e-d6a6-44d8-997c-e320cb487a99" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused"
Sep 09 21:14:41.883297 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:41.883263    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96"
Sep 09 21:14:41.883846 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:41.883705    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96"
Sep 09 21:14:41.884534 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:41.884507    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96"
Sep 09 21:14:42.604378 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:42.604344    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96"]
Sep 09 21:14:43.400016 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:43.399978    2581 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96" secret="" err="secret \"e2e-ipp-cleanup-5fc36f-openshift-default-dockercfg-sb64v\" not found"
Sep 09 21:14:44.405138 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:44.405083    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96" podUID="d7800a4e-d6a6-44d8-997c-e320cb487a99" containerName="istio-proxy" containerID="cri-o://32d400d933934bb9d996b407d90c361aaf223a425a1dbf43bee1ff4c08a6060e" gracePeriod=30
Sep 09 21:14:46.113380 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:46.113343    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7c0011-f2mm5"]
Sep 09 21:14:46.113792 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:46.113775    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc2d7dd3-043c-41aa-9b92-5d2d9e905dba" containerName="payload-processing"
Sep 09 21:14:46.113792 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:46.113790    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc2d7dd3-043c-41aa-9b92-5d2d9e905dba" containerName="payload-processing"
Sep 09 21:14:46.113950 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:46.113803    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30a1c2b6-7bd2-47c9-b692-ff55885fc77e" containerName="revoke-keys"
Sep 09 21:14:46.113950 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:46.113809    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="30a1c2b6-7bd2-47c9-b692-ff55885fc77e" containerName="revoke-keys"
Sep 09 21:14:46.113950 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:46.113884    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc2d7dd3-043c-41aa-9b92-5d2d9e905dba" containerName="payload-processing"
Sep 09 21:14:46.113950 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:46.113892    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="30a1c2b6-7bd2-47c9-b692-ff55885fc77e" containerName="revoke-keys"
Sep 09 21:14:46.116961 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:46.116944    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7c0011-f2mm5"
Sep 09 21:14:46.121656 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:46.121638    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-glz44\""
Sep 09 21:14:46.131617 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:46.131593    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7c0011-f2mm5"]
Sep 09 21:14:46.230542 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:46.230508    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/43507456-7508-4b3c-bd42-929e8d63dd06-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-7c0011-f2mm5\" (UID: \"43507456-7508-4b3c-bd42-929e8d63dd06\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7c0011-f2mm5"
Sep 09 21:14:46.331492 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:46.331454    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/43507456-7508-4b3c-bd42-929e8d63dd06-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-7c0011-f2mm5\" (UID: \"43507456-7508-4b3c-bd42-929e8d63dd06\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7c0011-f2mm5"
Sep 09 21:14:46.332095 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:46.332073    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/43507456-7508-4b3c-bd42-929e8d63dd06-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-7c0011-f2mm5\" (UID: \"43507456-7508-4b3c-bd42-929e8d63dd06\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7c0011-f2mm5"
Sep 09 21:14:46.427511 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:46.427430    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7c0011-f2mm5"
Sep 09 21:14:46.707431 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:46.707408    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7c0011-f2mm5"]
Sep 09 21:14:46.710367 ip-10-0-129-99 kubenswrapper[2581]: W0909 21:14:46.710341    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43507456_7508_4b3c_bd42_929e8d63dd06.slice/crio-ecf2398e05fd0e3ac04c709b18c71a6c961acb920774ccb00498448c177b7038 WatchSource:0}: Error finding container ecf2398e05fd0e3ac04c709b18c71a6c961acb920774ccb00498448c177b7038: Status 404 returned error can't find the container with id ecf2398e05fd0e3ac04c709b18c71a6c961acb920774ccb00498448c177b7038
Sep 09 21:14:47.422935 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:47.422898    2581 generic.go:356] "Generic (PLEG): container finished" podID="43507456-7508-4b3c-bd42-929e8d63dd06" containerID="d3c6447bb2649c01640192a40058d5249ea8600922bc660dcc9573c8a5751993" exitCode=0
Sep 09 21:14:47.423348 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:47.422944    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7c0011-f2mm5" event={"ID":"43507456-7508-4b3c-bd42-929e8d63dd06","Type":"ContainerDied","Data":"d3c6447bb2649c01640192a40058d5249ea8600922bc660dcc9573c8a5751993"}
Sep 09 21:14:47.423348 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:47.423002    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7c0011-f2mm5" event={"ID":"43507456-7508-4b3c-bd42-929e8d63dd06","Type":"ContainerStarted","Data":"ecf2398e05fd0e3ac04c709b18c71a6c961acb920774ccb00498448c177b7038"}
Sep 09 21:14:48.566880 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:48.566859    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7c0011-f2mm5"
Sep 09 21:14:48.655353 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:48.655320    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/43507456-7508-4b3c-bd42-929e8d63dd06-maas-api-service-ca\") pod \"43507456-7508-4b3c-bd42-929e8d63dd06\" (UID: \"43507456-7508-4b3c-bd42-929e8d63dd06\") "
Sep 09 21:14:48.655686 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:48.655661    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43507456-7508-4b3c-bd42-929e8d63dd06-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "43507456-7508-4b3c-bd42-929e8d63dd06" (UID: "43507456-7508-4b3c-bd42-929e8d63dd06"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:14:48.756437 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:48.756407    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/43507456-7508-4b3c-bd42-929e8d63dd06-maas-api-service-ca\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:14:49.434861 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:49.434839    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7c0011-f2mm5"
Sep 09 21:14:49.434861 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:49.434844    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7c0011-f2mm5" event={"ID":"43507456-7508-4b3c-bd42-929e8d63dd06","Type":"ContainerDied","Data":"ecf2398e05fd0e3ac04c709b18c71a6c961acb920774ccb00498448c177b7038"}
Sep 09 21:14:49.435025 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:49.434887    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ecf2398e05fd0e3ac04c709b18c71a6c961acb920774ccb00498448c177b7038"
Sep 09 21:14:49.665801 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:49.665744    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96"
Sep 09 21:14:49.766766 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:49.766732    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7800a4e-d6a6-44d8-997c-e320cb487a99-workload-certs\") pod \"d7800a4e-d6a6-44d8-997c-e320cb487a99\" (UID: \"d7800a4e-d6a6-44d8-997c-e320cb487a99\") "
Sep 09 21:14:49.766931 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:49.766789    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7800a4e-d6a6-44d8-997c-e320cb487a99-istio-envoy\") pod \"d7800a4e-d6a6-44d8-997c-e320cb487a99\" (UID: \"d7800a4e-d6a6-44d8-997c-e320cb487a99\") "
Sep 09 21:14:49.766931 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:49.766827    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7800a4e-d6a6-44d8-997c-e320cb487a99-istio-data\") pod \"d7800a4e-d6a6-44d8-997c-e320cb487a99\" (UID: \"d7800a4e-d6a6-44d8-997c-e320cb487a99\") "
Sep 09 21:14:49.766931 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:49.766855    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7800a4e-d6a6-44d8-997c-e320cb487a99-istio-token\") pod \"d7800a4e-d6a6-44d8-997c-e320cb487a99\" (UID: \"d7800a4e-d6a6-44d8-997c-e320cb487a99\") "
Sep 09 21:14:49.766931 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:49.766882    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-khjrc\" (UniqueName: \"kubernetes.io/projected/d7800a4e-d6a6-44d8-997c-e320cb487a99-kube-api-access-khjrc\") pod \"d7800a4e-d6a6-44d8-997c-e320cb487a99\" (UID: \"d7800a4e-d6a6-44d8-997c-e320cb487a99\") "
Sep 09 21:14:49.766931 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:49.766911    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7800a4e-d6a6-44d8-997c-e320cb487a99-credential-socket\") pod \"d7800a4e-d6a6-44d8-997c-e320cb487a99\" (UID: \"d7800a4e-d6a6-44d8-997c-e320cb487a99\") "
Sep 09 21:14:49.767251 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:49.766950    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7800a4e-d6a6-44d8-997c-e320cb487a99-istio-podinfo\") pod \"d7800a4e-d6a6-44d8-997c-e320cb487a99\" (UID: \"d7800a4e-d6a6-44d8-997c-e320cb487a99\") "
Sep 09 21:14:49.767251 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:49.767015    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7800a4e-d6a6-44d8-997c-e320cb487a99-workload-socket\") pod \"d7800a4e-d6a6-44d8-997c-e320cb487a99\" (UID: \"d7800a4e-d6a6-44d8-997c-e320cb487a99\") "
Sep 09 21:14:49.767251 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:49.767055    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7800a4e-d6a6-44d8-997c-e320cb487a99-istiod-ca-cert\") pod \"d7800a4e-d6a6-44d8-997c-e320cb487a99\" (UID: \"d7800a4e-d6a6-44d8-997c-e320cb487a99\") "
Sep 09 21:14:49.767251 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:49.767064    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7800a4e-d6a6-44d8-997c-e320cb487a99-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d7800a4e-d6a6-44d8-997c-e320cb487a99" (UID: "d7800a4e-d6a6-44d8-997c-e320cb487a99"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:14:49.767251 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:49.767204    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7800a4e-d6a6-44d8-997c-e320cb487a99-istio-data" (OuterVolumeSpecName: "istio-data") pod "d7800a4e-d6a6-44d8-997c-e320cb487a99" (UID: "d7800a4e-d6a6-44d8-997c-e320cb487a99"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:14:49.767510 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:49.767319    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7800a4e-d6a6-44d8-997c-e320cb487a99-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d7800a4e-d6a6-44d8-997c-e320cb487a99" (UID: "d7800a4e-d6a6-44d8-997c-e320cb487a99"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:14:49.767510 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:49.767354    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7800a4e-d6a6-44d8-997c-e320cb487a99-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d7800a4e-d6a6-44d8-997c-e320cb487a99" (UID: "d7800a4e-d6a6-44d8-997c-e320cb487a99"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:14:49.767510 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:49.767495    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7800a4e-d6a6-44d8-997c-e320cb487a99-workload-certs\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:14:49.767668 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:49.767507    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7800a4e-d6a6-44d8-997c-e320cb487a99-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d7800a4e-d6a6-44d8-997c-e320cb487a99" (UID: "d7800a4e-d6a6-44d8-997c-e320cb487a99"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:14:49.767668 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:49.767515    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7800a4e-d6a6-44d8-997c-e320cb487a99-istio-data\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:14:49.767668 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:49.767549    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7800a4e-d6a6-44d8-997c-e320cb487a99-credential-socket\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:14:49.767668 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:49.767567    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7800a4e-d6a6-44d8-997c-e320cb487a99-workload-socket\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:14:49.769130 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:49.769082    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7800a4e-d6a6-44d8-997c-e320cb487a99-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d7800a4e-d6a6-44d8-997c-e320cb487a99" (UID: "d7800a4e-d6a6-44d8-997c-e320cb487a99"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:14:49.769279 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:49.769259    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7800a4e-d6a6-44d8-997c-e320cb487a99-kube-api-access-khjrc" (OuterVolumeSpecName: "kube-api-access-khjrc") pod "d7800a4e-d6a6-44d8-997c-e320cb487a99" (UID: "d7800a4e-d6a6-44d8-997c-e320cb487a99"). InnerVolumeSpecName "kube-api-access-khjrc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:14:49.769637 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:49.769615    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d7800a4e-d6a6-44d8-997c-e320cb487a99-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d7800a4e-d6a6-44d8-997c-e320cb487a99" (UID: "d7800a4e-d6a6-44d8-997c-e320cb487a99"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 21:14:49.769693 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:49.769617    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7800a4e-d6a6-44d8-997c-e320cb487a99-istio-token" (OuterVolumeSpecName: "istio-token") pod "d7800a4e-d6a6-44d8-997c-e320cb487a99" (UID: "d7800a4e-d6a6-44d8-997c-e320cb487a99"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:14:49.868510 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:49.868464    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-khjrc\" (UniqueName: \"kubernetes.io/projected/d7800a4e-d6a6-44d8-997c-e320cb487a99-kube-api-access-khjrc\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:14:49.868510 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:49.868505    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7800a4e-d6a6-44d8-997c-e320cb487a99-istio-podinfo\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:14:49.868510 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:49.868515    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7800a4e-d6a6-44d8-997c-e320cb487a99-istiod-ca-cert\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:14:49.868510 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:49.868523    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7800a4e-d6a6-44d8-997c-e320cb487a99-istio-envoy\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:14:49.868760 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:49.868531    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7800a4e-d6a6-44d8-997c-e320cb487a99-istio-token\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:14:50.442491 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:50.442451    2581 generic.go:356] "Generic (PLEG): container finished" podID="d7800a4e-d6a6-44d8-997c-e320cb487a99" containerID="32d400d933934bb9d996b407d90c361aaf223a425a1dbf43bee1ff4c08a6060e" exitCode=0
Sep 09 21:14:50.442693 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:50.442502    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96"
Sep 09 21:14:50.442693 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:50.442504    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96" event={"ID":"d7800a4e-d6a6-44d8-997c-e320cb487a99","Type":"ContainerDied","Data":"32d400d933934bb9d996b407d90c361aaf223a425a1dbf43bee1ff4c08a6060e"}
Sep 09 21:14:50.442693 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:50.442642    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96" event={"ID":"d7800a4e-d6a6-44d8-997c-e320cb487a99","Type":"ContainerDied","Data":"65c214cc6f4623ddad8f6c08d1954ba3bcf0375f796cc5b27f03d0ad49e56a91"}
Sep 09 21:14:50.442693 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:50.442675    2581 scope.go:117] "RemoveContainer" containerID="32d400d933934bb9d996b407d90c361aaf223a425a1dbf43bee1ff4c08a6060e"
Sep 09 21:14:50.453463 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:50.453441    2581 scope.go:117] "RemoveContainer" containerID="2c200817b54216bc80176a3fcfb0c935107cedc9e83f31a50af58f591273f9b9"
Sep 09 21:14:50.463992 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:50.463973    2581 scope.go:117] "RemoveContainer" containerID="32d400d933934bb9d996b407d90c361aaf223a425a1dbf43bee1ff4c08a6060e"
Sep 09 21:14:50.464321 ip-10-0-129-99 kubenswrapper[2581]: E0909 21:14:50.464301    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"32d400d933934bb9d996b407d90c361aaf223a425a1dbf43bee1ff4c08a6060e\": container with ID starting with 32d400d933934bb9d996b407d90c361aaf223a425a1dbf43bee1ff4c08a6060e not found: ID does not exist" containerID="32d400d933934bb9d996b407d90c361aaf223a425a1dbf43bee1ff4c08a6060e"
Sep 09 21:14:50.464375 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:50.464330    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32d400d933934bb9d996b407d90c361aaf223a425a1dbf43bee1ff4c08a6060e"} err="failed to get container status \"32d400d933934bb9d996b407d90c361aaf223a425a1dbf43bee1ff4c08a6060e\": rpc error: code = NotFound desc = could not find container \"32d400d933934bb9d996b407d90c361aaf223a425a1dbf43bee1ff4c08a6060e\": container with ID starting with 32d400d933934bb9d996b407d90c361aaf223a425a1dbf43bee1ff4c08a6060e not found: ID does not exist"
Sep 09 21:14:50.464375 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:50.464346    2581 scope.go:117] "RemoveContainer" containerID="2c200817b54216bc80176a3fcfb0c935107cedc9e83f31a50af58f591273f9b9"
Sep 09 21:14:50.464602 ip-10-0-129-99 kubenswrapper[2581]: E0909 21:14:50.464587    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c200817b54216bc80176a3fcfb0c935107cedc9e83f31a50af58f591273f9b9\": container with ID starting with 2c200817b54216bc80176a3fcfb0c935107cedc9e83f31a50af58f591273f9b9 not found: ID does not exist" containerID="2c200817b54216bc80176a3fcfb0c935107cedc9e83f31a50af58f591273f9b9"
Sep 09 21:14:50.464660 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:50.464605    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c200817b54216bc80176a3fcfb0c935107cedc9e83f31a50af58f591273f9b9"} err="failed to get container status \"2c200817b54216bc80176a3fcfb0c935107cedc9e83f31a50af58f591273f9b9\": rpc error: code = NotFound desc = could not find container \"2c200817b54216bc80176a3fcfb0c935107cedc9e83f31a50af58f591273f9b9\": container with ID starting with 2c200817b54216bc80176a3fcfb0c935107cedc9e83f31a50af58f591273f9b9 not found: ID does not exist"
Sep 09 21:14:50.471909 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:50.471882    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96"]
Sep 09 21:14:50.480941 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:50.480921    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-5fc36f-openshift-default-7bc546f5c8-kjx96"]
Sep 09 21:14:50.820549 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:14:50.820517    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7800a4e-d6a6-44d8-997c-e320cb487a99" path="/var/lib/kubelet/pods/d7800a4e-d6a6-44d8-997c-e320cb487a99/volumes"
Sep 09 21:15:00.173031 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:00.172995    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-615c6f-29816475-2tjtd"]
Sep 09 21:15:00.173501 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:00.173484    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7800a4e-d6a6-44d8-997c-e320cb487a99" containerName="istio-proxy"
Sep 09 21:15:00.173553 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:00.173503    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7800a4e-d6a6-44d8-997c-e320cb487a99" containerName="istio-proxy"
Sep 09 21:15:00.173553 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:00.173525    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7800a4e-d6a6-44d8-997c-e320cb487a99" containerName="istio-proxy"
Sep 09 21:15:00.173553 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:00.173530    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7800a4e-d6a6-44d8-997c-e320cb487a99" containerName="istio-proxy"
Sep 09 21:15:00.173553 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:00.173539    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43507456-7508-4b3c-bd42-929e8d63dd06" containerName="revoke-keys"
Sep 09 21:15:00.173553 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:00.173545    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="43507456-7508-4b3c-bd42-929e8d63dd06" containerName="revoke-keys"
Sep 09 21:15:00.173709 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:00.173609    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="43507456-7508-4b3c-bd42-929e8d63dd06" containerName="revoke-keys"
Sep 09 21:15:00.173709 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:00.173618    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7800a4e-d6a6-44d8-997c-e320cb487a99" containerName="istio-proxy"
Sep 09 21:15:00.178339 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:00.178320    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-615c6f-29816475-2tjtd"
Sep 09 21:15:00.182412 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:00.182386    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-glz44\""
Sep 09 21:15:00.189516 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:00.189495    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-615c6f-29816475-2tjtd"
Sep 09 21:15:00.195316 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:00.195288    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-615c6f-29816475-2tjtd"]
Sep 09 21:15:00.199488 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:00.199461    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-7d51a3-29816475-6f76k"]
Sep 09 21:15:00.200793 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:00.200259    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7800a4e-d6a6-44d8-997c-e320cb487a99" containerName="istio-proxy"
Sep 09 21:15:00.204329 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:00.204300    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-7d51a3-29816475-6f76k"
Sep 09 21:15:00.230328 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:00.228123    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29816475-njf6n"]
Sep 09 21:15:00.232540 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:00.232507    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816475-njf6n"
Sep 09 21:15:00.235818 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:00.235788    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-7d51a3-29816475-6f76k"]
Sep 09 21:15:00.244285 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:00.244259    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29816475-njf6n"]
Sep 09 21:15:00.245322 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:00.245239    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-7d51a3-29816475-6f76k"
Sep 09 21:15:00.252605 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:00.252537    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816475-njf6n"
Sep 09 21:15:00.480640 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:00.480569    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-615c6f-29816475-2tjtd"]
Sep 09 21:15:00.484429 ip-10-0-129-99 kubenswrapper[2581]: W0909 21:15:00.484405    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7057f1ed_ef89_44c7_bf96_ba43ceb16d5b.slice/crio-f69a780a71d4c24981dc5a48c24d34a0b917a55dbe207823454aaad719d3e5ff WatchSource:0}: Error finding container f69a780a71d4c24981dc5a48c24d34a0b917a55dbe207823454aaad719d3e5ff: Status 404 returned error can't find the container with id f69a780a71d4c24981dc5a48c24d34a0b917a55dbe207823454aaad719d3e5ff
Sep 09 21:15:00.496713 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:00.496682    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-615c6f-29816475-2tjtd" event={"ID":"7057f1ed-ef89-44c7-bf96-ba43ceb16d5b","Type":"ContainerStarted","Data":"f69a780a71d4c24981dc5a48c24d34a0b917a55dbe207823454aaad719d3e5ff"}
Sep 09 21:15:00.558453 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:00.558425    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29816475-njf6n"]
Sep 09 21:15:00.563008 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:00.561468    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-7d51a3-29816475-6f76k"]
Sep 09 21:15:00.564532 ip-10-0-129-99 kubenswrapper[2581]: W0909 21:15:00.564502    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode00a36c2_67a4_40e1_8b79_7ae1de88ac08.slice/crio-ae3777c582c5a61e90143866615b54196143a2437e7154da7169f220bb71907b WatchSource:0}: Error finding container ae3777c582c5a61e90143866615b54196143a2437e7154da7169f220bb71907b: Status 404 returned error can't find the container with id ae3777c582c5a61e90143866615b54196143a2437e7154da7169f220bb71907b
Sep 09 21:15:01.504008 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:01.503968    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-7d51a3-29816475-6f76k" event={"ID":"e00a36c2-67a4-40e1-8b79-7ae1de88ac08","Type":"ContainerStarted","Data":"51a35259a4515adf5c10b57fdf3fcc44539514750764f9fc775aed1565cf4f3a"}
Sep 09 21:15:01.504499 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:01.504043    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-7d51a3-29816475-6f76k" event={"ID":"e00a36c2-67a4-40e1-8b79-7ae1de88ac08","Type":"ContainerStarted","Data":"ae3777c582c5a61e90143866615b54196143a2437e7154da7169f220bb71907b"}
Sep 09 21:15:01.505738 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:01.505642    2581 generic.go:356] "Generic (PLEG): container finished" podID="7057f1ed-ef89-44c7-bf96-ba43ceb16d5b" containerID="1af91b1fbca0debe831e92ceaa0ce9521459adce784c2ef5e4c63b351467a370" exitCode=0
Sep 09 21:15:01.505738 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:01.505690    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-615c6f-29816475-2tjtd" event={"ID":"7057f1ed-ef89-44c7-bf96-ba43ceb16d5b","Type":"ContainerDied","Data":"1af91b1fbca0debe831e92ceaa0ce9521459adce784c2ef5e4c63b351467a370"}
Sep 09 21:15:01.507562 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:01.507539    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816475-njf6n" event={"ID":"96646b8f-e903-42b5-96df-a4d350460db2","Type":"ContainerStarted","Data":"7c2e92182f2ee6d8685bf450e78e8d3286c0ada44bbfec12baaaeaf6d70f397f"}
Sep 09 21:15:01.507655 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:01.507579    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816475-njf6n" event={"ID":"96646b8f-e903-42b5-96df-a4d350460db2","Type":"ContainerStarted","Data":"4d54e0eb58a945f832684eadda68ecbc224c92129def711153bd8b266daa6480"}
Sep 09 21:15:01.526870 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:01.526826    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-7d51a3-29816475-6f76k" podStartSLOduration=1.526811559 podStartE2EDuration="1.526811559s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:15:01.526679273 +0000 UTC m=+2063.154307157" watchObservedRunningTime="2026-09-09 21:15:01.526811559 +0000 UTC m=+2063.154439432"
Sep 09 21:15:01.546574 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:01.546528    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816475-njf6n" podStartSLOduration=1.5465101570000002 podStartE2EDuration="1.546510157s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:15:01.546385452 +0000 UTC m=+2063.174013326" watchObservedRunningTime="2026-09-09 21:15:01.546510157 +0000 UTC m=+2063.174138033"
Sep 09 21:15:01.815600 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:01.815569    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-7c0011-649d4b577c-gfzmv"]
Sep 09 21:15:01.815799 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:01.815771    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-7c0011-649d4b577c-gfzmv" podUID="e7471ab4-63da-4678-8437-96c89cca34ba" containerName="payload-processing" containerID="cri-o://89737067a568f963e93adaff28cf0fe72ac4edd17a72f55ef0978c536b8256b1" gracePeriod=30
Sep 09 21:15:02.055916 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:02.055880    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7c0011-f2mm5"]
Sep 09 21:15:02.060160 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:02.060130    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-7c0011-649d4b577c-gfzmv"
Sep 09 21:15:02.064362 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:02.064340    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7c0011-f2mm5"]
Sep 09 21:15:02.197776 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:02.197696    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2qtbs\" (UniqueName: \"kubernetes.io/projected/e7471ab4-63da-4678-8437-96c89cca34ba-kube-api-access-2qtbs\") pod \"e7471ab4-63da-4678-8437-96c89cca34ba\" (UID: \"e7471ab4-63da-4678-8437-96c89cca34ba\") "
Sep 09 21:15:02.197910 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:02.197825    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e7471ab4-63da-4678-8437-96c89cca34ba-plugins-config-volume\") pod \"e7471ab4-63da-4678-8437-96c89cca34ba\" (UID: \"e7471ab4-63da-4678-8437-96c89cca34ba\") "
Sep 09 21:15:02.198223 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:02.198184    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7471ab4-63da-4678-8437-96c89cca34ba-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e7471ab4-63da-4678-8437-96c89cca34ba" (UID: "e7471ab4-63da-4678-8437-96c89cca34ba"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:15:02.199870 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:02.199848    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7471ab4-63da-4678-8437-96c89cca34ba-kube-api-access-2qtbs" (OuterVolumeSpecName: "kube-api-access-2qtbs") pod "e7471ab4-63da-4678-8437-96c89cca34ba" (UID: "e7471ab4-63da-4678-8437-96c89cca34ba"). InnerVolumeSpecName "kube-api-access-2qtbs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:15:02.299380 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:02.299345    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e7471ab4-63da-4678-8437-96c89cca34ba-plugins-config-volume\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:15:02.299380 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:02.299374    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2qtbs\" (UniqueName: \"kubernetes.io/projected/e7471ab4-63da-4678-8437-96c89cca34ba-kube-api-access-2qtbs\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:15:02.514333 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:02.514298    2581 generic.go:356] "Generic (PLEG): container finished" podID="96646b8f-e903-42b5-96df-a4d350460db2" containerID="7c2e92182f2ee6d8685bf450e78e8d3286c0ada44bbfec12baaaeaf6d70f397f" exitCode=0
Sep 09 21:15:02.514739 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:02.514380    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816475-njf6n" event={"ID":"96646b8f-e903-42b5-96df-a4d350460db2","Type":"ContainerDied","Data":"7c2e92182f2ee6d8685bf450e78e8d3286c0ada44bbfec12baaaeaf6d70f397f"}
Sep 09 21:15:02.516037 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:02.516012    2581 generic.go:356] "Generic (PLEG): container finished" podID="e00a36c2-67a4-40e1-8b79-7ae1de88ac08" containerID="51a35259a4515adf5c10b57fdf3fcc44539514750764f9fc775aed1565cf4f3a" exitCode=0
Sep 09 21:15:02.516197 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:02.516043    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-7d51a3-29816475-6f76k" event={"ID":"e00a36c2-67a4-40e1-8b79-7ae1de88ac08","Type":"ContainerDied","Data":"51a35259a4515adf5c10b57fdf3fcc44539514750764f9fc775aed1565cf4f3a"}
Sep 09 21:15:02.517385 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:02.517367    2581 generic.go:356] "Generic (PLEG): container finished" podID="e7471ab4-63da-4678-8437-96c89cca34ba" containerID="89737067a568f963e93adaff28cf0fe72ac4edd17a72f55ef0978c536b8256b1" exitCode=0
Sep 09 21:15:02.517508 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:02.517392    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-7c0011-649d4b577c-gfzmv" event={"ID":"e7471ab4-63da-4678-8437-96c89cca34ba","Type":"ContainerDied","Data":"89737067a568f963e93adaff28cf0fe72ac4edd17a72f55ef0978c536b8256b1"}
Sep 09 21:15:02.517508 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:02.517412    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-7c0011-649d4b577c-gfzmv"
Sep 09 21:15:02.517508 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:02.517444    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-7c0011-649d4b577c-gfzmv" event={"ID":"e7471ab4-63da-4678-8437-96c89cca34ba","Type":"ContainerDied","Data":"2ae30514138f10387378b0724973b32f585d869e5e0cc95fe26401f2662e5ea2"}
Sep 09 21:15:02.517508 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:02.517475    2581 scope.go:117] "RemoveContainer" containerID="89737067a568f963e93adaff28cf0fe72ac4edd17a72f55ef0978c536b8256b1"
Sep 09 21:15:02.527626 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:02.527604    2581 scope.go:117] "RemoveContainer" containerID="89737067a568f963e93adaff28cf0fe72ac4edd17a72f55ef0978c536b8256b1"
Sep 09 21:15:02.527903 ip-10-0-129-99 kubenswrapper[2581]: E0909 21:15:02.527883    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89737067a568f963e93adaff28cf0fe72ac4edd17a72f55ef0978c536b8256b1\": container with ID starting with 89737067a568f963e93adaff28cf0fe72ac4edd17a72f55ef0978c536b8256b1 not found: ID does not exist" containerID="89737067a568f963e93adaff28cf0fe72ac4edd17a72f55ef0978c536b8256b1"
Sep 09 21:15:02.527945 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:02.527912    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89737067a568f963e93adaff28cf0fe72ac4edd17a72f55ef0978c536b8256b1"} err="failed to get container status \"89737067a568f963e93adaff28cf0fe72ac4edd17a72f55ef0978c536b8256b1\": rpc error: code = NotFound desc = could not find container \"89737067a568f963e93adaff28cf0fe72ac4edd17a72f55ef0978c536b8256b1\": container with ID starting with 89737067a568f963e93adaff28cf0fe72ac4edd17a72f55ef0978c536b8256b1 not found: ID does not exist"
Sep 09 21:15:02.555417 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:02.555389    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-7c0011-649d4b577c-gfzmv"]
Sep 09 21:15:02.558504 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:02.558477    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-7c0011-649d4b577c-gfzmv"]
Sep 09 21:15:02.660764 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:02.660739    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-615c6f-29816475-2tjtd"
Sep 09 21:15:02.819828 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:02.819751    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43507456-7508-4b3c-bd42-929e8d63dd06" path="/var/lib/kubelet/pods/43507456-7508-4b3c-bd42-929e8d63dd06/volumes"
Sep 09 21:15:02.820157 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:02.820144    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7471ab4-63da-4678-8437-96c89cca34ba" path="/var/lib/kubelet/pods/e7471ab4-63da-4678-8437-96c89cca34ba/volumes"
Sep 09 21:15:03.523978 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:03.523945    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-615c6f-29816475-2tjtd"
Sep 09 21:15:03.524496 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:03.523950    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-615c6f-29816475-2tjtd" event={"ID":"7057f1ed-ef89-44c7-bf96-ba43ceb16d5b","Type":"ContainerDied","Data":"f69a780a71d4c24981dc5a48c24d34a0b917a55dbe207823454aaad719d3e5ff"}
Sep 09 21:15:03.524496 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:03.524086    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f69a780a71d4c24981dc5a48c24d34a0b917a55dbe207823454aaad719d3e5ff"
Sep 09 21:15:03.656391 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:03.656369    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-7d51a3-29816475-6f76k"
Sep 09 21:15:03.696572 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:03.696545    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816475-njf6n"
Sep 09 21:15:04.531418 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:04.531386    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-7d51a3-29816475-6f76k"
Sep 09 21:15:04.531835 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:04.531431    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-7d51a3-29816475-6f76k" event={"ID":"e00a36c2-67a4-40e1-8b79-7ae1de88ac08","Type":"ContainerDied","Data":"ae3777c582c5a61e90143866615b54196143a2437e7154da7169f220bb71907b"}
Sep 09 21:15:04.531835 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:04.531486    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ae3777c582c5a61e90143866615b54196143a2437e7154da7169f220bb71907b"
Sep 09 21:15:04.532892 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:04.532860    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816475-njf6n" event={"ID":"96646b8f-e903-42b5-96df-a4d350460db2","Type":"ContainerDied","Data":"4d54e0eb58a945f832684eadda68ecbc224c92129def711153bd8b266daa6480"}
Sep 09 21:15:04.532987 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:04.532907    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4d54e0eb58a945f832684eadda68ecbc224c92129def711153bd8b266daa6480"
Sep 09 21:15:04.532987 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:04.532878    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816475-njf6n"
Sep 09 21:15:38.982565 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:38.982538    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-wsv4v_2b1f9a6e-7a79-4e89-ac3f-4658d6d91550/console-operator/1.log"
Sep 09 21:15:38.987901 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:38.987876    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-wsv4v_2b1f9a6e-7a79-4e89-ac3f-4658d6d91550/console-operator/1.log"
Sep 09 21:15:38.989443 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:38.989423    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-75wj4_55b4fbae-29dc-41bf-a907-b1412b852cef/ovn-acl-logging/0.log"
Sep 09 21:15:38.994609 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:38.994588    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-75wj4_55b4fbae-29dc-41bf-a907-b1412b852cef/ovn-acl-logging/0.log"
Sep 09 21:15:39.434715 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:39.434631    2581 scope.go:117] "RemoveContainer" containerID="1cb1f6c6bd56f9927ca9865c000a44b4770cfb6e3c73c493d3ff557a8e6d7c70"
Sep 09 21:15:57.320975 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:57.320941    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr"]
Sep 09 21:15:57.321523 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:57.321207    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr" podUID="eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0" containerName="istio-proxy" containerID="cri-o://f291ac9faa1cd3e528c8d5bbd5756e755e3de57718e3df2e5da73f7ae37404a3" gracePeriod=30
Sep 09 21:15:57.730475 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:57.730392    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7d51a3-ckvwg"]
Sep 09 21:15:57.730834 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:57.730821    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e00a36c2-67a4-40e1-8b79-7ae1de88ac08" containerName="cleanup"
Sep 09 21:15:57.730877 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:57.730835    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e00a36c2-67a4-40e1-8b79-7ae1de88ac08" containerName="cleanup"
Sep 09 21:15:57.730877 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:57.730853    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7471ab4-63da-4678-8437-96c89cca34ba" containerName="payload-processing"
Sep 09 21:15:57.730877 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:57.730859    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7471ab4-63da-4678-8437-96c89cca34ba" containerName="payload-processing"
Sep 09 21:15:57.730877 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:57.730867    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96646b8f-e903-42b5-96df-a4d350460db2" containerName="cleanup"
Sep 09 21:15:57.730877 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:57.730872    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="96646b8f-e903-42b5-96df-a4d350460db2" containerName="cleanup"
Sep 09 21:15:57.731032 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:57.730880    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7057f1ed-ef89-44c7-bf96-ba43ceb16d5b" containerName="cleanup"
Sep 09 21:15:57.731032 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:57.730885    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="7057f1ed-ef89-44c7-bf96-ba43ceb16d5b" containerName="cleanup"
Sep 09 21:15:57.731032 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:57.730937    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e00a36c2-67a4-40e1-8b79-7ae1de88ac08" containerName="cleanup"
Sep 09 21:15:57.731032 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:57.730945    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="7057f1ed-ef89-44c7-bf96-ba43ceb16d5b" containerName="cleanup"
Sep 09 21:15:57.731032 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:57.730955    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7471ab4-63da-4678-8437-96c89cca34ba" containerName="payload-processing"
Sep 09 21:15:57.731032 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:57.730961    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="96646b8f-e903-42b5-96df-a4d350460db2" containerName="cleanup"
Sep 09 21:15:57.734285 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:57.734268    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7d51a3-ckvwg"
Sep 09 21:15:57.744337 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:57.744320    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-glz44\""
Sep 09 21:15:57.828404 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:57.828369    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7d51a3-ckvwg"]
Sep 09 21:15:57.830158 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:57.830126    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e13b2d6-6593-4a23-bca9-f58f3a5e45be-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-7d51a3-ckvwg\" (UID: \"0e13b2d6-6593-4a23-bca9-f58f3a5e45be\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7d51a3-ckvwg"
Sep 09 21:15:57.931335 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:57.931293    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e13b2d6-6593-4a23-bca9-f58f3a5e45be-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-7d51a3-ckvwg\" (UID: \"0e13b2d6-6593-4a23-bca9-f58f3a5e45be\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7d51a3-ckvwg"
Sep 09 21:15:57.931961 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:57.931935    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e13b2d6-6593-4a23-bca9-f58f3a5e45be-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-7d51a3-ckvwg\" (UID: \"0e13b2d6-6593-4a23-bca9-f58f3a5e45be\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7d51a3-ckvwg"
Sep 09 21:15:58.044056 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:58.044025    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7d51a3-ckvwg"
Sep 09 21:15:58.334392 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:58.334365    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7d51a3-ckvwg"]
Sep 09 21:15:58.335076 ip-10-0-129-99 kubenswrapper[2581]: W0909 21:15:58.335055    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e13b2d6_6593_4a23_bca9_f58f3a5e45be.slice/crio-4adc1e3d45d56342cbe50a35517b23ccc1a8423dcb8a9c5a73bb0560c5338f92 WatchSource:0}: Error finding container 4adc1e3d45d56342cbe50a35517b23ccc1a8423dcb8a9c5a73bb0560c5338f92: Status 404 returned error can't find the container with id 4adc1e3d45d56342cbe50a35517b23ccc1a8423dcb8a9c5a73bb0560c5338f92
Sep 09 21:15:58.825634 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:58.825600    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7d51a3-ckvwg" event={"ID":"0e13b2d6-6593-4a23-bca9-f58f3a5e45be","Type":"ContainerStarted","Data":"b0a6d6a10430d37d595d9a03a418c498d9201a712b3247395c532754683600de"}
Sep 09 21:15:58.825826 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:58.825655    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7d51a3-ckvwg" event={"ID":"0e13b2d6-6593-4a23-bca9-f58f3a5e45be","Type":"ContainerStarted","Data":"4adc1e3d45d56342cbe50a35517b23ccc1a8423dcb8a9c5a73bb0560c5338f92"}
Sep 09 21:15:58.860547 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:58.860505    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7d51a3-ckvwg" podStartSLOduration=1.8604902939999999 podStartE2EDuration="1.860490294s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:15:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:15:58.859262543 +0000 UTC m=+2120.486890418" watchObservedRunningTime="2026-09-09 21:15:58.860490294 +0000 UTC m=+2120.488118169"
Sep 09 21:15:59.832203 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:59.832128    2581 generic.go:356] "Generic (PLEG): container finished" podID="0e13b2d6-6593-4a23-bca9-f58f3a5e45be" containerID="b0a6d6a10430d37d595d9a03a418c498d9201a712b3247395c532754683600de" exitCode=0
Sep 09 21:15:59.832203 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:15:59.832169    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7d51a3-ckvwg" event={"ID":"0e13b2d6-6593-4a23-bca9-f58f3a5e45be","Type":"ContainerDied","Data":"b0a6d6a10430d37d595d9a03a418c498d9201a712b3247395c532754683600de"}
Sep 09 21:16:00.970729 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:00.970698    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7d51a3-ckvwg"
Sep 09 21:16:01.061035 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:01.061000    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e13b2d6-6593-4a23-bca9-f58f3a5e45be-maas-api-service-ca\") pod \"0e13b2d6-6593-4a23-bca9-f58f3a5e45be\" (UID: \"0e13b2d6-6593-4a23-bca9-f58f3a5e45be\") "
Sep 09 21:16:01.061452 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:01.061433    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e13b2d6-6593-4a23-bca9-f58f3a5e45be-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0e13b2d6-6593-4a23-bca9-f58f3a5e45be" (UID: "0e13b2d6-6593-4a23-bca9-f58f3a5e45be"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:16:01.162833 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:01.162733    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e13b2d6-6593-4a23-bca9-f58f3a5e45be-maas-api-service-ca\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:16:01.770467 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:01.770432    2581 patch_prober.go:28] interesting pod/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 09 21:16:01.770642 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:01.770489    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr" podUID="eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:16:01.842400 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:01.842370    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7d51a3-ckvwg"
Sep 09 21:16:01.842588 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:01.842404    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7d51a3-ckvwg" event={"ID":"0e13b2d6-6593-4a23-bca9-f58f3a5e45be","Type":"ContainerDied","Data":"4adc1e3d45d56342cbe50a35517b23ccc1a8423dcb8a9c5a73bb0560c5338f92"}
Sep 09 21:16:01.842588 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:01.842447    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4adc1e3d45d56342cbe50a35517b23ccc1a8423dcb8a9c5a73bb0560c5338f92"
Sep 09 21:16:02.571443 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:02.571416    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr"
Sep 09 21:16:02.676797 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:02.676719    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0-istiod-ca-cert\") pod \"eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0\" (UID: \"eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0\") "
Sep 09 21:16:02.676797 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:02.676779    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0-credential-socket\") pod \"eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0\" (UID: \"eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0\") "
Sep 09 21:16:02.676979 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:02.676811    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0-istio-envoy\") pod \"eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0\" (UID: \"eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0\") "
Sep 09 21:16:02.676979 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:02.676893    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-25jvz\" (UniqueName: \"kubernetes.io/projected/eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0-kube-api-access-25jvz\") pod \"eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0\" (UID: \"eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0\") "
Sep 09 21:16:02.676979 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:02.676924    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0-istio-podinfo\") pod \"eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0\" (UID: \"eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0\") "
Sep 09 21:16:02.676979 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:02.676971    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0-workload-certs\") pod \"eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0\" (UID: \"eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0\") "
Sep 09 21:16:02.677206 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:02.677003    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0-istio-data\") pod \"eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0\" (UID: \"eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0\") "
Sep 09 21:16:02.677206 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:02.677042    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0-workload-socket\") pod \"eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0\" (UID: \"eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0\") "
Sep 09 21:16:02.677206 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:02.677148    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0" (UID: "eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:16:02.677374 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:02.677239    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0" (UID: "eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:16:02.677374 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:02.677294    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0-istio-token\") pod \"eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0\" (UID: \"eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0\") "
Sep 09 21:16:02.677374 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:02.677295    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0" (UID: "eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:16:02.677544 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:02.677460    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0" (UID: "eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:16:02.677601 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:02.677541    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0-istio-data" (OuterVolumeSpecName: "istio-data") pod "eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0" (UID: "eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:16:02.677685 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:02.677665    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0-workload-certs\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:16:02.677750 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:02.677690    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0-istio-data\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:16:02.677750 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:02.677703    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0-workload-socket\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:16:02.677750 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:02.677715    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0-istiod-ca-cert\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:16:02.677750 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:02.677729    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0-credential-socket\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:16:02.679216 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:02.679188    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0" (UID: "eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:16:02.679216 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:02.679199    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0" (UID: "eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 21:16:02.679216 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:02.679209    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0-kube-api-access-25jvz" (OuterVolumeSpecName: "kube-api-access-25jvz") pod "eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0" (UID: "eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0"). InnerVolumeSpecName "kube-api-access-25jvz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:16:02.679414 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:02.679253    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0-istio-token" (OuterVolumeSpecName: "istio-token") pod "eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0" (UID: "eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:16:02.778902 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:02.778876    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0-istio-envoy\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:16:02.778902 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:02.778900    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-25jvz\" (UniqueName: \"kubernetes.io/projected/eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0-kube-api-access-25jvz\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:16:02.779046 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:02.778915    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0-istio-podinfo\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:16:02.779046 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:02.778928    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0-istio-token\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:16:02.849202 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:02.849176    2581 generic.go:356] "Generic (PLEG): container finished" podID="eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0" containerID="f291ac9faa1cd3e528c8d5bbd5756e755e3de57718e3df2e5da73f7ae37404a3" exitCode=0
Sep 09 21:16:02.849319 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:02.849225    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr"
Sep 09 21:16:02.849319 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:02.849256    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr" event={"ID":"eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0","Type":"ContainerDied","Data":"f291ac9faa1cd3e528c8d5bbd5756e755e3de57718e3df2e5da73f7ae37404a3"}
Sep 09 21:16:02.849319 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:02.849311    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr" event={"ID":"eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0","Type":"ContainerDied","Data":"98552c6ca6d0f5a44b904608ff8ff49c06840b9b29bb4c1951f039d0c86df50e"}
Sep 09 21:16:02.849483 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:02.849334    2581 scope.go:117] "RemoveContainer" containerID="f291ac9faa1cd3e528c8d5bbd5756e755e3de57718e3df2e5da73f7ae37404a3"
Sep 09 21:16:02.858095 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:02.858073    2581 scope.go:117] "RemoveContainer" containerID="86142652d8985c35e8c79266141007dcdfd00a08af81127e77edf5e29f516fd6"
Sep 09 21:16:02.865770 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:02.865753    2581 scope.go:117] "RemoveContainer" containerID="f291ac9faa1cd3e528c8d5bbd5756e755e3de57718e3df2e5da73f7ae37404a3"
Sep 09 21:16:02.866037 ip-10-0-129-99 kubenswrapper[2581]: E0909 21:16:02.866018    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f291ac9faa1cd3e528c8d5bbd5756e755e3de57718e3df2e5da73f7ae37404a3\": container with ID starting with f291ac9faa1cd3e528c8d5bbd5756e755e3de57718e3df2e5da73f7ae37404a3 not found: ID does not exist" containerID="f291ac9faa1cd3e528c8d5bbd5756e755e3de57718e3df2e5da73f7ae37404a3"
Sep 09 21:16:02.866175 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:02.866045    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f291ac9faa1cd3e528c8d5bbd5756e755e3de57718e3df2e5da73f7ae37404a3"} err="failed to get container status \"f291ac9faa1cd3e528c8d5bbd5756e755e3de57718e3df2e5da73f7ae37404a3\": rpc error: code = NotFound desc = could not find container \"f291ac9faa1cd3e528c8d5bbd5756e755e3de57718e3df2e5da73f7ae37404a3\": container with ID starting with f291ac9faa1cd3e528c8d5bbd5756e755e3de57718e3df2e5da73f7ae37404a3 not found: ID does not exist"
Sep 09 21:16:02.866175 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:02.866072    2581 scope.go:117] "RemoveContainer" containerID="86142652d8985c35e8c79266141007dcdfd00a08af81127e77edf5e29f516fd6"
Sep 09 21:16:02.866391 ip-10-0-129-99 kubenswrapper[2581]: E0909 21:16:02.866319    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86142652d8985c35e8c79266141007dcdfd00a08af81127e77edf5e29f516fd6\": container with ID starting with 86142652d8985c35e8c79266141007dcdfd00a08af81127e77edf5e29f516fd6 not found: ID does not exist" containerID="86142652d8985c35e8c79266141007dcdfd00a08af81127e77edf5e29f516fd6"
Sep 09 21:16:02.866391 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:02.866339    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86142652d8985c35e8c79266141007dcdfd00a08af81127e77edf5e29f516fd6"} err="failed to get container status \"86142652d8985c35e8c79266141007dcdfd00a08af81127e77edf5e29f516fd6\": rpc error: code = NotFound desc = could not find container \"86142652d8985c35e8c79266141007dcdfd00a08af81127e77edf5e29f516fd6\": container with ID starting with 86142652d8985c35e8c79266141007dcdfd00a08af81127e77edf5e29f516fd6 not found: ID does not exist"
Sep 09 21:16:02.873056 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:02.873034    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr"]
Sep 09 21:16:02.874970 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:02.874950    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-7c0011-openshift-default-78695b9686-4nvfr"]
Sep 09 21:16:04.821756 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:04.821723    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0" path="/var/lib/kubelet/pods/eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0/volumes"
Sep 09 21:16:08.325573 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:08.325537    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-7d51a3-29816475-6f76k"]
Sep 09 21:16:08.333566 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:08.333536    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-7d51a3-29816475-6f76k"]
Sep 09 21:16:08.389312 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:08.389281    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-7d51a3-6b7c7bd57d-4pjvc"]
Sep 09 21:16:08.389666 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:08.389632    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7d51a3-6b7c7bd57d-4pjvc" podUID="b6936836-46e2-4b9b-b33a-f6355c37dfe6" containerName="payload-pre-processing" containerID="cri-o://637183aa6518c6e0522261b98e6aefc139b90be220004e133a9a48a6d810ceb7" gracePeriod=30
Sep 09 21:16:08.664590 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:08.664563    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7d51a3-ckvwg"]
Sep 09 21:16:08.664781 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:08.664764    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7d51a3-6b7c7bd57d-4pjvc"
Sep 09 21:16:08.670191 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:08.670169    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7d51a3-ckvwg"]
Sep 09 21:16:08.738422 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:08.738391    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6936836-46e2-4b9b-b33a-f6355c37dfe6-plugins-config-volume\") pod \"b6936836-46e2-4b9b-b33a-f6355c37dfe6\" (UID: \"b6936836-46e2-4b9b-b33a-f6355c37dfe6\") "
Sep 09 21:16:08.738592 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:08.738527    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p7z5l\" (UniqueName: \"kubernetes.io/projected/b6936836-46e2-4b9b-b33a-f6355c37dfe6-kube-api-access-p7z5l\") pod \"b6936836-46e2-4b9b-b33a-f6355c37dfe6\" (UID: \"b6936836-46e2-4b9b-b33a-f6355c37dfe6\") "
Sep 09 21:16:08.738771 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:08.738749    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6936836-46e2-4b9b-b33a-f6355c37dfe6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b6936836-46e2-4b9b-b33a-f6355c37dfe6" (UID: "b6936836-46e2-4b9b-b33a-f6355c37dfe6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:16:08.740531 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:08.740506    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6936836-46e2-4b9b-b33a-f6355c37dfe6-kube-api-access-p7z5l" (OuterVolumeSpecName: "kube-api-access-p7z5l") pod "b6936836-46e2-4b9b-b33a-f6355c37dfe6" (UID: "b6936836-46e2-4b9b-b33a-f6355c37dfe6"). InnerVolumeSpecName "kube-api-access-p7z5l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:16:08.820720 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:08.820685    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e13b2d6-6593-4a23-bca9-f58f3a5e45be" path="/var/lib/kubelet/pods/0e13b2d6-6593-4a23-bca9-f58f3a5e45be/volumes"
Sep 09 21:16:08.821018 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:08.821004    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e00a36c2-67a4-40e1-8b79-7ae1de88ac08" path="/var/lib/kubelet/pods/e00a36c2-67a4-40e1-8b79-7ae1de88ac08/volumes"
Sep 09 21:16:08.839300 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:08.839274    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p7z5l\" (UniqueName: \"kubernetes.io/projected/b6936836-46e2-4b9b-b33a-f6355c37dfe6-kube-api-access-p7z5l\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:16:08.839300 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:08.839298    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6936836-46e2-4b9b-b33a-f6355c37dfe6-plugins-config-volume\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:16:08.882603 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:08.882567    2581 generic.go:356] "Generic (PLEG): container finished" podID="b6936836-46e2-4b9b-b33a-f6355c37dfe6" containerID="637183aa6518c6e0522261b98e6aefc139b90be220004e133a9a48a6d810ceb7" exitCode=0
Sep 09 21:16:08.882775 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:08.882612    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7d51a3-6b7c7bd57d-4pjvc"
Sep 09 21:16:08.882775 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:08.882645    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7d51a3-6b7c7bd57d-4pjvc" event={"ID":"b6936836-46e2-4b9b-b33a-f6355c37dfe6","Type":"ContainerDied","Data":"637183aa6518c6e0522261b98e6aefc139b90be220004e133a9a48a6d810ceb7"}
Sep 09 21:16:08.882775 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:08.882696    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7d51a3-6b7c7bd57d-4pjvc" event={"ID":"b6936836-46e2-4b9b-b33a-f6355c37dfe6","Type":"ContainerDied","Data":"e411e6192b0bc146fb455379db4a1a920c2d5f2a65cabc6997415916a43aa42d"}
Sep 09 21:16:08.882775 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:08.882718    2581 scope.go:117] "RemoveContainer" containerID="637183aa6518c6e0522261b98e6aefc139b90be220004e133a9a48a6d810ceb7"
Sep 09 21:16:08.891776 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:08.891759    2581 scope.go:117] "RemoveContainer" containerID="637183aa6518c6e0522261b98e6aefc139b90be220004e133a9a48a6d810ceb7"
Sep 09 21:16:08.892031 ip-10-0-129-99 kubenswrapper[2581]: E0909 21:16:08.892014    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"637183aa6518c6e0522261b98e6aefc139b90be220004e133a9a48a6d810ceb7\": container with ID starting with 637183aa6518c6e0522261b98e6aefc139b90be220004e133a9a48a6d810ceb7 not found: ID does not exist" containerID="637183aa6518c6e0522261b98e6aefc139b90be220004e133a9a48a6d810ceb7"
Sep 09 21:16:08.892082 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:08.892040    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"637183aa6518c6e0522261b98e6aefc139b90be220004e133a9a48a6d810ceb7"} err="failed to get container status \"637183aa6518c6e0522261b98e6aefc139b90be220004e133a9a48a6d810ceb7\": rpc error: code = NotFound desc = could not find container \"637183aa6518c6e0522261b98e6aefc139b90be220004e133a9a48a6d810ceb7\": container with ID starting with 637183aa6518c6e0522261b98e6aefc139b90be220004e133a9a48a6d810ceb7 not found: ID does not exist"
Sep 09 21:16:08.905623 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:08.905566    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-7d51a3-6b7c7bd57d-4pjvc"]
Sep 09 21:16:08.912476 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:08.912452    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-7d51a3-6b7c7bd57d-4pjvc"]
Sep 09 21:16:10.819849 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:10.819814    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6936836-46e2-4b9b-b33a-f6355c37dfe6" path="/var/lib/kubelet/pods/b6936836-46e2-4b9b-b33a-f6355c37dfe6/volumes"
Sep 09 21:16:15.774746 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:15.774710    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-615c6f-9jls7"]
Sep 09 21:16:15.775155 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:15.775141    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0" containerName="istio-proxy"
Sep 09 21:16:15.775155 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:15.775154    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0" containerName="istio-proxy"
Sep 09 21:16:15.775237 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:15.775164    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0" containerName="istio-proxy"
Sep 09 21:16:15.775237 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:15.775170    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0" containerName="istio-proxy"
Sep 09 21:16:15.775237 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:15.775191    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6936836-46e2-4b9b-b33a-f6355c37dfe6" containerName="payload-pre-processing"
Sep 09 21:16:15.775237 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:15.775197    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6936836-46e2-4b9b-b33a-f6355c37dfe6" containerName="payload-pre-processing"
Sep 09 21:16:15.775237 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:15.775205    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e13b2d6-6593-4a23-bca9-f58f3a5e45be" containerName="revoke-keys"
Sep 09 21:16:15.775237 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:15.775211    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e13b2d6-6593-4a23-bca9-f58f3a5e45be" containerName="revoke-keys"
Sep 09 21:16:15.775431 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:15.775262    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e13b2d6-6593-4a23-bca9-f58f3a5e45be" containerName="revoke-keys"
Sep 09 21:16:15.775431 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:15.775274    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0" containerName="istio-proxy"
Sep 09 21:16:15.775431 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:15.775280    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="eabfbc4f-b560-4ada-a33e-d6b83e3aa8e0" containerName="istio-proxy"
Sep 09 21:16:15.775431 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:15.775286    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6936836-46e2-4b9b-b33a-f6355c37dfe6" containerName="payload-pre-processing"
Sep 09 21:16:15.778459 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:15.778442    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-615c6f-9jls7"
Sep 09 21:16:15.781778 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:15.781760    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-glz44\""
Sep 09 21:16:15.789942 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:15.789919    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-615c6f-9jls7"]
Sep 09 21:16:15.916879 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:15.916853    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/382ea301-8de9-4e83-b1fd-60e9ba8262f9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-615c6f-9jls7\" (UID: \"382ea301-8de9-4e83-b1fd-60e9ba8262f9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-615c6f-9jls7"
Sep 09 21:16:16.017617 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:16.017584    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/382ea301-8de9-4e83-b1fd-60e9ba8262f9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-615c6f-9jls7\" (UID: \"382ea301-8de9-4e83-b1fd-60e9ba8262f9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-615c6f-9jls7"
Sep 09 21:16:16.018247 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:16.018216    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/382ea301-8de9-4e83-b1fd-60e9ba8262f9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-615c6f-9jls7\" (UID: \"382ea301-8de9-4e83-b1fd-60e9ba8262f9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-615c6f-9jls7"
Sep 09 21:16:16.088800 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:16.088718    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-615c6f-9jls7"
Sep 09 21:16:16.374436 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:16.374411    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-615c6f-9jls7"]
Sep 09 21:16:16.376773 ip-10-0-129-99 kubenswrapper[2581]: W0909 21:16:16.376745    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod382ea301_8de9_4e83_b1fd_60e9ba8262f9.slice/crio-ad7c7645e16f4e84b9c663d5d36e5460c979f21bcb77efed5011bc4f3d137438 WatchSource:0}: Error finding container ad7c7645e16f4e84b9c663d5d36e5460c979f21bcb77efed5011bc4f3d137438: Status 404 returned error can't find the container with id ad7c7645e16f4e84b9c663d5d36e5460c979f21bcb77efed5011bc4f3d137438
Sep 09 21:16:16.925080 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:16.925045    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-615c6f-9jls7" event={"ID":"382ea301-8de9-4e83-b1fd-60e9ba8262f9","Type":"ContainerStarted","Data":"0c6de404fd5c0b3785221d308591909f089d836d502ed45b54eb0c9147f6d856"}
Sep 09 21:16:16.925553 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:16.925095    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-615c6f-9jls7" event={"ID":"382ea301-8de9-4e83-b1fd-60e9ba8262f9","Type":"ContainerStarted","Data":"ad7c7645e16f4e84b9c663d5d36e5460c979f21bcb77efed5011bc4f3d137438"}
Sep 09 21:16:16.950436 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:16.950383    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-615c6f-9jls7" podStartSLOduration=1.950368068 podStartE2EDuration="1.950368068s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:16:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:16:16.947753372 +0000 UTC m=+2138.575381247" watchObservedRunningTime="2026-09-09 21:16:16.950368068 +0000 UTC m=+2138.577995941"
Sep 09 21:16:17.931663 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:17.931629    2581 generic.go:356] "Generic (PLEG): container finished" podID="382ea301-8de9-4e83-b1fd-60e9ba8262f9" containerID="0c6de404fd5c0b3785221d308591909f089d836d502ed45b54eb0c9147f6d856" exitCode=0
Sep 09 21:16:17.932083 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:17.931709    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-615c6f-9jls7" event={"ID":"382ea301-8de9-4e83-b1fd-60e9ba8262f9","Type":"ContainerDied","Data":"0c6de404fd5c0b3785221d308591909f089d836d502ed45b54eb0c9147f6d856"}
Sep 09 21:16:19.066940 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:19.066914    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-615c6f-9jls7"
Sep 09 21:16:19.248811 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:19.248779    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/382ea301-8de9-4e83-b1fd-60e9ba8262f9-maas-api-service-ca\") pod \"382ea301-8de9-4e83-b1fd-60e9ba8262f9\" (UID: \"382ea301-8de9-4e83-b1fd-60e9ba8262f9\") "
Sep 09 21:16:19.249157 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:19.249132    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/382ea301-8de9-4e83-b1fd-60e9ba8262f9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "382ea301-8de9-4e83-b1fd-60e9ba8262f9" (UID: "382ea301-8de9-4e83-b1fd-60e9ba8262f9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:16:19.350474 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:19.350437    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/382ea301-8de9-4e83-b1fd-60e9ba8262f9-maas-api-service-ca\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:16:19.943008 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:19.942981    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-615c6f-9jls7"
Sep 09 21:16:19.943204 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:19.942981    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-615c6f-9jls7" event={"ID":"382ea301-8de9-4e83-b1fd-60e9ba8262f9","Type":"ContainerDied","Data":"ad7c7645e16f4e84b9c663d5d36e5460c979f21bcb77efed5011bc4f3d137438"}
Sep 09 21:16:19.943204 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:19.943126    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ad7c7645e16f4e84b9c663d5d36e5460c979f21bcb77efed5011bc4f3d137438"
Sep 09 21:16:26.398939 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:26.398903    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-615c6f-29816475-2tjtd"]
Sep 09 21:16:26.404567 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:26.404533    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-615c6f-29816475-2tjtd"]
Sep 09 21:16:26.441043 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:26.441010    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-615c6f-6d489cd67c-twlch"]
Sep 09 21:16:26.441286 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:26.441259    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-615c6f-6d489cd67c-twlch" podUID="e02cebef-4376-4266-b958-2556fd55e0c3" containerName="payload-processing" containerID="cri-o://4533f96f04c5045c1e05d8777ae7ad4441ef6c39209a0d502a4483062087a7d0" gracePeriod=30
Sep 09 21:16:26.454588 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:26.454547    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-615c6f-778b5c6b5d-qk8b7"]
Sep 09 21:16:26.454996 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:26.454967    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-615c6f-778b5c6b5d-qk8b7" podUID="9cc52881-d895-4ab9-8bd3-49273342b95c" containerName="payload-pre-processing" containerID="cri-o://1c35a6dda8cdfab997341e84ce98760d0fc5930d75b65036cb805bc467d2a356" gracePeriod=30
Sep 09 21:16:26.655194 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:26.655087    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-615c6f-9jls7"]
Sep 09 21:16:26.658611 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:26.658585    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-615c6f-9jls7"]
Sep 09 21:16:26.701797 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:26.701773    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-615c6f-6d489cd67c-twlch"
Sep 09 21:16:26.719981 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:26.719952    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vz8tb\" (UniqueName: \"kubernetes.io/projected/e02cebef-4376-4266-b958-2556fd55e0c3-kube-api-access-vz8tb\") pod \"e02cebef-4376-4266-b958-2556fd55e0c3\" (UID: \"e02cebef-4376-4266-b958-2556fd55e0c3\") "
Sep 09 21:16:26.720141 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:26.719992    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e02cebef-4376-4266-b958-2556fd55e0c3-plugins-config-volume\") pod \"e02cebef-4376-4266-b958-2556fd55e0c3\" (UID: \"e02cebef-4376-4266-b958-2556fd55e0c3\") "
Sep 09 21:16:26.720442 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:26.720417    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e02cebef-4376-4266-b958-2556fd55e0c3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e02cebef-4376-4266-b958-2556fd55e0c3" (UID: "e02cebef-4376-4266-b958-2556fd55e0c3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:16:26.723128 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:26.723083    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e02cebef-4376-4266-b958-2556fd55e0c3-kube-api-access-vz8tb" (OuterVolumeSpecName: "kube-api-access-vz8tb") pod "e02cebef-4376-4266-b958-2556fd55e0c3" (UID: "e02cebef-4376-4266-b958-2556fd55e0c3"). InnerVolumeSpecName "kube-api-access-vz8tb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:16:26.739847 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:26.739817    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-615c6f-778b5c6b5d-qk8b7"
Sep 09 21:16:26.819986 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:26.819954    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="382ea301-8de9-4e83-b1fd-60e9ba8262f9" path="/var/lib/kubelet/pods/382ea301-8de9-4e83-b1fd-60e9ba8262f9/volumes"
Sep 09 21:16:26.820352 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:26.820336    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7057f1ed-ef89-44c7-bf96-ba43ceb16d5b" path="/var/lib/kubelet/pods/7057f1ed-ef89-44c7-bf96-ba43ceb16d5b/volumes"
Sep 09 21:16:26.821055 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:26.821031    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xf7sz\" (UniqueName: \"kubernetes.io/projected/9cc52881-d895-4ab9-8bd3-49273342b95c-kube-api-access-xf7sz\") pod \"9cc52881-d895-4ab9-8bd3-49273342b95c\" (UID: \"9cc52881-d895-4ab9-8bd3-49273342b95c\") "
Sep 09 21:16:26.821213 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:26.821078    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9cc52881-d895-4ab9-8bd3-49273342b95c-plugins-config-volume\") pod \"9cc52881-d895-4ab9-8bd3-49273342b95c\" (UID: \"9cc52881-d895-4ab9-8bd3-49273342b95c\") "
Sep 09 21:16:26.821470 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:26.821448    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vz8tb\" (UniqueName: \"kubernetes.io/projected/e02cebef-4376-4266-b958-2556fd55e0c3-kube-api-access-vz8tb\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:16:26.821574 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:26.821474    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e02cebef-4376-4266-b958-2556fd55e0c3-plugins-config-volume\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:16:26.821638 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:26.821610    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9cc52881-d895-4ab9-8bd3-49273342b95c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9cc52881-d895-4ab9-8bd3-49273342b95c" (UID: "9cc52881-d895-4ab9-8bd3-49273342b95c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:16:26.823498 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:26.823472    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9cc52881-d895-4ab9-8bd3-49273342b95c-kube-api-access-xf7sz" (OuterVolumeSpecName: "kube-api-access-xf7sz") pod "9cc52881-d895-4ab9-8bd3-49273342b95c" (UID: "9cc52881-d895-4ab9-8bd3-49273342b95c"). InnerVolumeSpecName "kube-api-access-xf7sz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:16:26.922262 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:26.922168    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xf7sz\" (UniqueName: \"kubernetes.io/projected/9cc52881-d895-4ab9-8bd3-49273342b95c-kube-api-access-xf7sz\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:16:26.922262 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:26.922198    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9cc52881-d895-4ab9-8bd3-49273342b95c-plugins-config-volume\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:16:26.979740 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:26.979703    2581 generic.go:356] "Generic (PLEG): container finished" podID="e02cebef-4376-4266-b958-2556fd55e0c3" containerID="4533f96f04c5045c1e05d8777ae7ad4441ef6c39209a0d502a4483062087a7d0" exitCode=0
Sep 09 21:16:26.979912 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:26.979752    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-615c6f-6d489cd67c-twlch"
Sep 09 21:16:26.979912 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:26.979795    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-615c6f-6d489cd67c-twlch" event={"ID":"e02cebef-4376-4266-b958-2556fd55e0c3","Type":"ContainerDied","Data":"4533f96f04c5045c1e05d8777ae7ad4441ef6c39209a0d502a4483062087a7d0"}
Sep 09 21:16:26.979912 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:26.979849    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-615c6f-6d489cd67c-twlch" event={"ID":"e02cebef-4376-4266-b958-2556fd55e0c3","Type":"ContainerDied","Data":"1cfd8d2acb79f735b2843f7f7df379277602bd95bda3f8af12a1f8e6901f930f"}
Sep 09 21:16:26.979912 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:26.979880    2581 scope.go:117] "RemoveContainer" containerID="4533f96f04c5045c1e05d8777ae7ad4441ef6c39209a0d502a4483062087a7d0"
Sep 09 21:16:26.981284 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:26.981259    2581 generic.go:356] "Generic (PLEG): container finished" podID="9cc52881-d895-4ab9-8bd3-49273342b95c" containerID="1c35a6dda8cdfab997341e84ce98760d0fc5930d75b65036cb805bc467d2a356" exitCode=0
Sep 09 21:16:26.981404 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:26.981308    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-615c6f-778b5c6b5d-qk8b7"
Sep 09 21:16:26.981404 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:26.981333    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-615c6f-778b5c6b5d-qk8b7" event={"ID":"9cc52881-d895-4ab9-8bd3-49273342b95c","Type":"ContainerDied","Data":"1c35a6dda8cdfab997341e84ce98760d0fc5930d75b65036cb805bc467d2a356"}
Sep 09 21:16:26.981404 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:26.981368    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-615c6f-778b5c6b5d-qk8b7" event={"ID":"9cc52881-d895-4ab9-8bd3-49273342b95c","Type":"ContainerDied","Data":"eb54b0bc9b6046538e40c28d91706d16bcc0167d14d87118a57275c7f4abd296"}
Sep 09 21:16:26.991938 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:26.991917    2581 scope.go:117] "RemoveContainer" containerID="4533f96f04c5045c1e05d8777ae7ad4441ef6c39209a0d502a4483062087a7d0"
Sep 09 21:16:26.992261 ip-10-0-129-99 kubenswrapper[2581]: E0909 21:16:26.992239    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4533f96f04c5045c1e05d8777ae7ad4441ef6c39209a0d502a4483062087a7d0\": container with ID starting with 4533f96f04c5045c1e05d8777ae7ad4441ef6c39209a0d502a4483062087a7d0 not found: ID does not exist" containerID="4533f96f04c5045c1e05d8777ae7ad4441ef6c39209a0d502a4483062087a7d0"
Sep 09 21:16:26.992342 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:26.992270    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4533f96f04c5045c1e05d8777ae7ad4441ef6c39209a0d502a4483062087a7d0"} err="failed to get container status \"4533f96f04c5045c1e05d8777ae7ad4441ef6c39209a0d502a4483062087a7d0\": rpc error: code = NotFound desc = could not find container \"4533f96f04c5045c1e05d8777ae7ad4441ef6c39209a0d502a4483062087a7d0\": container with ID starting with 4533f96f04c5045c1e05d8777ae7ad4441ef6c39209a0d502a4483062087a7d0 not found: ID does not exist"
Sep 09 21:16:26.992342 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:26.992289    2581 scope.go:117] "RemoveContainer" containerID="1c35a6dda8cdfab997341e84ce98760d0fc5930d75b65036cb805bc467d2a356"
Sep 09 21:16:27.001460 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:27.001440    2581 scope.go:117] "RemoveContainer" containerID="1c35a6dda8cdfab997341e84ce98760d0fc5930d75b65036cb805bc467d2a356"
Sep 09 21:16:27.001760 ip-10-0-129-99 kubenswrapper[2581]: E0909 21:16:27.001742    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c35a6dda8cdfab997341e84ce98760d0fc5930d75b65036cb805bc467d2a356\": container with ID starting with 1c35a6dda8cdfab997341e84ce98760d0fc5930d75b65036cb805bc467d2a356 not found: ID does not exist" containerID="1c35a6dda8cdfab997341e84ce98760d0fc5930d75b65036cb805bc467d2a356"
Sep 09 21:16:27.001808 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:27.001769    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c35a6dda8cdfab997341e84ce98760d0fc5930d75b65036cb805bc467d2a356"} err="failed to get container status \"1c35a6dda8cdfab997341e84ce98760d0fc5930d75b65036cb805bc467d2a356\": rpc error: code = NotFound desc = could not find container \"1c35a6dda8cdfab997341e84ce98760d0fc5930d75b65036cb805bc467d2a356\": container with ID starting with 1c35a6dda8cdfab997341e84ce98760d0fc5930d75b65036cb805bc467d2a356 not found: ID does not exist"
Sep 09 21:16:27.001868 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:27.001852    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-615c6f-6d489cd67c-twlch"]
Sep 09 21:16:27.008532 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:27.008511    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-615c6f-6d489cd67c-twlch"]
Sep 09 21:16:27.035798 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:27.035773    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-615c6f-778b5c6b5d-qk8b7"]
Sep 09 21:16:27.047479 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:27.047454    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-615c6f-778b5c6b5d-qk8b7"]
Sep 09 21:16:28.819685 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:28.819652    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9cc52881-d895-4ab9-8bd3-49273342b95c" path="/var/lib/kubelet/pods/9cc52881-d895-4ab9-8bd3-49273342b95c/volumes"
Sep 09 21:16:28.820153 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:28.820006    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e02cebef-4376-4266-b958-2556fd55e0c3" path="/var/lib/kubelet/pods/e02cebef-4376-4266-b958-2556fd55e0c3/volumes"
Sep 09 21:16:34.170918 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:34.170836    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-0b5d4c-qqjr7"]
Sep 09 21:16:34.171330 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:34.171233    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="382ea301-8de9-4e83-b1fd-60e9ba8262f9" containerName="revoke-keys"
Sep 09 21:16:34.171330 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:34.171245    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="382ea301-8de9-4e83-b1fd-60e9ba8262f9" containerName="revoke-keys"
Sep 09 21:16:34.171330 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:34.171265    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9cc52881-d895-4ab9-8bd3-49273342b95c" containerName="payload-pre-processing"
Sep 09 21:16:34.171330 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:34.171270    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="9cc52881-d895-4ab9-8bd3-49273342b95c" containerName="payload-pre-processing"
Sep 09 21:16:34.171330 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:34.171283    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e02cebef-4376-4266-b958-2556fd55e0c3" containerName="payload-processing"
Sep 09 21:16:34.171330 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:34.171289    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e02cebef-4376-4266-b958-2556fd55e0c3" containerName="payload-processing"
Sep 09 21:16:34.171532 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:34.171341    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="382ea301-8de9-4e83-b1fd-60e9ba8262f9" containerName="revoke-keys"
Sep 09 21:16:34.171532 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:34.171351    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="9cc52881-d895-4ab9-8bd3-49273342b95c" containerName="payload-pre-processing"
Sep 09 21:16:34.171532 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:34.171356    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e02cebef-4376-4266-b958-2556fd55e0c3" containerName="payload-processing"
Sep 09 21:16:34.175966 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:34.175947    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-0b5d4c-qqjr7"
Sep 09 21:16:34.179320 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:34.179298    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-glz44\""
Sep 09 21:16:34.184687 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:34.184666    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-0b5d4c-qqjr7"]
Sep 09 21:16:34.294209 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:34.294175    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff98c003-73ee-4281-a285-da66a91dc957-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-0b5d4c-qqjr7\" (UID: \"ff98c003-73ee-4281-a285-da66a91dc957\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-0b5d4c-qqjr7"
Sep 09 21:16:34.395462 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:34.395423    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff98c003-73ee-4281-a285-da66a91dc957-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-0b5d4c-qqjr7\" (UID: \"ff98c003-73ee-4281-a285-da66a91dc957\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-0b5d4c-qqjr7"
Sep 09 21:16:34.396039 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:34.396018    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff98c003-73ee-4281-a285-da66a91dc957-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-0b5d4c-qqjr7\" (UID: \"ff98c003-73ee-4281-a285-da66a91dc957\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-0b5d4c-qqjr7"
Sep 09 21:16:34.486339 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:34.486318    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-0b5d4c-qqjr7"
Sep 09 21:16:34.787822 ip-10-0-129-99 kubenswrapper[2581]: W0909 21:16:34.787788    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff98c003_73ee_4281_a285_da66a91dc957.slice/crio-a2a2790ad9d2ed97f53e28d4a355d07cdae41a3c5a7c632a85f668bec736383a WatchSource:0}: Error finding container a2a2790ad9d2ed97f53e28d4a355d07cdae41a3c5a7c632a85f668bec736383a: Status 404 returned error can't find the container with id a2a2790ad9d2ed97f53e28d4a355d07cdae41a3c5a7c632a85f668bec736383a
Sep 09 21:16:34.789127 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:34.789072    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-0b5d4c-qqjr7"]
Sep 09 21:16:35.026012 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:35.025979    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-0b5d4c-qqjr7" event={"ID":"ff98c003-73ee-4281-a285-da66a91dc957","Type":"ContainerStarted","Data":"a2a2790ad9d2ed97f53e28d4a355d07cdae41a3c5a7c632a85f668bec736383a"}
Sep 09 21:16:36.032759 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:36.032722    2581 generic.go:356] "Generic (PLEG): container finished" podID="ff98c003-73ee-4281-a285-da66a91dc957" containerID="0b242aeb189b01dfe300448ec43a397cab368ba42e7f58bd4b12be6acaded2d2" exitCode=0
Sep 09 21:16:36.033151 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:36.032807    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-0b5d4c-qqjr7" event={"ID":"ff98c003-73ee-4281-a285-da66a91dc957","Type":"ContainerDied","Data":"0b242aeb189b01dfe300448ec43a397cab368ba42e7f58bd4b12be6acaded2d2"}
Sep 09 21:16:37.168779 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:37.168755    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-0b5d4c-qqjr7"
Sep 09 21:16:37.223516 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:37.223485    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff98c003-73ee-4281-a285-da66a91dc957-maas-api-service-ca\") pod \"ff98c003-73ee-4281-a285-da66a91dc957\" (UID: \"ff98c003-73ee-4281-a285-da66a91dc957\") "
Sep 09 21:16:37.223842 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:37.223819    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff98c003-73ee-4281-a285-da66a91dc957-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ff98c003-73ee-4281-a285-da66a91dc957" (UID: "ff98c003-73ee-4281-a285-da66a91dc957"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:16:37.324546 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:37.324467    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff98c003-73ee-4281-a285-da66a91dc957-maas-api-service-ca\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:16:38.045254 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:38.045222    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-0b5d4c-qqjr7"
Sep 09 21:16:38.045254 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:38.045239    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-0b5d4c-qqjr7" event={"ID":"ff98c003-73ee-4281-a285-da66a91dc957","Type":"ContainerDied","Data":"a2a2790ad9d2ed97f53e28d4a355d07cdae41a3c5a7c632a85f668bec736383a"}
Sep 09 21:16:38.045475 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:38.045289    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a2a2790ad9d2ed97f53e28d4a355d07cdae41a3c5a7c632a85f668bec736383a"
Sep 09 21:16:45.026561 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:45.026518    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-0b5d4c-qqjr7"]
Sep 09 21:16:45.028377 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:45.028351    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-0b5d4c-qqjr7"]
Sep 09 21:16:46.822279 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:46.822249    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff98c003-73ee-4281-a285-da66a91dc957" path="/var/lib/kubelet/pods/ff98c003-73ee-4281-a285-da66a91dc957/volumes"
Sep 09 21:16:52.390562 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:52.390521    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm"]
Sep 09 21:16:52.391022 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:52.390770    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm" podUID="cbb20d4d-d468-4972-ac78-1678e69b548b" containerName="istio-proxy" containerID="cri-o://8207eabdd1ce6afe9ecae8cc6e59a88004fcfa397853ff490438a0f74008e0b7" gracePeriod=30
Sep 09 21:16:57.646157 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:57.646133    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm"
Sep 09 21:16:57.816367 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:57.816331    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cbb20d4d-d468-4972-ac78-1678e69b548b-istiod-ca-cert\") pod \"cbb20d4d-d468-4972-ac78-1678e69b548b\" (UID: \"cbb20d4d-d468-4972-ac78-1678e69b548b\") "
Sep 09 21:16:57.816548 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:57.816381    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbb20d4d-d468-4972-ac78-1678e69b548b-workload-socket\") pod \"cbb20d4d-d468-4972-ac78-1678e69b548b\" (UID: \"cbb20d4d-d468-4972-ac78-1678e69b548b\") "
Sep 09 21:16:57.816548 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:57.816420    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cbb20d4d-d468-4972-ac78-1678e69b548b-istio-podinfo\") pod \"cbb20d4d-d468-4972-ac78-1678e69b548b\" (UID: \"cbb20d4d-d468-4972-ac78-1678e69b548b\") "
Sep 09 21:16:57.816548 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:57.816454    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbb20d4d-d468-4972-ac78-1678e69b548b-credential-socket\") pod \"cbb20d4d-d468-4972-ac78-1678e69b548b\" (UID: \"cbb20d4d-d468-4972-ac78-1678e69b548b\") "
Sep 09 21:16:57.816548 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:57.816531    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cbb20d4d-d468-4972-ac78-1678e69b548b-workload-certs\") pod \"cbb20d4d-d468-4972-ac78-1678e69b548b\" (UID: \"cbb20d4d-d468-4972-ac78-1678e69b548b\") "
Sep 09 21:16:57.816730 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:57.816556    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vj2w5\" (UniqueName: \"kubernetes.io/projected/cbb20d4d-d468-4972-ac78-1678e69b548b-kube-api-access-vj2w5\") pod \"cbb20d4d-d468-4972-ac78-1678e69b548b\" (UID: \"cbb20d4d-d468-4972-ac78-1678e69b548b\") "
Sep 09 21:16:57.816730 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:57.816586    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cbb20d4d-d468-4972-ac78-1678e69b548b-istio-data\") pod \"cbb20d4d-d468-4972-ac78-1678e69b548b\" (UID: \"cbb20d4d-d468-4972-ac78-1678e69b548b\") "
Sep 09 21:16:57.816730 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:57.816634    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cbb20d4d-d468-4972-ac78-1678e69b548b-istio-envoy\") pod \"cbb20d4d-d468-4972-ac78-1678e69b548b\" (UID: \"cbb20d4d-d468-4972-ac78-1678e69b548b\") "
Sep 09 21:16:57.816730 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:57.816671    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cbb20d4d-d468-4972-ac78-1678e69b548b-istio-token\") pod \"cbb20d4d-d468-4972-ac78-1678e69b548b\" (UID: \"cbb20d4d-d468-4972-ac78-1678e69b548b\") "
Sep 09 21:16:57.816945 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:57.816774    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbb20d4d-d468-4972-ac78-1678e69b548b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "cbb20d4d-d468-4972-ac78-1678e69b548b" (UID: "cbb20d4d-d468-4972-ac78-1678e69b548b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:16:57.816945 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:57.816775    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cbb20d4d-d468-4972-ac78-1678e69b548b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "cbb20d4d-d468-4972-ac78-1678e69b548b" (UID: "cbb20d4d-d468-4972-ac78-1678e69b548b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:16:57.816945 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:57.816866    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbb20d4d-d468-4972-ac78-1678e69b548b-istio-data" (OuterVolumeSpecName: "istio-data") pod "cbb20d4d-d468-4972-ac78-1678e69b548b" (UID: "cbb20d4d-d468-4972-ac78-1678e69b548b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:16:57.817146 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:57.816934    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbb20d4d-d468-4972-ac78-1678e69b548b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "cbb20d4d-d468-4972-ac78-1678e69b548b" (UID: "cbb20d4d-d468-4972-ac78-1678e69b548b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:16:57.817222 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:57.817137    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbb20d4d-d468-4972-ac78-1678e69b548b-credential-socket\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:16:57.817222 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:57.817154    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbb20d4d-d468-4972-ac78-1678e69b548b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "cbb20d4d-d468-4972-ac78-1678e69b548b" (UID: "cbb20d4d-d468-4972-ac78-1678e69b548b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:16:57.817222 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:57.817163    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cbb20d4d-d468-4972-ac78-1678e69b548b-istio-data\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:16:57.817222 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:57.817210    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cbb20d4d-d468-4972-ac78-1678e69b548b-istiod-ca-cert\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:16:57.817382 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:57.817227    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbb20d4d-d468-4972-ac78-1678e69b548b-workload-socket\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:16:57.818700 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:57.818678    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cbb20d4d-d468-4972-ac78-1678e69b548b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "cbb20d4d-d468-4972-ac78-1678e69b548b" (UID: "cbb20d4d-d468-4972-ac78-1678e69b548b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 21:16:57.818817 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:57.818784    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cbb20d4d-d468-4972-ac78-1678e69b548b-istio-token" (OuterVolumeSpecName: "istio-token") pod "cbb20d4d-d468-4972-ac78-1678e69b548b" (UID: "cbb20d4d-d468-4972-ac78-1678e69b548b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:16:57.819201 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:57.819177    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbb20d4d-d468-4972-ac78-1678e69b548b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "cbb20d4d-d468-4972-ac78-1678e69b548b" (UID: "cbb20d4d-d468-4972-ac78-1678e69b548b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:16:57.819278 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:57.819183    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cbb20d4d-d468-4972-ac78-1678e69b548b-kube-api-access-vj2w5" (OuterVolumeSpecName: "kube-api-access-vj2w5") pod "cbb20d4d-d468-4972-ac78-1678e69b548b" (UID: "cbb20d4d-d468-4972-ac78-1678e69b548b"). InnerVolumeSpecName "kube-api-access-vj2w5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:16:57.918592 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:57.918553    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cbb20d4d-d468-4972-ac78-1678e69b548b-istio-podinfo\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:16:57.918592 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:57.918584    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cbb20d4d-d468-4972-ac78-1678e69b548b-workload-certs\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:16:57.918592 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:57.918596    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vj2w5\" (UniqueName: \"kubernetes.io/projected/cbb20d4d-d468-4972-ac78-1678e69b548b-kube-api-access-vj2w5\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:16:57.918830 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:57.918609    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cbb20d4d-d468-4972-ac78-1678e69b548b-istio-envoy\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:16:57.918830 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:57.918621    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cbb20d4d-d468-4972-ac78-1678e69b548b-istio-token\") on node \"ip-10-0-129-99.ec2.internal\" DevicePath \"\""
Sep 09 21:16:58.149414 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:58.149319    2581 generic.go:356] "Generic (PLEG): container finished" podID="cbb20d4d-d468-4972-ac78-1678e69b548b" containerID="8207eabdd1ce6afe9ecae8cc6e59a88004fcfa397853ff490438a0f74008e0b7" exitCode=0
Sep 09 21:16:58.149414 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:58.149384    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm" event={"ID":"cbb20d4d-d468-4972-ac78-1678e69b548b","Type":"ContainerDied","Data":"8207eabdd1ce6afe9ecae8cc6e59a88004fcfa397853ff490438a0f74008e0b7"}
Sep 09 21:16:58.149634 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:58.149436    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm" event={"ID":"cbb20d4d-d468-4972-ac78-1678e69b548b","Type":"ContainerDied","Data":"a1fd198886ffa5f7ff4df117b30542f1b5240cc9ecc8867e3c384073a51a64dc"}
Sep 09 21:16:58.149634 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:58.149388    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm"
Sep 09 21:16:58.149634 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:58.149463    2581 scope.go:117] "RemoveContainer" containerID="8207eabdd1ce6afe9ecae8cc6e59a88004fcfa397853ff490438a0f74008e0b7"
Sep 09 21:16:58.158563 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:58.158544    2581 scope.go:117] "RemoveContainer" containerID="79fb0b9def7e823767e221081fbd8459b5d4ff1c75a1a06f0007ccaaf91d766a"
Sep 09 21:16:58.166428 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:58.166410    2581 scope.go:117] "RemoveContainer" containerID="8207eabdd1ce6afe9ecae8cc6e59a88004fcfa397853ff490438a0f74008e0b7"
Sep 09 21:16:58.166695 ip-10-0-129-99 kubenswrapper[2581]: E0909 21:16:58.166665    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8207eabdd1ce6afe9ecae8cc6e59a88004fcfa397853ff490438a0f74008e0b7\": container with ID starting with 8207eabdd1ce6afe9ecae8cc6e59a88004fcfa397853ff490438a0f74008e0b7 not found: ID does not exist" containerID="8207eabdd1ce6afe9ecae8cc6e59a88004fcfa397853ff490438a0f74008e0b7"
Sep 09 21:16:58.166745 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:58.166691    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8207eabdd1ce6afe9ecae8cc6e59a88004fcfa397853ff490438a0f74008e0b7"} err="failed to get container status \"8207eabdd1ce6afe9ecae8cc6e59a88004fcfa397853ff490438a0f74008e0b7\": rpc error: code = NotFound desc = could not find container \"8207eabdd1ce6afe9ecae8cc6e59a88004fcfa397853ff490438a0f74008e0b7\": container with ID starting with 8207eabdd1ce6afe9ecae8cc6e59a88004fcfa397853ff490438a0f74008e0b7 not found: ID does not exist"
Sep 09 21:16:58.166745 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:58.166706    2581 scope.go:117] "RemoveContainer" containerID="79fb0b9def7e823767e221081fbd8459b5d4ff1c75a1a06f0007ccaaf91d766a"
Sep 09 21:16:58.166920 ip-10-0-129-99 kubenswrapper[2581]: E0909 21:16:58.166904    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79fb0b9def7e823767e221081fbd8459b5d4ff1c75a1a06f0007ccaaf91d766a\": container with ID starting with 79fb0b9def7e823767e221081fbd8459b5d4ff1c75a1a06f0007ccaaf91d766a not found: ID does not exist" containerID="79fb0b9def7e823767e221081fbd8459b5d4ff1c75a1a06f0007ccaaf91d766a"
Sep 09 21:16:58.166959 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:58.166926    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79fb0b9def7e823767e221081fbd8459b5d4ff1c75a1a06f0007ccaaf91d766a"} err="failed to get container status \"79fb0b9def7e823767e221081fbd8459b5d4ff1c75a1a06f0007ccaaf91d766a\": rpc error: code = NotFound desc = could not find container \"79fb0b9def7e823767e221081fbd8459b5d4ff1c75a1a06f0007ccaaf91d766a\": container with ID starting with 79fb0b9def7e823767e221081fbd8459b5d4ff1c75a1a06f0007ccaaf91d766a not found: ID does not exist"
Sep 09 21:16:58.175643 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:58.175622    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm"]
Sep 09 21:16:58.179943 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:58.179923    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-0b5d4c-openshift-default-7595d5d48b-zwnkm"]
Sep 09 21:16:58.820494 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:16:58.820463    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cbb20d4d-d468-4972-ac78-1678e69b548b" path="/var/lib/kubelet/pods/cbb20d4d-d468-4972-ac78-1678e69b548b/volumes"
Sep 09 21:20:39.015394 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:20:39.015362    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-wsv4v_2b1f9a6e-7a79-4e89-ac3f-4658d6d91550/console-operator/1.log"
Sep 09 21:20:39.025345 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:20:39.025318    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-75wj4_55b4fbae-29dc-41bf-a907-b1412b852cef/ovn-acl-logging/0.log"
Sep 09 21:20:39.025515 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:20:39.025330    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-wsv4v_2b1f9a6e-7a79-4e89-ac3f-4658d6d91550/console-operator/1.log"
Sep 09 21:20:39.032235 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:20:39.032213    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-75wj4_55b4fbae-29dc-41bf-a907-b1412b852cef/ovn-acl-logging/0.log"
Sep 09 21:20:39.511052 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:20:39.511023    2581 scope.go:117] "RemoveContainer" containerID="0fa058e88059d9cfd33ca1c7adb4d09a335be31502a91daad170ef6f7c406942"
Sep 09 21:21:39.529088 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:21:39.529043    2581 scope.go:117] "RemoveContainer" containerID="1af91b1fbca0debe831e92ceaa0ce9521459adce784c2ef5e4c63b351467a370"
Sep 09 21:21:39.537485 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:21:39.537466    2581 scope.go:117] "RemoveContainer" containerID="51a35259a4515adf5c10b57fdf3fcc44539514750764f9fc775aed1565cf4f3a"
Sep 09 21:21:39.544612 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:21:39.544590    2581 scope.go:117] "RemoveContainer" containerID="d3c6447bb2649c01640192a40058d5249ea8600922bc660dcc9573c8a5751993"
Sep 09 21:22:39.570095 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:22:39.570009    2581 scope.go:117] "RemoveContainer" containerID="0c6de404fd5c0b3785221d308591909f089d836d502ed45b54eb0c9147f6d856"
Sep 09 21:22:39.577989 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:22:39.577965    2581 scope.go:117] "RemoveContainer" containerID="0b242aeb189b01dfe300448ec43a397cab368ba42e7f58bd4b12be6acaded2d2"
Sep 09 21:22:39.585386 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:22:39.585368    2581 scope.go:117] "RemoveContainer" containerID="b0a6d6a10430d37d595d9a03a418c498d9201a712b3247395c532754683600de"
Sep 09 21:22:56.011484 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:22:56.011458    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-8l6p8_8dcbd9c0-e30e-44ac-997d-27e33b31371d/server/0.log"
Sep 09 21:22:56.135331 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:22:56.135300    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-6k5n7_fd53ab69-cbaa-4965-9333-96549bbd2c70/manager/0.log"
Sep 09 21:22:56.492996 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:22:56.492972    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6d46659dfc-vjcbm_ba2f8e6c-79c7-4f53-8fdf-bc0b952a7082/copy-manifests/0.log"
Sep 09 21:22:56.500232 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:22:56.500182    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6d46659dfc-vjcbm_ba2f8e6c-79c7-4f53-8fdf-bc0b952a7082/manager/0.log"
Sep 09 21:22:57.207890 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:22:57.207859    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29816475-njf6n_96646b8f-e903-42b5-96df-a4d350460db2/cleanup/0.log"
Sep 09 21:22:57.318019 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:22:57.317978    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-pkz5w_06ff6c9f-5dbb-4ca2-a82e-5c3681890936/postgres/0.log"
Sep 09 21:22:59.101557 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:22:59.101527    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7_a96bea0e-3a19-4fbf-84b4-e06fb29de75e/istio-proxy/0.log"
Sep 09 21:22:59.214823 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:22:59.214795    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-wpxt6_cc763fcc-7c3b-4187-a9fa-3763f70cb349/discovery/0.log"
Sep 09 21:22:59.934454 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:22:59.934428    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7f6c8f5bc8-h8tgh_d5d449a3-ab1b-4899-842d-fcc9accb0838/router/0.log"
Sep 09 21:23:00.385569 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:00.385539    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-2d7ms_a5ef005c-d8f6-4d09-ac08-2fa7483c59e5/main/0.log"
Sep 09 21:23:00.740175 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:00.740138    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5rgbm_3230d3ca-4f75-4a0d-92eb-7ba1b71cafaa/main/0.log"
Sep 09 21:23:00.861041 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:00.861003    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-clx96_8afcba57-faa6-4184-91d0-a8b97d76f495/main/0.log"
Sep 09 21:23:09.088902 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:09.088861    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-bjwzq_ad64d253-c155-4e3c-b872-cdebb0616a0e/global-pull-secret-syncer/0.log"
Sep 09 21:23:09.362684 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:09.362589    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-lx79f_0bd19a8a-4d11-4e06-b2f9-7f04ccd35574/konnectivity-agent/0.log"
Sep 09 21:23:09.395552 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:09.395525    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-99.ec2.internal_75cd1de7188eb1602ddc4211bd282ec4/haproxy/0.log"
Sep 09 21:23:15.820571 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:15.820541    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-684mg_4978df08-2309-45d0-85f2-614c151114ea/node-exporter/0.log"
Sep 09 21:23:15.850709 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:15.850684    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-684mg_4978df08-2309-45d0-85f2-614c151114ea/kube-rbac-proxy/0.log"
Sep 09 21:23:15.880559 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:15.880533    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-684mg_4978df08-2309-45d0-85f2-614c151114ea/init-textfile/0.log"
Sep 09 21:23:16.137760 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:16.137680    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-jxl74_59748361-30b6-44c6-abbb-3a488d57f060/kube-rbac-proxy-main/0.log"
Sep 09 21:23:16.168341 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:16.168316    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-jxl74_59748361-30b6-44c6-abbb-3a488d57f060/kube-rbac-proxy-self/0.log"
Sep 09 21:23:16.200907 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:16.200876    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-jxl74_59748361-30b6-44c6-abbb-3a488d57f060/openshift-state-metrics/0.log"
Sep 09 21:23:16.258659 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:16.258633    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c8f2b641-7983-4cd8-9efd-29b1b462ff7b/prometheus/0.log"
Sep 09 21:23:16.299271 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:16.299244    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c8f2b641-7983-4cd8-9efd-29b1b462ff7b/config-reloader/0.log"
Sep 09 21:23:16.346164 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:16.346136    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c8f2b641-7983-4cd8-9efd-29b1b462ff7b/thanos-sidecar/0.log"
Sep 09 21:23:16.386873 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:16.386833    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c8f2b641-7983-4cd8-9efd-29b1b462ff7b/kube-rbac-proxy-web/0.log"
Sep 09 21:23:16.441173 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:16.441059    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c8f2b641-7983-4cd8-9efd-29b1b462ff7b/kube-rbac-proxy/0.log"
Sep 09 21:23:16.488561 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:16.488527    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c8f2b641-7983-4cd8-9efd-29b1b462ff7b/kube-rbac-proxy-thanos/0.log"
Sep 09 21:23:16.525002 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:16.524977    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c8f2b641-7983-4cd8-9efd-29b1b462ff7b/init-config-reloader/0.log"
Sep 09 21:23:16.575924 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:16.575896    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-2njn8_b94a227a-a76f-4bf9-9a49-91088409cb26/prometheus-operator/0.log"
Sep 09 21:23:16.614161 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:16.614132    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-2njn8_b94a227a-a76f-4bf9-9a49-91088409cb26/kube-rbac-proxy/0.log"
Sep 09 21:23:16.862708 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:16.862684    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-75b99b9678-9pr69_e34537f9-daf0-43cb-a80b-6ad6f3ace41b/thanos-query/0.log"
Sep 09 21:23:16.903462 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:16.903435    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-75b99b9678-9pr69_e34537f9-daf0-43cb-a80b-6ad6f3ace41b/kube-rbac-proxy-web/0.log"
Sep 09 21:23:16.933401 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:16.933376    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-75b99b9678-9pr69_e34537f9-daf0-43cb-a80b-6ad6f3ace41b/kube-rbac-proxy/0.log"
Sep 09 21:23:16.968728 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:16.968704    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-75b99b9678-9pr69_e34537f9-daf0-43cb-a80b-6ad6f3ace41b/prom-label-proxy/0.log"
Sep 09 21:23:17.002460 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:17.002436    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-75b99b9678-9pr69_e34537f9-daf0-43cb-a80b-6ad6f3ace41b/kube-rbac-proxy-rules/0.log"
Sep 09 21:23:17.054362 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:17.054334    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-75b99b9678-9pr69_e34537f9-daf0-43cb-a80b-6ad6f3ace41b/kube-rbac-proxy-metrics/0.log"
Sep 09 21:23:18.508992 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:18.508962    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-79977f9844-jx7vj_7c35fa5e-a4d9-48bb-9fcf-63cca4ed0fd2/networking-console-plugin/0.log"
Sep 09 21:23:18.922077 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:18.921986    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-dsjk4/perf-node-gather-daemonset-n24zk"]
Sep 09 21:23:18.922546 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:18.922524    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff98c003-73ee-4281-a285-da66a91dc957" containerName="revoke-keys"
Sep 09 21:23:18.922546 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:18.922547    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff98c003-73ee-4281-a285-da66a91dc957" containerName="revoke-keys"
Sep 09 21:23:18.922697 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:18.922556    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cbb20d4d-d468-4972-ac78-1678e69b548b" containerName="istio-proxy"
Sep 09 21:23:18.922697 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:18.922563    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="cbb20d4d-d468-4972-ac78-1678e69b548b" containerName="istio-proxy"
Sep 09 21:23:18.922697 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:18.922631    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="cbb20d4d-d468-4972-ac78-1678e69b548b" containerName="istio-proxy"
Sep 09 21:23:18.922697 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:18.922642    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff98c003-73ee-4281-a285-da66a91dc957" containerName="revoke-keys"
Sep 09 21:23:18.925675 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:18.925659    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dsjk4/perf-node-gather-daemonset-n24zk"
Sep 09 21:23:18.928632 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:18.928610    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dsjk4\"/\"kube-root-ca.crt\""
Sep 09 21:23:18.929691 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:18.929676    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-dsjk4\"/\"default-dockercfg-78ph6\""
Sep 09 21:23:18.929794 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:18.929740    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dsjk4\"/\"openshift-service-ca.crt\""
Sep 09 21:23:18.937616 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:18.937596    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dsjk4/perf-node-gather-daemonset-n24zk"]
Sep 09 21:23:19.074097 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:19.074068    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7cfbb597-e526-4f06-b2cd-b44d37a12c65-podres\") pod \"perf-node-gather-daemonset-n24zk\" (UID: \"7cfbb597-e526-4f06-b2cd-b44d37a12c65\") " pod="openshift-must-gather-dsjk4/perf-node-gather-daemonset-n24zk"
Sep 09 21:23:19.074282 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:19.074142    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjspn\" (UniqueName: \"kubernetes.io/projected/7cfbb597-e526-4f06-b2cd-b44d37a12c65-kube-api-access-kjspn\") pod \"perf-node-gather-daemonset-n24zk\" (UID: \"7cfbb597-e526-4f06-b2cd-b44d37a12c65\") " pod="openshift-must-gather-dsjk4/perf-node-gather-daemonset-n24zk"
Sep 09 21:23:19.074282 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:19.074205    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7cfbb597-e526-4f06-b2cd-b44d37a12c65-sys\") pod \"perf-node-gather-daemonset-n24zk\" (UID: \"7cfbb597-e526-4f06-b2cd-b44d37a12c65\") " pod="openshift-must-gather-dsjk4/perf-node-gather-daemonset-n24zk"
Sep 09 21:23:19.074282 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:19.074273    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7cfbb597-e526-4f06-b2cd-b44d37a12c65-lib-modules\") pod \"perf-node-gather-daemonset-n24zk\" (UID: \"7cfbb597-e526-4f06-b2cd-b44d37a12c65\") " pod="openshift-must-gather-dsjk4/perf-node-gather-daemonset-n24zk"
Sep 09 21:23:19.074412 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:19.074364    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7cfbb597-e526-4f06-b2cd-b44d37a12c65-proc\") pod \"perf-node-gather-daemonset-n24zk\" (UID: \"7cfbb597-e526-4f06-b2cd-b44d37a12c65\") " pod="openshift-must-gather-dsjk4/perf-node-gather-daemonset-n24zk"
Sep 09 21:23:19.174846 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:19.174764    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7cfbb597-e526-4f06-b2cd-b44d37a12c65-sys\") pod \"perf-node-gather-daemonset-n24zk\" (UID: \"7cfbb597-e526-4f06-b2cd-b44d37a12c65\") " pod="openshift-must-gather-dsjk4/perf-node-gather-daemonset-n24zk"
Sep 09 21:23:19.174846 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:19.174825    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7cfbb597-e526-4f06-b2cd-b44d37a12c65-lib-modules\") pod \"perf-node-gather-daemonset-n24zk\" (UID: \"7cfbb597-e526-4f06-b2cd-b44d37a12c65\") " pod="openshift-must-gather-dsjk4/perf-node-gather-daemonset-n24zk"
Sep 09 21:23:19.175055 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:19.174853    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7cfbb597-e526-4f06-b2cd-b44d37a12c65-sys\") pod \"perf-node-gather-daemonset-n24zk\" (UID: \"7cfbb597-e526-4f06-b2cd-b44d37a12c65\") " pod="openshift-must-gather-dsjk4/perf-node-gather-daemonset-n24zk"
Sep 09 21:23:19.175055 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:19.174875    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7cfbb597-e526-4f06-b2cd-b44d37a12c65-proc\") pod \"perf-node-gather-daemonset-n24zk\" (UID: \"7cfbb597-e526-4f06-b2cd-b44d37a12c65\") " pod="openshift-must-gather-dsjk4/perf-node-gather-daemonset-n24zk"
Sep 09 21:23:19.175055 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:19.174891    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7cfbb597-e526-4f06-b2cd-b44d37a12c65-podres\") pod \"perf-node-gather-daemonset-n24zk\" (UID: \"7cfbb597-e526-4f06-b2cd-b44d37a12c65\") " pod="openshift-must-gather-dsjk4/perf-node-gather-daemonset-n24zk"
Sep 09 21:23:19.175055 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:19.174917    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjspn\" (UniqueName: \"kubernetes.io/projected/7cfbb597-e526-4f06-b2cd-b44d37a12c65-kube-api-access-kjspn\") pod \"perf-node-gather-daemonset-n24zk\" (UID: \"7cfbb597-e526-4f06-b2cd-b44d37a12c65\") " pod="openshift-must-gather-dsjk4/perf-node-gather-daemonset-n24zk"
Sep 09 21:23:19.175055 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:19.174954    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7cfbb597-e526-4f06-b2cd-b44d37a12c65-lib-modules\") pod \"perf-node-gather-daemonset-n24zk\" (UID: \"7cfbb597-e526-4f06-b2cd-b44d37a12c65\") " pod="openshift-must-gather-dsjk4/perf-node-gather-daemonset-n24zk"
Sep 09 21:23:19.175055 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:19.174955    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7cfbb597-e526-4f06-b2cd-b44d37a12c65-proc\") pod \"perf-node-gather-daemonset-n24zk\" (UID: \"7cfbb597-e526-4f06-b2cd-b44d37a12c65\") " pod="openshift-must-gather-dsjk4/perf-node-gather-daemonset-n24zk"
Sep 09 21:23:19.175055 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:19.175007    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7cfbb597-e526-4f06-b2cd-b44d37a12c65-podres\") pod \"perf-node-gather-daemonset-n24zk\" (UID: \"7cfbb597-e526-4f06-b2cd-b44d37a12c65\") " pod="openshift-must-gather-dsjk4/perf-node-gather-daemonset-n24zk"
Sep 09 21:23:19.184368 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:19.184340    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjspn\" (UniqueName: \"kubernetes.io/projected/7cfbb597-e526-4f06-b2cd-b44d37a12c65-kube-api-access-kjspn\") pod \"perf-node-gather-daemonset-n24zk\" (UID: \"7cfbb597-e526-4f06-b2cd-b44d37a12c65\") " pod="openshift-must-gather-dsjk4/perf-node-gather-daemonset-n24zk"
Sep 09 21:23:19.235305 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:19.235280    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dsjk4/perf-node-gather-daemonset-n24zk"
Sep 09 21:23:19.274013 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:19.273985    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-wsv4v_2b1f9a6e-7a79-4e89-ac3f-4658d6d91550/console-operator/1.log"
Sep 09 21:23:19.279536 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:19.279512    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-wsv4v_2b1f9a6e-7a79-4e89-ac3f-4658d6d91550/console-operator/2.log"
Sep 09 21:23:19.525269 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:19.525244    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dsjk4/perf-node-gather-daemonset-n24zk"]
Sep 09 21:23:19.527515 ip-10-0-129-99 kubenswrapper[2581]: W0909 21:23:19.527493    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod7cfbb597_e526_4f06_b2cd_b44d37a12c65.slice/crio-06379ff4918bd06a44bbbcd8789d47b9de1162c5f0b3a5f46e8c9723b46e8ff5 WatchSource:0}: Error finding container 06379ff4918bd06a44bbbcd8789d47b9de1162c5f0b3a5f46e8c9723b46e8ff5: Status 404 returned error can't find the container with id 06379ff4918bd06a44bbbcd8789d47b9de1162c5f0b3a5f46e8c9723b46e8ff5
Sep 09 21:23:20.034625 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:20.034589    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dsjk4/perf-node-gather-daemonset-n24zk" event={"ID":"7cfbb597-e526-4f06-b2cd-b44d37a12c65","Type":"ContainerStarted","Data":"e8555ca7478363efc48631c2afebe2ec827654dccca9b72df0a98aaf504c4454"}
Sep 09 21:23:20.034828 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:20.034642    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dsjk4/perf-node-gather-daemonset-n24zk" event={"ID":"7cfbb597-e526-4f06-b2cd-b44d37a12c65","Type":"ContainerStarted","Data":"06379ff4918bd06a44bbbcd8789d47b9de1162c5f0b3a5f46e8c9723b46e8ff5"}
Sep 09 21:23:20.034828 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:20.034699    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-dsjk4/perf-node-gather-daemonset-n24zk"
Sep 09 21:23:20.054281 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:20.054237    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-dsjk4/perf-node-gather-daemonset-n24zk" podStartSLOduration=2.054222682 podStartE2EDuration="2.054222682s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:23:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:23:20.052782555 +0000 UTC m=+2561.680410419" watchObservedRunningTime="2026-09-09 21:23:20.054222682 +0000 UTC m=+2561.681850546"
Sep 09 21:23:21.916575 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:21.916548    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-g2tsp_48a52e3f-68fb-496f-b974-320a8539e7f7/dns/0.log"
Sep 09 21:23:21.951339 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:21.951315    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-g2tsp_48a52e3f-68fb-496f-b974-320a8539e7f7/kube-rbac-proxy/0.log"
Sep 09 21:23:22.159011 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:22.158982    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-cfg6s_5096efd2-6fbf-45a2-930b-acdc3970eabd/dns-node-resolver/0.log"
Sep 09 21:23:22.779814 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:22.779782    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-69887ff495-l9r6j_45d6b919-92b0-4092-8522-039bcb692260/registry/0.log"
Sep 09 21:23:22.855187 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:22.855162    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-f8mqt_0104bf49-c8d9-493a-b189-e570ebf93e62/node-ca/0.log"
Sep 09 21:23:23.907701 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:23.907675    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7_a96bea0e-3a19-4fbf-84b4-e06fb29de75e/istio-proxy/0.log"
Sep 09 21:23:24.196049 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:24.195959    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-wpxt6_cc763fcc-7c3b-4187-a9fa-3763f70cb349/discovery/0.log"
Sep 09 21:23:24.436870 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:24.436839    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7f6c8f5bc8-h8tgh_d5d449a3-ab1b-4899-842d-fcc9accb0838/router/0.log"
Sep 09 21:23:25.365452 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:25.365420    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4zgnb_933d2649-0bcc-4c52-a0ac-65f33ba49741/serve-healthcheck-canary/0.log"
Sep 09 21:23:25.921966 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:25.921937    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-r5kxh_4eb46131-2f57-408b-9d0f-a9ee9b55c8ef/insights-operator/0.log"
Sep 09 21:23:25.922287 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:25.922265    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-r5kxh_4eb46131-2f57-408b-9d0f-a9ee9b55c8ef/insights-operator/1.log"
Sep 09 21:23:25.948394 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:25.948366    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2ptr7_64dbb975-388d-48c4-bd16-7b33658225d0/kube-rbac-proxy/0.log"
Sep 09 21:23:25.973620 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:25.973596    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2ptr7_64dbb975-388d-48c4-bd16-7b33658225d0/exporter/0.log"
Sep 09 21:23:26.000597 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:26.000574    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2ptr7_64dbb975-388d-48c4-bd16-7b33658225d0/extractor/0.log"
Sep 09 21:23:26.048215 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:26.048195    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-dsjk4/perf-node-gather-daemonset-n24zk"
Sep 09 21:23:29.448746 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:29.448715    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-8l6p8_8dcbd9c0-e30e-44ac-997d-27e33b31371d/server/0.log"
Sep 09 21:23:29.547947 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:29.547863    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-6k5n7_fd53ab69-cbaa-4965-9333-96549bbd2c70/manager/0.log"
Sep 09 21:23:29.635439 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:29.635409    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7_a96bea0e-3a19-4fbf-84b4-e06fb29de75e/istio-proxy/0.log"
Sep 09 21:23:29.900883 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:29.900800    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6d46659dfc-vjcbm_ba2f8e6c-79c7-4f53-8fdf-bc0b952a7082/manager/0.log"
Sep 09 21:23:29.938219 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:29.938194    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6d46659dfc-vjcbm_ba2f8e6c-79c7-4f53-8fdf-bc0b952a7082/copy-manifests/0.log"
Sep 09 21:23:29.974755 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:29.974726    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-wpxt6_cc763fcc-7c3b-4187-a9fa-3763f70cb349/discovery/0.log"
Sep 09 21:23:30.153137 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:30.153045    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7f6c8f5bc8-h8tgh_d5d449a3-ab1b-4899-842d-fcc9accb0838/router/0.log"
Sep 09 21:23:31.354548 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:31.354520    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-58fbd4c7f6-mvldh_d970c548-c924-495f-8362-85fe35786ef4/manager/0.log"
Sep 09 21:23:31.746809 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:31.746776    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-m2bj7_a96bea0e-3a19-4fbf-84b4-e06fb29de75e/istio-proxy/0.log"
Sep 09 21:23:32.044535 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:32.044457    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-wpxt6_cc763fcc-7c3b-4187-a9fa-3763f70cb349/discovery/0.log"
Sep 09 21:23:32.213780 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:32.213753    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7f6c8f5bc8-h8tgh_d5d449a3-ab1b-4899-842d-fcc9accb0838/router/0.log"
Sep 09 21:23:36.225914 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:36.225883    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-wg2t6_ccb53e29-73b5-4a60-a39e-51354976c399/migrator/0.log"
Sep 09 21:23:36.252827 ip-10-0-129-99 kubenswrapper[2581]: I0909 21:23:36.252798    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-wg2t6_ccb53e29-73b5-4a60-a39e-51354976c399/graceful-termination/0.log"