Oct 30 19:30:51.817234 ip-10-0-141-42 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 19:30:52.197446 ip-10-0-141-42 kubenswrapper[2578]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 19:30:52.197446 ip-10-0-141-42 kubenswrapper[2578]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 19:30:52.197446 ip-10-0-141-42 kubenswrapper[2578]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 19:30:52.197446 ip-10-0-141-42 kubenswrapper[2578]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 19:30:52.200666 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200585    2578 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 19:30:52.200666 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200666    2578 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 19:30:52.200666 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200670    2578 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 19:30:52.200754 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200673    2578 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 19:30:52.200754 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200676    2578 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 19:30:52.200754 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200679    2578 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 19:30:52.200754 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200681    2578 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 19:30:52.200754 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200684    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 19:30:52.200754 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200687    2578 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 19:30:52.200754 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200689    2578 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 19:30:52.200754 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200692    2578 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 19:30:52.200754 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200697    2578 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 19:30:52.200754 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200701    2578 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 19:30:52.200754 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200704    2578 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 19:30:52.200754 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200707    2578 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 19:30:52.200754 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200709    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 19:30:52.200754 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200712    2578 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 19:30:52.200754 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200715    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 19:30:52.200754 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200718    2578 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 19:30:52.200754 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200727    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 19:30:52.200754 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200730    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 19:30:52.200754 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200732    2578 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 19:30:52.200754 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200735    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 19:30:52.201239 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200738    2578 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 19:30:52.201239 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200740    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 19:30:52.201239 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200743    2578 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 19:30:52.201239 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200745    2578 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 19:30:52.201239 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200748    2578 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 19:30:52.201239 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200750    2578 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 19:30:52.201239 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200753    2578 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 19:30:52.201239 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200756    2578 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 19:30:52.201239 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200758    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 19:30:52.201239 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200761    2578 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 19:30:52.201239 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200764    2578 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 19:30:52.201239 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200767    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 19:30:52.201239 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200769    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 19:30:52.201239 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200772    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 19:30:52.201239 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200776    2578 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 19:30:52.201239 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200780    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 19:30:52.201239 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200783    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 19:30:52.201239 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200786    2578 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 19:30:52.201239 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200789    2578 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 19:30:52.201683 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200792    2578 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 19:30:52.201683 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200794    2578 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 19:30:52.201683 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200797    2578 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 19:30:52.201683 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200799    2578 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 19:30:52.201683 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200802    2578 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 19:30:52.201683 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200804    2578 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 19:30:52.201683 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200807    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 19:30:52.201683 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200809    2578 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 19:30:52.201683 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200812    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 19:30:52.201683 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200818    2578 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 19:30:52.201683 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200821    2578 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 19:30:52.201683 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200823    2578 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 19:30:52.201683 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200826    2578 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 19:30:52.201683 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200829    2578 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 19:30:52.201683 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200832    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 19:30:52.201683 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200835    2578 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 19:30:52.201683 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200837    2578 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 19:30:52.201683 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200840    2578 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 19:30:52.201683 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200843    2578 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 19:30:52.201683 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200845    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 19:30:52.202198 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200848    2578 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 19:30:52.202198 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200850    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 19:30:52.202198 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200853    2578 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 19:30:52.202198 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200855    2578 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 19:30:52.202198 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200858    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 19:30:52.202198 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200862    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 19:30:52.202198 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200864    2578 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 19:30:52.202198 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200867    2578 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 19:30:52.202198 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200869    2578 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 19:30:52.202198 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200871    2578 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 19:30:52.202198 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200874    2578 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 19:30:52.202198 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200876    2578 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 19:30:52.202198 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200879    2578 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 19:30:52.202198 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200881    2578 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 19:30:52.202198 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200884    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 19:30:52.202198 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200887    2578 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 19:30:52.202198 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200890    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 19:30:52.202198 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200892    2578 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 19:30:52.202198 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200895    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 19:30:52.202650 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200897    2578 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 19:30:52.202650 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200900    2578 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 19:30:52.202650 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200903    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 19:30:52.202650 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200905    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 19:30:52.202650 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200908    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 19:30:52.202650 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200912    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 19:30:52.202650 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200914    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 19:30:52.202650 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200917    2578 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 19:30:52.202650 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200932    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 19:30:52.202650 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200935    2578 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 19:30:52.202650 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200937    2578 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 19:30:52.202650 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200940    2578 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 19:30:52.202650 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200943    2578 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 19:30:52.202650 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200946    2578 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 19:30:52.202650 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200948    2578 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 19:30:52.202650 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200951    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 19:30:52.202650 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200954    2578 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 19:30:52.202650 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200956    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 19:30:52.202650 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200960    2578 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 19:30:52.202650 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200963    2578 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 19:30:52.203128 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200965    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 19:30:52.203128 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200968    2578 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 19:30:52.203128 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200971    2578 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 19:30:52.203128 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200974    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 19:30:52.203128 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.200977    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 19:30:52.203347 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203334    2578 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 19:30:52.203347 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203347    2578 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 19:30:52.203399 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203351    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 19:30:52.203399 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203354    2578 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 19:30:52.203399 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203357    2578 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 19:30:52.203399 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203360    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 19:30:52.203399 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203363    2578 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 19:30:52.203399 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203365    2578 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 19:30:52.203399 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203368    2578 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 19:30:52.203399 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203370    2578 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 19:30:52.203399 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203373    2578 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 19:30:52.203399 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203382    2578 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 19:30:52.203399 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203385    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 19:30:52.203399 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203387    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 19:30:52.203399 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203390    2578 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 19:30:52.203399 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203393    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 19:30:52.203399 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203395    2578 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 19:30:52.203399 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203398    2578 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 19:30:52.203399 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203400    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 19:30:52.203399 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203403    2578 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 19:30:52.203399 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203406    2578 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 19:30:52.203861 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203409    2578 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 19:30:52.203861 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203412    2578 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 19:30:52.203861 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203414    2578 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 19:30:52.203861 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203422    2578 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 19:30:52.203861 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203425    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 19:30:52.203861 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203427    2578 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 19:30:52.203861 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203430    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 19:30:52.203861 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203433    2578 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 19:30:52.203861 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203437    2578 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 19:30:52.203861 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203440    2578 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 19:30:52.203861 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203443    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 19:30:52.203861 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203445    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 19:30:52.203861 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203448    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 19:30:52.203861 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203450    2578 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 19:30:52.203861 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203453    2578 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 19:30:52.203861 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203455    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 19:30:52.203861 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203458    2578 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 19:30:52.203861 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203460    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 19:30:52.203861 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203462    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 19:30:52.203861 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203465    2578 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 19:30:52.204349 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203468    2578 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 19:30:52.204349 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203470    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 19:30:52.204349 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203473    2578 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 19:30:52.204349 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203481    2578 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 19:30:52.204349 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203484    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 19:30:52.204349 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203487    2578 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 19:30:52.204349 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203489    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 19:30:52.204349 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203492    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 19:30:52.204349 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203494    2578 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 19:30:52.204349 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203496    2578 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 19:30:52.204349 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203499    2578 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 19:30:52.204349 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203501    2578 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 19:30:52.204349 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203504    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 19:30:52.204349 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203507    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 19:30:52.204349 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203510    2578 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 19:30:52.204349 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203513    2578 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 19:30:52.204349 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203515    2578 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 19:30:52.204349 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203518    2578 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 19:30:52.204349 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203520    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 19:30:52.204869 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203523    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 19:30:52.204869 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203525    2578 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 19:30:52.204869 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203528    2578 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 19:30:52.204869 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203530    2578 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 19:30:52.204869 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203533    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 19:30:52.204869 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203535    2578 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 19:30:52.204869 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203538    2578 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 19:30:52.204869 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203540    2578 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 19:30:52.204869 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203543    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 19:30:52.204869 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203545    2578 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 19:30:52.204869 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203548    2578 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 19:30:52.204869 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203550    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 19:30:52.204869 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203553    2578 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 19:30:52.204869 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203556    2578 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 19:30:52.204869 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203558    2578 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 19:30:52.204869 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203561    2578 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 19:30:52.204869 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203563    2578 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 19:30:52.204869 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203568    2578 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 19:30:52.204869 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203572    2578 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 19:30:52.205339 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203575    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 19:30:52.205339 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203578    2578 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 19:30:52.205339 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203580    2578 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 19:30:52.205339 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203583    2578 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 19:30:52.205339 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203586    2578 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 19:30:52.205339 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203588    2578 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 19:30:52.205339 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203592    2578 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 19:30:52.205339 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203595    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 19:30:52.205339 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203598    2578 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 19:30:52.205339 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203601    2578 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 19:30:52.205339 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203603    2578 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 19:30:52.205339 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203606    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 19:30:52.205339 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203608    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 19:30:52.205339 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203611    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 19:30:52.205339 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203613    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 19:30:52.205339 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203616    2578 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 19:30:52.205339 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203618    2578 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 19:30:52.205339 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203621    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 19:30:52.205339 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203624    2578 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 19:30:52.205339 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203626    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 19:30:52.205816 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203628    2578 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 19:30:52.205816 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203631    2578 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 19:30:52.205816 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203634    2578 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 19:30:52.205816 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203636    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 19:30:52.205816 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203638    2578 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 19:30:52.205816 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203641    2578 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 19:30:52.205816 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.203644    2578 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 19:30:52.205816 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204180    2578 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 19:30:52.205816 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204186    2578 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 19:30:52.205816 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204189    2578 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 19:30:52.205816 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204198    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 19:30:52.205816 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204200    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 19:30:52.205816 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204203    2578 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 19:30:52.205816 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204205    2578 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 19:30:52.205816 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204208    2578 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 19:30:52.205816 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204210    2578 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 19:30:52.205816 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204213    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 19:30:52.205816 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204216    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 19:30:52.205816 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204218    2578 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 19:30:52.205816 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204221    2578 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 19:30:52.206305 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204224    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 19:30:52.206305 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204227    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 19:30:52.206305 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204230    2578 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 19:30:52.206305 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204232    2578 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 19:30:52.206305 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204235    2578 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 19:30:52.206305 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204237    2578 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 19:30:52.206305 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204240    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 19:30:52.206305 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204243    2578 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 19:30:52.206305 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204245    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 19:30:52.206305 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204248    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 19:30:52.206305 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204250    2578 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 19:30:52.206305 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204253    2578 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 19:30:52.206305 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204255    2578 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 19:30:52.206305 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204258    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 19:30:52.206305 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204260    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 19:30:52.206305 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204263    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 19:30:52.206305 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204266    2578 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 19:30:52.206305 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204268    2578 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 19:30:52.206305 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204271    2578 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 19:30:52.206305 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204274    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 19:30:52.206783 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204276    2578 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 19:30:52.206783 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204279    2578 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 19:30:52.206783 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204282    2578 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 19:30:52.206783 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204290    2578 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 19:30:52.206783 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204293    2578 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 19:30:52.206783 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204302    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 19:30:52.206783 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204305    2578 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 19:30:52.206783 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204308    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 19:30:52.206783 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204311    2578 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 19:30:52.206783 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204313    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 19:30:52.206783 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204315    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 19:30:52.206783 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204318    2578 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 19:30:52.206783 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204321    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 19:30:52.206783 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204323    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 19:30:52.206783 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204326    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 19:30:52.206783 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204328    2578 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 19:30:52.206783 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204331    2578 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 19:30:52.206783 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204333    2578 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 19:30:52.206783 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204336    2578 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 19:30:52.207253 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204339    2578 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 19:30:52.207253 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204347    2578 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 19:30:52.207253 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204350    2578 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 19:30:52.207253 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204353    2578 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 19:30:52.207253 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204356    2578 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 19:30:52.207253 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204358    2578 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 19:30:52.207253 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204361    2578 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 19:30:52.207253 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204364    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 19:30:52.207253 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204366    2578 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 19:30:52.207253 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204369    2578 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 19:30:52.207253 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204372    2578 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 19:30:52.207253 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204375    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 19:30:52.207253 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204377    2578 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 19:30:52.207253 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204380    2578 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 19:30:52.207253 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204383    2578 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 19:30:52.207253 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204385    2578 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 19:30:52.207253 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204388    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 19:30:52.207253 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204396    2578 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 19:30:52.207253 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204399    2578 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 19:30:52.207693 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204403    2578 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 19:30:52.207693 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204407    2578 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 19:30:52.207693 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204409    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 19:30:52.207693 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204412    2578 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 19:30:52.207693 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204415    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 19:30:52.207693 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204418    2578 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 19:30:52.207693 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204421    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 19:30:52.207693 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204423    2578 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 19:30:52.207693 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204426    2578 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 19:30:52.207693 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204430    2578 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 19:30:52.207693 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204434    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 19:30:52.207693 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204437    2578 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 19:30:52.207693 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204441    2578 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 19:30:52.207693 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204444    2578 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 19:30:52.207693 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204446    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 19:30:52.207693 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204450    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 19:30:52.207693 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204452    2578 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 19:30:52.207693 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204455    2578 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 19:30:52.207693 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204458    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 19:30:52.208151 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204460    2578 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 19:30:52.208151 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204463    2578 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 19:30:52.208151 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204466    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 19:30:52.208151 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204469    2578 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 19:30:52.208151 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204472    2578 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 19:30:52.208151 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204474    2578 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 19:30:52.208151 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204477    2578 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 19:30:52.208151 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204480    2578 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 19:30:52.208151 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204482    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 19:30:52.208151 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204485    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 19:30:52.208151 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204488    2578 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 19:30:52.208151 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204491    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 19:30:52.208151 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204499    2578 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 19:30:52.208151 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204502    2578 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 19:30:52.208151 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204504    2578 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 19:30:52.208151 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.204507    2578 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 19:30:52.208151 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204619    2578 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 19:30:52.208151 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204627    2578 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 19:30:52.208151 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204640    2578 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 19:30:52.208151 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204645    2578 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 19:30:52.208151 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204649    2578 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 19:30:52.208650 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204653    2578 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 19:30:52.208650 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204657    2578 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 19:30:52.208650 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204661    2578 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 19:30:52.208650 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204664    2578 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 19:30:52.208650 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204667    2578 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 19:30:52.208650 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204671    2578 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 19:30:52.208650 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204674    2578 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 19:30:52.208650 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204677    2578 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 19:30:52.208650 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204680    2578 flags.go:64] FLAG: --cgroup-root=""
Oct 30 19:30:52.208650 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204683    2578 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 19:30:52.208650 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204686    2578 flags.go:64] FLAG: --client-ca-file=""
Oct 30 19:30:52.208650 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204689    2578 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 19:30:52.208650 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204692    2578 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 19:30:52.208650 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204700    2578 flags.go:64] FLAG: --cluster-domain=""
Oct 30 19:30:52.208650 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204703    2578 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 19:30:52.208650 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204706    2578 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 19:30:52.208650 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204709    2578 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 19:30:52.208650 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204713    2578 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 19:30:52.208650 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204717    2578 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 19:30:52.208650 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204720    2578 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 19:30:52.208650 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204723    2578 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 19:30:52.208650 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204726    2578 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 19:30:52.208650 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204729    2578 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 19:30:52.208650 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204732    2578 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 19:30:52.209250 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204741    2578 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 19:30:52.209250 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204744    2578 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 19:30:52.209250 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204747    2578 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 19:30:52.209250 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204751    2578 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 19:30:52.209250 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204755    2578 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 19:30:52.209250 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204758    2578 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 19:30:52.209250 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204761    2578 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 19:30:52.209250 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204764    2578 flags.go:64] FLAG: --enable-server="true"
Oct 30 19:30:52.209250 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204767    2578 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 19:30:52.209250 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204774    2578 flags.go:64] FLAG: --event-burst="100"
Oct 30 19:30:52.209250 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204777    2578 flags.go:64] FLAG: --event-qps="50"
Oct 30 19:30:52.209250 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204780    2578 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 19:30:52.209250 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204784    2578 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 19:30:52.209250 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204787    2578 flags.go:64] FLAG: --eviction-hard=""
Oct 30 19:30:52.209250 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204791    2578 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 19:30:52.209250 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204794    2578 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 19:30:52.209250 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204797    2578 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 19:30:52.209250 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204800    2578 flags.go:64] FLAG: --eviction-soft=""
Oct 30 19:30:52.209250 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204803    2578 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 19:30:52.209250 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204806    2578 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 19:30:52.209250 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204809    2578 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 19:30:52.209250 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204812    2578 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 19:30:52.209250 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204815    2578 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 19:30:52.209250 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204817    2578 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 19:30:52.209250 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204820    2578 flags.go:64] FLAG: --feature-gates=""
Oct 30 19:30:52.209899 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204826    2578 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 19:30:52.209899 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204829    2578 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 19:30:52.209899 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204832    2578 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 19:30:52.209899 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204835    2578 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 19:30:52.209899 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204838    2578 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 19:30:52.209899 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204841    2578 flags.go:64] FLAG: --help="false"
Oct 30 19:30:52.209899 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204844    2578 flags.go:64] FLAG: --hostname-override="ip-10-0-141-42.us-east-2.compute.internal"
Oct 30 19:30:52.209899 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204848    2578 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 19:30:52.209899 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204857    2578 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 19:30:52.209899 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204860    2578 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 19:30:52.209899 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204863    2578 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 19:30:52.209899 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204867    2578 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 19:30:52.209899 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204870    2578 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 19:30:52.209899 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204872    2578 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 19:30:52.209899 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204875    2578 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 19:30:52.209899 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204879    2578 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 19:30:52.209899 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204882    2578 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 19:30:52.209899 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204885    2578 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 19:30:52.209899 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204888    2578 flags.go:64] FLAG: --kube-reserved=""
Oct 30 19:30:52.209899 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204891    2578 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 19:30:52.209899 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204894    2578 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 19:30:52.209899 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204897    2578 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 19:30:52.209899 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204900    2578 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 19:30:52.210455 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204903    2578 flags.go:64] FLAG: --lock-file=""
Oct 30 19:30:52.210455 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204906    2578 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 19:30:52.210455 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204909    2578 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 19:30:52.210455 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204912    2578 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 19:30:52.210455 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204918    2578 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 19:30:52.210455 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204935    2578 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 19:30:52.210455 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204938    2578 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 19:30:52.210455 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204941    2578 flags.go:64] FLAG: --logging-format="text"
Oct 30 19:30:52.210455 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204944    2578 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 19:30:52.210455 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204947    2578 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 19:30:52.210455 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204950    2578 flags.go:64] FLAG: --manifest-url=""
Oct 30 19:30:52.210455 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204953    2578 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 19:30:52.210455 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204957    2578 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 19:30:52.210455 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204960    2578 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 19:30:52.210455 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204964    2578 flags.go:64] FLAG: --max-pods="110"
Oct 30 19:30:52.210455 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204967    2578 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 19:30:52.210455 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204970    2578 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 19:30:52.210455 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204973    2578 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 19:30:52.210455 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204981    2578 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 19:30:52.210455 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204984    2578 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 19:30:52.210455 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204987    2578 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 19:30:52.210455 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204990    2578 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 19:30:52.210455 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.204997    2578 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 19:30:52.210455 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.205000    2578 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 19:30:52.210455 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.205003    2578 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 19:30:52.211118 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.205006    2578 flags.go:64] FLAG: --pod-cidr=""
Oct 30 19:30:52.211118 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.205009    2578 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 19:30:52.211118 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.205015    2578 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 19:30:52.211118 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.205018    2578 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 19:30:52.211118 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.205021    2578 flags.go:64] FLAG: --port="10250"
Oct 30 19:30:52.211118 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.205024    2578 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 19:30:52.211118 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.205027    2578 flags.go:64] FLAG: --provider-id="aws:///us-east-2a/i-0c2e0ee918aa12755"
Oct 30 19:30:52.211118 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.205031    2578 flags.go:64] FLAG: --qos-reserved=""
Oct 30 19:30:52.211118 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.205034    2578 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 19:30:52.211118 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.205037    2578 flags.go:64] FLAG: --register-node="true"
Oct 30 19:30:52.211118 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.205043    2578 flags.go:64] FLAG: --register-with-taints=""
Oct 30 19:30:52.211118 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.205048    2578 flags.go:64] FLAG: --registry-burst="10"
Oct 30 19:30:52.211118 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.205051    2578 flags.go:64] FLAG: --registry-qps="5"
Oct 30 19:30:52.211118 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.205053    2578 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 19:30:52.211118 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.205056    2578 flags.go:64] FLAG: --reserved-memory=""
Oct 30 19:30:52.211118 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.205061    2578 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 19:30:52.211118 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.205064    2578 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 19:30:52.211118 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.205067    2578 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 19:30:52.211118 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.205069    2578 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 19:30:52.211118 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.205072    2578 flags.go:64] FLAG: --runonce="false"
Oct 30 19:30:52.211118 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.205075    2578 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 19:30:52.211118 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.205078    2578 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 19:30:52.211118 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.205081    2578 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 19:30:52.211118 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.205084    2578 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 19:30:52.211118 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.205087    2578 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 19:30:52.211118 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.205091    2578 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 19:30:52.211745 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.205095    2578 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 19:30:52.211745 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.205098    2578 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 19:30:52.211745 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.205101    2578 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 19:30:52.211745 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.205104    2578 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 19:30:52.211745 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.205107    2578 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 19:30:52.211745 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.205110    2578 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 19:30:52.211745 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.205113    2578 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 19:30:52.211745 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.205118    2578 flags.go:64] FLAG: --system-cgroups=""
Oct 30 19:30:52.211745 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.205122    2578 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 19:30:52.211745 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.205127    2578 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 19:30:52.211745 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.205130    2578 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 19:30:52.211745 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.205133    2578 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 19:30:52.211745 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.205138    2578 flags.go:64] FLAG: --tls-min-version=""
Oct 30 19:30:52.211745 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.205141    2578 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 19:30:52.211745 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.205144    2578 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 19:30:52.211745 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.205147    2578 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 19:30:52.211745 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.205150    2578 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 19:30:52.211745 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.205153    2578 flags.go:64] FLAG: --v="2"
Oct 30 19:30:52.211745 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.205157    2578 flags.go:64] FLAG: --version="false"
Oct 30 19:30:52.211745 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.205161    2578 flags.go:64] FLAG: --vmodule=""
Oct 30 19:30:52.211745 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.205166    2578 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 19:30:52.211745 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.205170    2578 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 19:30:52.211745 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205326    2578 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 19:30:52.211745 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205331    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 19:30:52.212332 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205334    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 19:30:52.212332 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205338    2578 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 19:30:52.212332 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205341    2578 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 19:30:52.212332 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205344    2578 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 19:30:52.212332 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205347    2578 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 19:30:52.212332 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205349    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 19:30:52.212332 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205352    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 19:30:52.212332 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205355    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 19:30:52.212332 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205358    2578 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 19:30:52.212332 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205366    2578 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 19:30:52.212332 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205369    2578 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 19:30:52.212332 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205371    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 19:30:52.212332 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205374    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 19:30:52.212332 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205377    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 19:30:52.212332 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205380    2578 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 19:30:52.212332 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205385    2578 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 19:30:52.212332 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205389    2578 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 19:30:52.212332 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205392    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 19:30:52.212332 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205395    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 19:30:52.212782 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205398    2578 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 19:30:52.212782 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205400    2578 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 19:30:52.212782 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205403    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 19:30:52.212782 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205406    2578 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 19:30:52.212782 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205408    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 19:30:52.212782 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205411    2578 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 19:30:52.212782 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205413    2578 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 19:30:52.212782 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205416    2578 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 19:30:52.212782 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205418    2578 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 19:30:52.212782 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205421    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 19:30:52.212782 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205423    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 19:30:52.212782 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205426    2578 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 19:30:52.212782 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205428    2578 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 19:30:52.212782 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205431    2578 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 19:30:52.212782 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205434    2578 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 19:30:52.212782 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205436    2578 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 19:30:52.212782 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205439    2578 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 19:30:52.212782 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205441    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 19:30:52.212782 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205444    2578 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 19:30:52.213238 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205446    2578 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 19:30:52.213238 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205449    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 19:30:52.213238 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205451    2578 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 19:30:52.213238 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205454    2578 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 19:30:52.213238 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205462    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 19:30:52.213238 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205464    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 19:30:52.213238 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205467    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 19:30:52.213238 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205470    2578 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 19:30:52.213238 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205472    2578 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 19:30:52.213238 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205476    2578 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 19:30:52.213238 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205480    2578 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 19:30:52.213238 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205483    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 19:30:52.213238 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205485    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 19:30:52.213238 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205488    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 19:30:52.213238 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205490    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 19:30:52.213238 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205493    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 19:30:52.213238 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205495    2578 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 19:30:52.213238 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205498    2578 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 19:30:52.213238 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205500    2578 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 19:30:52.213238 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205503    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 19:30:52.213742 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205505    2578 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 19:30:52.213742 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205508    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 19:30:52.213742 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205511    2578 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 19:30:52.213742 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205513    2578 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 19:30:52.213742 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205515    2578 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 19:30:52.213742 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205518    2578 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 19:30:52.213742 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205521    2578 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 19:30:52.213742 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205523    2578 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 19:30:52.213742 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205526    2578 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 19:30:52.213742 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205528    2578 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 19:30:52.213742 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205531    2578 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 19:30:52.213742 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205533    2578 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 19:30:52.213742 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205536    2578 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 19:30:52.213742 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205538    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 19:30:52.213742 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205541    2578 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 19:30:52.213742 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205543    2578 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 19:30:52.213742 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205546    2578 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 19:30:52.213742 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205554    2578 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 19:30:52.213742 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205557    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 19:30:52.213742 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205559    2578 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 19:30:52.213742 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205562    2578 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 19:30:52.214247 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205566    2578 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 19:30:52.214247 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205570    2578 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 19:30:52.214247 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205573    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 19:30:52.214247 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205576    2578 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 19:30:52.214247 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205578    2578 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 19:30:52.214247 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205581    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 19:30:52.214247 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205583    2578 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 19:30:52.214247 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205587    2578 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 19:30:52.214247 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205591    2578 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 19:30:52.214247 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205594    2578 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 19:30:52.214247 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205597    2578 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 19:30:52.214247 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205600    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 19:30:52.214247 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205602    2578 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 19:30:52.214247 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205605    2578 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 19:30:52.214247 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205607    2578 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 19:30:52.214247 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205610    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 19:30:52.214247 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205613    2578 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 19:30:52.214247 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205615    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 19:30:52.214247 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205618    2578 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 19:30:52.214703 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205620    2578 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 19:30:52.214703 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205623    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 19:30:52.214703 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205625    2578 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 19:30:52.214703 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205634    2578 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 19:30:52.214703 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205636    2578 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 19:30:52.214703 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205639    2578 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 19:30:52.214703 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.205664    2578 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 19:30:52.214703 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205700    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 19:30:52.214703 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205705    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 19:30:52.214703 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205708    2578 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 19:30:52.214703 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205718    2578 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 19:30:52.214703 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205721    2578 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 19:30:52.214703 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205724    2578 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 19:30:52.214703 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205726    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 19:30:52.214703 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205732    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 19:30:52.215088 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205734    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 19:30:52.215088 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205737    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 19:30:52.215088 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205739    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 19:30:52.215088 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205742    2578 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 19:30:52.215088 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205744    2578 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 19:30:52.215088 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205747    2578 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 19:30:52.215088 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205750    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 19:30:52.215088 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205752    2578 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 19:30:52.215088 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205755    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 19:30:52.215088 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205757    2578 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 19:30:52.215088 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205760    2578 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 19:30:52.215088 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205763    2578 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 19:30:52.215088 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205765    2578 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 19:30:52.215088 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205768    2578 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 19:30:52.215088 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205770    2578 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 19:30:52.215088 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205773    2578 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 19:30:52.215088 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205775    2578 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 19:30:52.215088 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205778    2578 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 19:30:52.215088 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205780    2578 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 19:30:52.215088 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205783    2578 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 19:30:52.215088 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205785    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 19:30:52.215568 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205788    2578 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 19:30:52.215568 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205790    2578 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 19:30:52.215568 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205793    2578 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 19:30:52.215568 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205795    2578 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 19:30:52.215568 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205798    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 19:30:52.215568 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205800    2578 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 19:30:52.215568 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205803    2578 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 19:30:52.215568 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205811    2578 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 19:30:52.215568 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205814    2578 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 19:30:52.215568 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205816    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 19:30:52.215568 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205825    2578 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 19:30:52.215568 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205828    2578 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 19:30:52.215568 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205831    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 19:30:52.215568 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205833    2578 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 19:30:52.215568 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205837    2578 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 19:30:52.215568 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205841    2578 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 19:30:52.215568 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205844    2578 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 19:30:52.215568 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205847    2578 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 19:30:52.215568 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205850    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 19:30:52.216076 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205852    2578 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 19:30:52.216076 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205855    2578 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 19:30:52.216076 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205858    2578 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 19:30:52.216076 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205860    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 19:30:52.216076 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205863    2578 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 19:30:52.216076 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205866    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 19:30:52.216076 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205868    2578 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 19:30:52.216076 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205871    2578 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 19:30:52.216076 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205874    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 19:30:52.216076 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205876    2578 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 19:30:52.216076 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205879    2578 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 19:30:52.216076 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205881    2578 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 19:30:52.216076 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205884    2578 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 19:30:52.216076 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205886    2578 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 19:30:52.216076 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205889    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 19:30:52.216076 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205891    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 19:30:52.216076 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205894    2578 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 19:30:52.216076 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205896    2578 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 19:30:52.216076 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205899    2578 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 19:30:52.216652 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205901    2578 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 19:30:52.216652 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205904    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 19:30:52.216652 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205912    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 19:30:52.216652 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205915    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 19:30:52.216652 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205933    2578 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 19:30:52.216652 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205937    2578 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 19:30:52.216652 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205942    2578 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 19:30:52.216652 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205946    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 19:30:52.216652 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205950    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 19:30:52.216652 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205953    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 19:30:52.216652 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205958    2578 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 19:30:52.216652 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205961    2578 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 19:30:52.216652 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205964    2578 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 19:30:52.216652 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205967    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 19:30:52.216652 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205970    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 19:30:52.216652 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205972    2578 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 19:30:52.216652 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205975    2578 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 19:30:52.216652 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205978    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 19:30:52.216652 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205980    2578 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 19:30:52.217128 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205983    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 19:30:52.217128 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205986    2578 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 19:30:52.217128 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205988    2578 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 19:30:52.217128 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205991    2578 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 19:30:52.217128 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205994    2578 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 19:30:52.217128 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205996    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 19:30:52.217128 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.205999    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 19:30:52.217128 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.206001    2578 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 19:30:52.217128 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.206004    2578 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 19:30:52.217128 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.206006    2578 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 19:30:52.217128 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.206009    2578 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 19:30:52.217128 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.206012    2578 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 19:30:52.217128 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.206014    2578 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 19:30:52.217128 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.206017    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 19:30:52.217128 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.206020    2578 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 19:30:52.217128 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.206022    2578 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 19:30:52.217128 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.206031    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 19:30:52.217128 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.206036    2578 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 19:30:52.217128 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.206040    2578 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 19:30:52.217571 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.206043    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 19:30:52.217571 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.212976    2578 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 19:30:52.217571 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.212992    2578 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 19:30:52.217571 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.214302    2578 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 19:30:52.217571 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.214309    2578 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 19:30:52.217571 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214383    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 19:30:52.217571 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214389    2578 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 19:30:52.217571 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214392    2578 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 19:30:52.217571 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214396    2578 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 19:30:52.217571 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214400    2578 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 19:30:52.217571 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214404    2578 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 19:30:52.217571 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214407    2578 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 19:30:52.217571 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214410    2578 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 19:30:52.217571 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214413    2578 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 19:30:52.217571 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214416    2578 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 19:30:52.217571 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214418    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 19:30:52.217571 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214421    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 19:30:52.217571 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214423    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 19:30:52.217571 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214426    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 19:30:52.217571 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214429    2578 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 19:30:52.218059 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214431    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 19:30:52.218059 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214434    2578 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 19:30:52.218059 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214437    2578 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 19:30:52.218059 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214439    2578 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 19:30:52.218059 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214442    2578 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 19:30:52.218059 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214445    2578 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 19:30:52.218059 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214447    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 19:30:52.218059 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214450    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 19:30:52.218059 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214452    2578 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 19:30:52.218059 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214456    2578 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 19:30:52.218059 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214458    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 19:30:52.218059 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214461    2578 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 19:30:52.218059 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214464    2578 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 19:30:52.218059 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214467    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 19:30:52.218059 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214470    2578 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 19:30:52.218059 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214473    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 19:30:52.218059 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214475    2578 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 19:30:52.218059 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214478    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 19:30:52.218059 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214480    2578 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 19:30:52.218504 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214483    2578 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 19:30:52.218504 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214486    2578 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 19:30:52.218504 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214488    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 19:30:52.218504 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214491    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 19:30:52.218504 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214494    2578 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 19:30:52.218504 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214497    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 19:30:52.218504 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214499    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 19:30:52.218504 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214502    2578 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 19:30:52.218504 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214504    2578 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 19:30:52.218504 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214507    2578 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 19:30:52.218504 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214510    2578 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 19:30:52.218504 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214512    2578 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 19:30:52.218504 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214515    2578 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 19:30:52.218504 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214517    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 19:30:52.218504 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214520    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 19:30:52.218504 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214523    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 19:30:52.218504 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214525    2578 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 19:30:52.218504 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214528    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 19:30:52.218504 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214531    2578 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 19:30:52.218504 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214533    2578 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 19:30:52.219034 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214536    2578 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 19:30:52.219034 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214539    2578 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 19:30:52.219034 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214541    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 19:30:52.219034 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214544    2578 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 19:30:52.219034 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214547    2578 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 19:30:52.219034 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214550    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 19:30:52.219034 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214553    2578 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 19:30:52.219034 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214556    2578 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 19:30:52.219034 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214558    2578 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 19:30:52.219034 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214561    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 19:30:52.219034 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214563    2578 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 19:30:52.219034 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214566    2578 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 19:30:52.219034 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214569    2578 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 19:30:52.219034 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214571    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 19:30:52.219034 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214574    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 19:30:52.219034 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214576    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 19:30:52.219034 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214579    2578 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 19:30:52.219034 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214582    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 19:30:52.219034 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214584    2578 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 19:30:52.219508 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214586    2578 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 19:30:52.219508 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214589    2578 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 19:30:52.219508 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214591    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 19:30:52.219508 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214594    2578 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 19:30:52.219508 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214596    2578 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 19:30:52.219508 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214599    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 19:30:52.219508 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214601    2578 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 19:30:52.219508 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214604    2578 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 19:30:52.219508 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214606    2578 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 19:30:52.219508 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214609    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 19:30:52.219508 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214611    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 19:30:52.219508 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214614    2578 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 19:30:52.219508 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214618    2578 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 19:30:52.219508 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214621    2578 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 19:30:52.219508 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214624    2578 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 19:30:52.219508 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214627    2578 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 19:30:52.219508 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214630    2578 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 19:30:52.219508 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214633    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 19:30:52.219508 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214637    2578 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 19:30:52.219963 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214640    2578 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 19:30:52.219963 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214643    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 19:30:52.219963 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214645    2578 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 19:30:52.219963 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214648    2578 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 19:30:52.219963 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214651    2578 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 19:30:52.219963 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214653    2578 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 19:30:52.219963 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214656    2578 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 19:30:52.219963 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214658    2578 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 19:30:52.219963 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214661    2578 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 19:30:52.219963 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214663    2578 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 19:30:52.219963 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214666    2578 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 19:30:52.219963 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214669    2578 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 19:30:52.219963 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214672    2578 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 19:30:52.219963 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214674    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 19:30:52.219963 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.214703    2578 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 19:30:52.219963 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214841    2578 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 19:30:52.220344 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214846    2578 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 19:30:52.220344 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214848    2578 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 19:30:52.220344 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214851    2578 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 19:30:52.220344 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214853    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 19:30:52.220344 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214856    2578 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 19:30:52.220344 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214859    2578 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 19:30:52.220344 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214861    2578 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 19:30:52.220344 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214864    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 19:30:52.220344 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214866    2578 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 19:30:52.220344 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214869    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 19:30:52.220344 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214871    2578 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 19:30:52.220344 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214874    2578 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 19:30:52.220344 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214877    2578 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 19:30:52.220344 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214879    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 19:30:52.220344 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214882    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 19:30:52.220344 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214885    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 19:30:52.220344 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214888    2578 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 19:30:52.220344 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214890    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 19:30:52.220344 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214892    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 19:30:52.220977 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214897    2578 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 19:30:52.220977 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214900    2578 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 19:30:52.220977 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214903    2578 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 19:30:52.220977 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214906    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 19:30:52.220977 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214908    2578 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 19:30:52.220977 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214911    2578 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 19:30:52.220977 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214913    2578 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 19:30:52.220977 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214916    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 19:30:52.220977 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214918    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 19:30:52.220977 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214934    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 19:30:52.220977 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214937    2578 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 19:30:52.220977 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214939    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 19:30:52.220977 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214942    2578 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 19:30:52.220977 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214944    2578 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 19:30:52.220977 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214947    2578 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 19:30:52.220977 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214949    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 19:30:52.220977 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214952    2578 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 19:30:52.220977 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214954    2578 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 19:30:52.220977 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214957    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 19:30:52.221628 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214959    2578 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 19:30:52.221628 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214962    2578 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 19:30:52.221628 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214964    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 19:30:52.221628 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214966    2578 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 19:30:52.221628 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214969    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 19:30:52.221628 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214972    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 19:30:52.221628 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214974    2578 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 19:30:52.221628 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214977    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 19:30:52.221628 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214979    2578 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 19:30:52.221628 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214982    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 19:30:52.221628 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214985    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 19:30:52.221628 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214987    2578 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 19:30:52.221628 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214990    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 19:30:52.221628 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214993    2578 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 19:30:52.221628 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214995    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 19:30:52.221628 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.214998    2578 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 19:30:52.221628 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215000    2578 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 19:30:52.221628 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215003    2578 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 19:30:52.221628 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215005    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 19:30:52.221628 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215007    2578 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 19:30:52.222162 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215011    2578 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 19:30:52.222162 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215014    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 19:30:52.222162 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215017    2578 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 19:30:52.222162 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215020    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 19:30:52.222162 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215023    2578 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 19:30:52.222162 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215025    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 19:30:52.222162 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215027    2578 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 19:30:52.222162 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215030    2578 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 19:30:52.222162 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215033    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 19:30:52.222162 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215035    2578 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 19:30:52.222162 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215037    2578 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 19:30:52.222162 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215040    2578 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 19:30:52.222162 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215043    2578 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 19:30:52.222162 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215045    2578 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 19:30:52.222162 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215048    2578 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 19:30:52.222162 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215050    2578 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 19:30:52.222162 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215053    2578 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 19:30:52.222162 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215058    2578 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 19:30:52.222162 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215061    2578 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 19:30:52.222722 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215063    2578 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 19:30:52.222722 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215066    2578 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 19:30:52.222722 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215069    2578 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 19:30:52.222722 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215071    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 19:30:52.222722 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215074    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 19:30:52.222722 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215077    2578 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 19:30:52.222722 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215079    2578 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 19:30:52.222722 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215082    2578 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 19:30:52.222722 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215084    2578 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 19:30:52.222722 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215087    2578 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 19:30:52.222722 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215089    2578 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 19:30:52.222722 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215092    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 19:30:52.222722 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215095    2578 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 19:30:52.222722 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215098    2578 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 19:30:52.222722 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215101    2578 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 19:30:52.222722 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215103    2578 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 19:30:52.222722 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215106    2578 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 19:30:52.222722 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215108    2578 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 19:30:52.222722 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215111    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 19:30:52.222722 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215113    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 19:30:52.223251 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215115    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 19:30:52.223251 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215118    2578 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 19:30:52.223251 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215120    2578 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 19:30:52.223251 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215123    2578 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 19:30:52.223251 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215125    2578 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 19:30:52.223251 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215128    2578 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 19:30:52.223251 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215130    2578 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 19:30:52.223251 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215133    2578 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 19:30:52.223251 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.215156    2578 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 19:30:52.223251 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215207    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 19:30:52.223251 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215211    2578 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 19:30:52.223251 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215214    2578 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 19:30:52.223251 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215217    2578 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 19:30:52.223251 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215220    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 19:30:52.223251 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215223    2578 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 19:30:52.223251 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215227    2578 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 19:30:52.223631 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215231    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 19:30:52.223631 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215235    2578 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 19:30:52.223631 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215238    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 19:30:52.223631 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215240    2578 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 19:30:52.223631 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215243    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 19:30:52.223631 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215245    2578 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 19:30:52.223631 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215248    2578 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 19:30:52.223631 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215250    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 19:30:52.223631 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215253    2578 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 19:30:52.223631 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215256    2578 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 19:30:52.223631 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215258    2578 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 19:30:52.223631 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215261    2578 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 19:30:52.223631 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215263    2578 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 19:30:52.223631 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215266    2578 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 19:30:52.223631 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215268    2578 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 19:30:52.223631 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215271    2578 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 19:30:52.223631 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215273    2578 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 19:30:52.223631 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215276    2578 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 19:30:52.223631 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215279    2578 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 19:30:52.223631 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215281    2578 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 19:30:52.224158 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215284    2578 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 19:30:52.224158 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215287    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 19:30:52.224158 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215289    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 19:30:52.224158 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215292    2578 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 19:30:52.224158 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215294    2578 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 19:30:52.224158 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215296    2578 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 19:30:52.224158 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215299    2578 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 19:30:52.224158 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215301    2578 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 19:30:52.224158 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215304    2578 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 19:30:52.224158 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215306    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 19:30:52.224158 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215309    2578 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 19:30:52.224158 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215311    2578 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 19:30:52.224158 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215314    2578 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 19:30:52.224158 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215316    2578 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 19:30:52.224158 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215319    2578 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 19:30:52.224158 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215322    2578 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 19:30:52.224158 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215325    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 19:30:52.224158 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215327    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 19:30:52.224158 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215330    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 19:30:52.224158 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215332    2578 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 19:30:52.224629 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215335    2578 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 19:30:52.224629 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215337    2578 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 19:30:52.224629 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215340    2578 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 19:30:52.224629 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215342    2578 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 19:30:52.224629 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215345    2578 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 19:30:52.224629 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215347    2578 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 19:30:52.224629 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215350    2578 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 19:30:52.224629 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215352    2578 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 19:30:52.224629 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215355    2578 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 19:30:52.224629 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215357    2578 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 19:30:52.224629 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215360    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 19:30:52.224629 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215362    2578 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 19:30:52.224629 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215365    2578 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 19:30:52.224629 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215367    2578 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 19:30:52.224629 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215370    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 19:30:52.224629 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215372    2578 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 19:30:52.224629 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215375    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 19:30:52.224629 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215377    2578 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 19:30:52.224629 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215380    2578 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 19:30:52.224629 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215383    2578 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 19:30:52.225237 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215386    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 19:30:52.225237 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215388    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 19:30:52.225237 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215391    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 19:30:52.225237 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215393    2578 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 19:30:52.225237 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215396    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 19:30:52.225237 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215398    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 19:30:52.225237 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215402    2578 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 19:30:52.225237 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215405    2578 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 19:30:52.225237 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215408    2578 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 19:30:52.225237 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215413    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 19:30:52.225237 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215416    2578 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 19:30:52.225237 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215419    2578 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 19:30:52.225237 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215421    2578 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 19:30:52.225237 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215424    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 19:30:52.225237 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215427    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 19:30:52.225237 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215430    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 19:30:52.225237 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215432    2578 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 19:30:52.225237 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215435    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 19:30:52.225237 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215437    2578 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 19:30:52.225706 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215439    2578 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 19:30:52.225706 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215442    2578 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 19:30:52.225706 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215444    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 19:30:52.225706 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215447    2578 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 19:30:52.225706 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215449    2578 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 19:30:52.225706 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215452    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 19:30:52.225706 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215454    2578 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 19:30:52.225706 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215457    2578 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 19:30:52.225706 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215459    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 19:30:52.225706 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215462    2578 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 19:30:52.225706 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215465    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 19:30:52.225706 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215467    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 19:30:52.225706 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215470    2578 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 19:30:52.225706 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215472    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 19:30:52.225706 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215475    2578 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 19:30:52.225706 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215477    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 19:30:52.225706 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215480    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 19:30:52.225706 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215482    2578 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 19:30:52.225706 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215484    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 19:30:52.225706 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:52.215487    2578 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 19:30:52.226205 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.215537    2578 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 19:30:52.226205 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.217659    2578 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Oct 30 19:30:52.226205 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.218568    2578 server.go:1014] "Starting client certificate rotation"
Oct 30 19:30:52.226205 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.218665    2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 19:30:52.226205 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.218700    2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 19:30:52.226205 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.221771    2578 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 19:30:52.226205 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.223956    2578 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 19:30:52.230823 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.230807    2578 log.go:25] "Validated CRI v1 runtime API"
Oct 30 19:30:52.235423 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.235408    2578 log.go:25] "Validated CRI v1 image API"
Oct 30 19:30:52.236252 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.236224    2578 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 19:30:52.238660 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.238643    2578 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 d6bc9f45-8852-4ac1-b191-801a3d47d7d9:/dev/nvme0n1p4 f03684ec-7bb8-4c81-a0fa-d307748c4205:/dev/nvme0n1p3]
Oct 30 19:30:52.238711 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.238660    2578 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Oct 30 19:30:52.244751 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.244635    2578 manager.go:217] Machine: {Timestamp:2026-10-30 19:30:52.243191603 +0000 UTC m=+0.314531995 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3103864 MemoryCapacity:33164337152 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec264751e003c2da3bc412ca3540ceea SystemUUID:ec264751-e003-c2da-3bc4-12ca3540ceea BootID:60fb3ca4-2a00-4bc5-8550-e209e9d3545b Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:ed:e0:f7:8b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:ed:e0:f7:8b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:1e:fa:2b:9f:ac:ed Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164337152 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 19:30:52.244751 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.244747    2578 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 19:30:52.244875 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.244856    2578 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 19:30:52.247763 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.247737    2578 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 19:30:52.247895 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.247766    2578 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-42.us-east-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 19:30:52.247956 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.247898    2578 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 19:30:52.247956 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.247906    2578 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 19:30:52.247956 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.247955    2578 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 19:30:52.248037 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.247972    2578 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 19:30:52.248721 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.248711    2578 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 19:30:52.252092 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.252080    2578 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 19:30:52.252211 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.252202    2578 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 19:30:52.252912 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.252903    2578 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 19:30:52.252955 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.252916    2578 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 19:30:52.252955 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.252937    2578 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 19:30:52.252955 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.252945    2578 kubelet.go:405] "Adding apiserver pod source"
Oct 30 19:30:52.252955 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.252955    2578 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 19:30:52.253293 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.253275    2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 19:30:52.256853 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.256828    2578 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 19:30:52.260935 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.260902    2578 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 19:30:52.261485 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.261471    2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 19:30:52.262328 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.262315    2578 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 19:30:52.262369 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.262347    2578 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 19:30:52.262518 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.262508    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 19:30:52.262550 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.262524    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 19:30:52.262550 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.262530    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 19:30:52.262550 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.262536    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 19:30:52.262550 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.262541    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 19:30:52.262550 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.262548    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 19:30:52.263148 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.263138    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 19:30:52.263148 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.263148    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 19:30:52.263209 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.263158    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 19:30:52.263209 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.263166    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 19:30:52.263209 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.263176    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 19:30:52.263209 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.263184    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 19:30:52.264464 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.264455    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 19:30:52.264464 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.264464    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 19:30:52.267497 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.267477    2578 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-42.us-east-2.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Oct 30 19:30:52.268218 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.268201    2578 server.go:1263] "Started kubelet"
Oct 30 19:30:52.268352 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.268314    2578 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 19:30:52.268427 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.268372    2578 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 19:30:52.268427 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.268384    2578 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 19:30:52.268959 ip-10-0-141-42 systemd[1]: Started Kubernetes Kubelet.
Oct 30 19:30:52.269521 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.269484    2578 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 19:30:52.272679 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.272536    2578 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 19:30:52.274220 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:30:52.274203    2578 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Oct 30 19:30:52.274714 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.274696    2578 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 19:30:52.274714 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.274702    2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 19:30:52.274832 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:30:52.274808    2578 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-141-42.us-east-2.compute.internal\" not found"
Oct 30 19:30:52.275438 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.275416    2578 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 19:30:52.275438 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.275417    2578 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 19:30:52.275563 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.275444    2578 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 19:30:52.275563 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.275555    2578 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 19:30:52.275636 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.275567    2578 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 19:30:52.275915 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.275899    2578 factory.go:55] Registering systemd factory
Oct 30 19:30:52.276003 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.275933    2578 factory.go:223] Registration of the systemd container factory successfully
Oct 30 19:30:52.276121 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.276104    2578 factory.go:153] Registering CRI-O factory
Oct 30 19:30:52.276121 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.276120    2578 factory.go:223] Registration of the crio container factory successfully
Oct 30 19:30:52.276274 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.276159    2578 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 19:30:52.276274 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.276176    2578 factory.go:103] Registering Raw factory
Oct 30 19:30:52.276274 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.276191    2578 manager.go:1199] Started watching for new ooms in manager
Oct 30 19:30:52.277155 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.277139    2578 manager.go:319] Starting recovery of all containers
Oct 30 19:30:52.282130 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:30:52.282103    2578 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-42.us-east-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Oct 30 19:30:52.283312 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:30:52.283285    2578 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-42.us-east-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node"
Oct 30 19:30:52.283424 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:30:52.282264    2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-42.us-east-2.compute.internal.18e3657a3aa9351a  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-42.us-east-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-42.us-east-2.compute.internal,},FirstTimestamp:2026-10-30 19:30:52.26817257 +0000 UTC m=+0.339512961,LastTimestamp:2026-10-30 19:30:52.26817257 +0000 UTC m=+0.339512961,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-42.us-east-2.compute.internal,}"
Oct 30 19:30:52.283839 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:30:52.283798    2578 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service"
Oct 30 19:30:52.285695 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:30:52.285670    2578 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver"
Oct 30 19:30:52.289156 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.289138    2578 manager.go:324] Recovery completed
Oct 30 19:30:52.290145 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.290121    2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xhcwx"
Oct 30 19:30:52.294008 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.293991    2578 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 19:30:52.298886 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.298869    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-42.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 19:30:52.298981 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.298904    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-42.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 19:30:52.298981 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.298937    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-42.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 19:30:52.299504 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.299492    2578 cpu_manager.go:226] "Starting" policy="none"
Oct 30 19:30:52.299504 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.299499    2578 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 19:30:52.299601 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.299503    2578 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 19:30:52.299601 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.299569    2578 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 19:30:52.300318 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.300293    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-42.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 19:30:52.300395 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.300331    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-42.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 19:30:52.300395 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.300344    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-42.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 19:30:52.301407 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.301387    2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xhcwx"
Oct 30 19:30:52.302167 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:30:52.302084    2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-42.us-east-2.compute.internal.18e3657a3c7de23d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-42.us-east-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-141-42.us-east-2.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-141-42.us-east-2.compute.internal,},FirstTimestamp:2026-10-30 19:30:52.298887741 +0000 UTC m=+0.370228135,LastTimestamp:2026-10-30 19:30:52.298887741 +0000 UTC m=+0.370228135,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-42.us-east-2.compute.internal,}"
Oct 30 19:30:52.302254 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.302206    2578 policy_none.go:50] "Start"
Oct 30 19:30:52.302254 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.302218    2578 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 19:30:52.302254 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.302230    2578 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 19:30:52.304108 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.304093    2578 policy_none.go:44] "Start"
Oct 30 19:30:52.331460 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.331430    2578 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 19:30:52.333746 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.333730    2578 manager.go:341] "Starting Device Plugin manager"
Oct 30 19:30:52.333834 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:30:52.333756    2578 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 19:30:52.333834 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.333765    2578 server.go:87] "Starting device plugin registration server"
Oct 30 19:30:52.334019 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.334005    2578 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 19:30:52.334081 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.334021    2578 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 19:30:52.334122 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.334087    2578 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 19:30:52.334190 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.334156    2578 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 19:30:52.334190 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.334173    2578 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 19:30:52.335111 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:30:52.335088    2578 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 19:30:52.335201 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:30:52.335132    2578 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-42.us-east-2.compute.internal\" not found"
Oct 30 19:30:52.402275 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.402247    2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 19:30:52.403423 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.403407    2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 19:30:52.403512 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.403431    2578 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 19:30:52.403512 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.403453    2578 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 19:30:52.403604 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:30:52.403515    2578 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Oct 30 19:30:52.410498 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.410477    2578 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 19:30:52.436205 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.436174    2578 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 19:30:52.437195 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.437181    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-42.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 19:30:52.437259 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.437207    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-42.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 19:30:52.437259 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.437218    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-42.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 19:30:52.437259 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.437242    2578 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-141-42.us-east-2.compute.internal"
Oct 30 19:30:52.446453 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.446429    2578 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-141-42.us-east-2.compute.internal"
Oct 30 19:30:52.446518 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:30:52.446462    2578 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-42.us-east-2.compute.internal\": node \"ip-10-0-141-42.us-east-2.compute.internal\" not found"
Oct 30 19:30:52.466762 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:30:52.466701    2578 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-141-42.us-east-2.compute.internal\" not found"
Oct 30 19:30:52.504457 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.504209    2578 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-42.us-east-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-42.us-east-2.compute.internal"]
Oct 30 19:30:52.504543 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.504499    2578 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 19:30:52.505369 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.505354    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-42.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 19:30:52.505424 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.505384    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-42.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 19:30:52.505424 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.505403    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-42.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 19:30:52.507571 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.507559    2578 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 19:30:52.507708 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.507693    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-42.us-east-2.compute.internal"
Oct 30 19:30:52.507744 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.507726    2578 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 19:30:52.508276 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.508262    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-42.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 19:30:52.508378 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.508292    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-42.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 19:30:52.508378 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.508313    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-42.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 19:30:52.508378 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.508263    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-42.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 19:30:52.508378 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.508361    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-42.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 19:30:52.508378 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.508373    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-42.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 19:30:52.510463 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.510447    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-42.us-east-2.compute.internal"
Oct 30 19:30:52.510556 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.510479    2578 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 19:30:52.511306 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.511275    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-42.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 19:30:52.511397 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.511322    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-42.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 19:30:52.511397 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.511341    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-42.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 19:30:52.534280 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:30:52.534263    2578 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-42.us-east-2.compute.internal\" not found" node="ip-10-0-141-42.us-east-2.compute.internal"
Oct 30 19:30:52.536315 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:30:52.536291    2578 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-42.us-east-2.compute.internal\" not found" node="ip-10-0-141-42.us-east-2.compute.internal"
Oct 30 19:30:52.567634 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:30:52.567613    2578 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-141-42.us-east-2.compute.internal\" not found"
Oct 30 19:30:52.577036 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.577019    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c73221e6d3bc811e32d70f92ba61e280-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-42.us-east-2.compute.internal\" (UID: \"c73221e6d3bc811e32d70f92ba61e280\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-42.us-east-2.compute.internal"
Oct 30 19:30:52.577132 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.577049    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c73221e6d3bc811e32d70f92ba61e280-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-42.us-east-2.compute.internal\" (UID: \"c73221e6d3bc811e32d70f92ba61e280\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-42.us-east-2.compute.internal"
Oct 30 19:30:52.577132 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.577077    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/cb07204d0709daabd91966727e3665a5-config\") pod \"kube-apiserver-proxy-ip-10-0-141-42.us-east-2.compute.internal\" (UID: \"cb07204d0709daabd91966727e3665a5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-42.us-east-2.compute.internal"
Oct 30 19:30:52.668417 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:30:52.668390    2578 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-141-42.us-east-2.compute.internal\" not found"
Oct 30 19:30:52.677744 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.677717    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c73221e6d3bc811e32d70f92ba61e280-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-42.us-east-2.compute.internal\" (UID: \"c73221e6d3bc811e32d70f92ba61e280\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-42.us-east-2.compute.internal"
Oct 30 19:30:52.677826 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.677753    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/cb07204d0709daabd91966727e3665a5-config\") pod \"kube-apiserver-proxy-ip-10-0-141-42.us-east-2.compute.internal\" (UID: \"cb07204d0709daabd91966727e3665a5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-42.us-east-2.compute.internal"
Oct 30 19:30:52.677826 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.677777    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c73221e6d3bc811e32d70f92ba61e280-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-42.us-east-2.compute.internal\" (UID: \"c73221e6d3bc811e32d70f92ba61e280\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-42.us-east-2.compute.internal"
Oct 30 19:30:52.677826 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.677801    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c73221e6d3bc811e32d70f92ba61e280-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-42.us-east-2.compute.internal\" (UID: \"c73221e6d3bc811e32d70f92ba61e280\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-42.us-east-2.compute.internal"
Oct 30 19:30:52.677826 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.677815    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/cb07204d0709daabd91966727e3665a5-config\") pod \"kube-apiserver-proxy-ip-10-0-141-42.us-east-2.compute.internal\" (UID: \"cb07204d0709daabd91966727e3665a5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-42.us-east-2.compute.internal"
Oct 30 19:30:52.677987 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.677850    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c73221e6d3bc811e32d70f92ba61e280-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-42.us-east-2.compute.internal\" (UID: \"c73221e6d3bc811e32d70f92ba61e280\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-42.us-east-2.compute.internal"
Oct 30 19:30:52.769178 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:30:52.769104    2578 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-141-42.us-east-2.compute.internal\" not found"
Oct 30 19:30:52.837701 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.837673    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-42.us-east-2.compute.internal"
Oct 30 19:30:52.839819 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:52.839803    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-42.us-east-2.compute.internal"
Oct 30 19:30:52.869230 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:30:52.869207    2578 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-141-42.us-east-2.compute.internal\" not found"
Oct 30 19:30:52.969888 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:30:52.969719    2578 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-141-42.us-east-2.compute.internal\" not found"
Oct 30 19:30:53.070239 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:30:53.070169    2578 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-141-42.us-east-2.compute.internal\" not found"
Oct 30 19:30:53.160918 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:53.160864    2578 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 19:30:54.185166 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:30:54.185143    2578 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-141-42.us-east-2.compute.internal\" not found"
Oct 30 19:30:54.233621 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:54.233593    2578 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Oct 30 19:30:54.234140 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:54.233704    2578 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 30 19:30:54.234140 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:54.233743    2578 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 30 19:30:54.285715 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:30:54.285688    2578 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-141-42.us-east-2.compute.internal\" not found"
Oct 30 19:30:54.317440 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:54.317416    2578 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 19:30:54.320816 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:54.320779    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 19:25:53 +0000 UTC" deadline="2028-04-20 17:38:03.527060001 +0000 UTC"
Oct 30 19:30:54.320871 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:54.320817    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12910h7m9.206245536s"
Oct 30 19:30:54.389849 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:54.389821    2578 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-42.us-east-2.compute.internal"
Oct 30 19:30:54.408785 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:54.408763    2578 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 19:30:54.409586 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:54.409572    2578 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-42.us-east-2.compute.internal"
Oct 30 19:30:54.430066 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:54.430043    2578 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 19:30:54.485620 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:54.485585    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb07204d0709daabd91966727e3665a5.slice/crio-df0e67a4af24841460982f934d29e2f26264261184ee6a69067718ae926c739f WatchSource:0}: Error finding container df0e67a4af24841460982f934d29e2f26264261184ee6a69067718ae926c739f: Status 404 returned error can't find the container with id df0e67a4af24841460982f934d29e2f26264261184ee6a69067718ae926c739f
Oct 30 19:30:54.486153 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:54.486133    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc73221e6d3bc811e32d70f92ba61e280.slice/crio-4134771bb05a59b201e34b75eb0925ec7b9e192de9248a9f0161110e6dc89a32 WatchSource:0}: Error finding container 4134771bb05a59b201e34b75eb0925ec7b9e192de9248a9f0161110e6dc89a32: Status 404 returned error can't find the container with id 4134771bb05a59b201e34b75eb0925ec7b9e192de9248a9f0161110e6dc89a32
Oct 30 19:30:54.491110 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:54.491095    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 19:30:54.680497 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:54.680402    2578 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 19:30:55.268744 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.268711    2578 apiserver.go:52] "Watching apiserver"
Oct 30 19:30:55.282928 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.282901    2578 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 19:30:55.283300 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.283279    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-526rb","kube-system/kube-apiserver-proxy-ip-10-0-141-42.us-east-2.compute.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-42.us-east-2.compute.internal","openshift-multus/multus-additional-cni-plugins-9n92x","openshift-network-diagnostics/network-check-target-4f5sl","openshift-network-operator/iptables-alerter-5jq48","kube-system/konnectivity-agent-bx5vk","openshift-dns/node-resolver-859jf","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hts7f","openshift-cluster-node-tuning-operator/tuned-g4c6r","openshift-image-registry/node-ca-j4cdl","openshift-multus/multus-97df4","openshift-multus/network-metrics-daemon-vlqp7","openshift-ovn-kubernetes/ovnkube-node-74lps"]
Oct 30 19:30:55.285851 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.285828    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-bx5vk"
Oct 30 19:30:55.288597 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.288575    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vlqp7"
Oct 30 19:30:55.288701 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:30:55.288646    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vlqp7" podUID="82384cc1-894f-4bb6-9b1d-567c72004575"
Oct 30 19:30:55.290615 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.290592    2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Oct 30 19:30:55.291124 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.291095    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:30:55.291230 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.291133    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 19:30:55.291354 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.291336    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-x9qfs\""
Oct 30 19:30:55.291435 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.291409    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 19:30:55.296218 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.296198    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9n92x"
Oct 30 19:30:55.297531 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.297515    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4f5sl"
Oct 30 19:30:55.297649 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:30:55.297595    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4f5sl" podUID="7ff4ab79-1d7c-49ac-9d63-31772ad68b42"
Oct 30 19:30:55.300368 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.300347    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-5jq48"
Oct 30 19:30:55.300724 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.300702    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fs44x\" (UniqueName: \"kubernetes.io/projected/82384cc1-894f-4bb6-9b1d-567c72004575-kube-api-access-fs44x\") pod \"network-metrics-daemon-vlqp7\" (UID: \"82384cc1-894f-4bb6-9b1d-567c72004575\") " pod="openshift-multus/network-metrics-daemon-vlqp7"
Oct 30 19:30:55.300834 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.300767    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-run-systemd\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:30:55.300834 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.300798    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-etc-openvswitch\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:30:55.300934 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.300860    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-run-ovn\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:30:55.300934 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.300903    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-host-cni-netd\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:30:55.301054 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.300995    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/82384cc1-894f-4bb6-9b1d-567c72004575-metrics-certs\") pod \"network-metrics-daemon-vlqp7\" (UID: \"82384cc1-894f-4bb6-9b1d-567c72004575\") " pod="openshift-multus/network-metrics-daemon-vlqp7"
Oct 30 19:30:55.301054 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.301027    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-host-run-netns\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:30:55.301151 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.301052    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-var-lib-openvswitch\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:30:55.301151 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.301092    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-log-socket\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:30:55.301273 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.301141    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d4d142b9-22f1-427f-a374-ba4918487946-ovnkube-script-lib\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:30:55.301273 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.301175    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-host-kubelet\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:30:55.301273 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.301200    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-systemd-units\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:30:55.301273 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.301235    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-host-run-ovn-kubernetes\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:30:55.301273 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.301270    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d4d142b9-22f1-427f-a374-ba4918487946-env-overrides\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:30:55.301553 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.301313    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d4d142b9-22f1-427f-a374-ba4918487946-ovn-node-metrics-cert\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:30:55.301553 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.301343    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gt5qd\" (UniqueName: \"kubernetes.io/projected/d4d142b9-22f1-427f-a374-ba4918487946-kube-api-access-gt5qd\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:30:55.301553 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.301376    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/17f17805-edc3-4dc5-8e75-819350666927-agent-certs\") pod \"konnectivity-agent-bx5vk\" (UID: \"17f17805-edc3-4dc5-8e75-819350666927\") " pod="kube-system/konnectivity-agent-bx5vk"
Oct 30 19:30:55.301553 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.301400    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/17f17805-edc3-4dc5-8e75-819350666927-konnectivity-ca\") pod \"konnectivity-agent-bx5vk\" (UID: \"17f17805-edc3-4dc5-8e75-819350666927\") " pod="kube-system/konnectivity-agent-bx5vk"
Oct 30 19:30:55.301553 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.301424    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-host-slash\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:30:55.301553 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.301447    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-run-openvswitch\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:30:55.301553 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.301510    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-node-log\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:30:55.301553 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.301534    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-host-cni-bin\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:30:55.301907 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.301558    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:30:55.301907 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.301581    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d4d142b9-22f1-427f-a374-ba4918487946-ovnkube-config\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:30:55.303223 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.303205    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hts7f"
Oct 30 19:30:55.305893 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.305874    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-526rb"
Oct 30 19:30:55.305996 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:30:55.305952    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-526rb" podUID="141cf3eb-d99b-4876-b65a-a2847584ff5d"
Oct 30 19:30:55.308044 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.308025    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-859jf"
Oct 30 19:30:55.310600 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.310579    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:30:55.312042 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.311245    2578 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 19:30:55.314662 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.314642    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-j4cdl"
Oct 30 19:30:55.316606 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.316586    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-97df4"
Oct 30 19:30:55.328663 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.328643    2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-xzjjd"
Oct 30 19:30:55.340983 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.340773    2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-xzjjd"
Oct 30 19:30:55.391441 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.391412    2578 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 19:30:55.391771 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.391718    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 19:30:55.392026 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.392006    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 19:30:55.392133 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.392039    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 19:30:55.392133 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.392054    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 19:30:55.392133 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.392087    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 19:30:55.392133 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.392100    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 19:30:55.392133 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.392108    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 19:30:55.392338 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.392315    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 19:30:55.392373 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.392344    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 19:30:55.394661 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.394302    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 19:30:55.394661 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.394404    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 19:30:55.394661 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.394527    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 19:30:55.394661 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.394571    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 19:30:55.394661 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.394635    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 19:30:55.394958 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.394875    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 19:30:55.394958 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.394918    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 19:30:55.395062 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.394975    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 19:30:55.395062 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.395010    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 19:30:55.395062 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.395023    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 19:30:55.396643 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.396622    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 19:30:55.396730 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.396667    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 19:30:55.396776 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.396741    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 19:30:55.397230 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.397209    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 19:30:55.397230 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.397240    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-4mthb\""
Oct 30 19:30:55.397230 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.397283    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 19:30:55.397475 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.397358    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-g4c9f\""
Oct 30 19:30:55.398967 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.398913    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-7c6bv\""
Oct 30 19:30:55.398967 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.398934    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-7mr7r\""
Oct 30 19:30:55.399105 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.399053    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-79gww\""
Oct 30 19:30:55.399105 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.399078    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 19:30:55.399105 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.399086    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-vcqml\""
Oct 30 19:30:55.399105 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.399105    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-ztltl\""
Oct 30 19:30:55.399244 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.399134    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-5r9wt\""
Oct 30 19:30:55.402231 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.402200    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/63678441-271f-46d7-ac1d-57a60bb78b12-cni-binary-copy\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:30:55.402327 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.402236    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-host-var-lib-cni-multus\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:30:55.402327 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.402262    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-multus-conf-dir\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:30:55.402327 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.402287    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-etc-kubernetes\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:30:55.402506 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.402363    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0100ec89-6366-4946-99ce-4a254850c8fd-iptables-alerter-script\") pod \"iptables-alerter-5jq48\" (UID: \"0100ec89-6366-4946-99ce-4a254850c8fd\") " pod="openshift-network-operator/iptables-alerter-5jq48"
Oct 30 19:30:55.402506 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.402438    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-etc-sysctl-d\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:30:55.402506 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.402466    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-multus-socket-dir-parent\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:30:55.402664 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.402523    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d0036363-bf0e-44d7-b285-44546ac729e6-registration-dir\") pod \"aws-ebs-csi-driver-node-hts7f\" (UID: \"d0036363-bf0e-44d7-b285-44546ac729e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hts7f"
Oct 30 19:30:55.402664 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.402550    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d0036363-bf0e-44d7-b285-44546ac729e6-device-dir\") pod \"aws-ebs-csi-driver-node-hts7f\" (UID: \"d0036363-bf0e-44d7-b285-44546ac729e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hts7f"
Oct 30 19:30:55.402664 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.402596    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/17f17805-edc3-4dc5-8e75-819350666927-agent-certs\") pod \"konnectivity-agent-bx5vk\" (UID: \"17f17805-edc3-4dc5-8e75-819350666927\") " pod="kube-system/konnectivity-agent-bx5vk"
Oct 30 19:30:55.402664 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.402622    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhsf4\" (UniqueName: \"kubernetes.io/projected/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-kube-api-access-mhsf4\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:30:55.402866 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.402675    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6eca1a1e-101a-43bb-9fde-505c3182c10a-host\") pod \"node-ca-j4cdl\" (UID: \"6eca1a1e-101a-43bb-9fde-505c3182c10a\") " pod="openshift-image-registry/node-ca-j4cdl"
Oct 30 19:30:55.402866 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.402699    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-multus-cni-dir\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:30:55.402866 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.402722    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1977f3d7-c7f6-44cc-8a59-9d4f35adc0f7-hosts-file\") pod \"node-resolver-859jf\" (UID: \"1977f3d7-c7f6-44cc-8a59-9d4f35adc0f7\") " pod="openshift-dns/node-resolver-859jf"
Oct 30 19:30:55.402866 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.402763    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-run-systemd\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:30:55.402866 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.402788    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nn6bt\" (UniqueName: \"kubernetes.io/projected/7ff4ab79-1d7c-49ac-9d63-31772ad68b42-kube-api-access-nn6bt\") pod \"network-check-target-4f5sl\" (UID: \"7ff4ab79-1d7c-49ac-9d63-31772ad68b42\") " pod="openshift-network-diagnostics/network-check-target-4f5sl"
Oct 30 19:30:55.402866 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.402811    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-host-run-netns\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:30:55.402866 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.402850    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/79ea5164-32e2-4505-b92a-1e50495a6492-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9n92x\" (UID: \"79ea5164-32e2-4505-b92a-1e50495a6492\") " pod="openshift-multus/multus-additional-cni-plugins-9n92x"
Oct 30 19:30:55.403188 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.402877    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d0036363-bf0e-44d7-b285-44546ac729e6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hts7f\" (UID: \"d0036363-bf0e-44d7-b285-44546ac729e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hts7f"
Oct 30 19:30:55.403188 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.402892    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/82384cc1-894f-4bb6-9b1d-567c72004575-metrics-certs\") pod \"network-metrics-daemon-vlqp7\" (UID: \"82384cc1-894f-4bb6-9b1d-567c72004575\") " pod="openshift-multus/network-metrics-daemon-vlqp7"
Oct 30 19:30:55.403188 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.402951    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d4d142b9-22f1-427f-a374-ba4918487946-ovnkube-script-lib\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:30:55.403188 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.402975    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gt5qd\" (UniqueName: \"kubernetes.io/projected/d4d142b9-22f1-427f-a374-ba4918487946-kube-api-access-gt5qd\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:30:55.403188 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.402999    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-systemd-units\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:30:55.403188 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.403016    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bls4\" (UniqueName: \"kubernetes.io/projected/6eca1a1e-101a-43bb-9fde-505c3182c10a-kube-api-access-8bls4\") pod \"node-ca-j4cdl\" (UID: \"6eca1a1e-101a-43bb-9fde-505c3182c10a\") " pod="openshift-image-registry/node-ca-j4cdl"
Oct 30 19:30:55.403188 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.403032    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-host-slash\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:30:55.403188 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.403050    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-host-cni-bin\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:30:55.403188 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.403081    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:30:55.403188 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.403096    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-run\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:30:55.403188 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.403118    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-lib-modules\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:30:55.403188 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.403142    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-hostroot\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:30:55.403188 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.403158    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fs44x\" (UniqueName: \"kubernetes.io/projected/82384cc1-894f-4bb6-9b1d-567c72004575-kube-api-access-fs44x\") pod \"network-metrics-daemon-vlqp7\" (UID: \"82384cc1-894f-4bb6-9b1d-567c72004575\") " pod="openshift-multus/network-metrics-daemon-vlqp7"
Oct 30 19:30:55.403188 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.403187    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-sys\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:30:55.403845 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.403262    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-host\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:30:55.403845 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.403286    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-etc-tuned\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:30:55.403845 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.403304    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-host-run-k8s-cni-cncf-io\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:30:55.403845 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.403327    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-host-var-lib-cni-bin\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:30:55.403845 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.403343    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n87xw\" (UniqueName: \"kubernetes.io/projected/79ea5164-32e2-4505-b92a-1e50495a6492-kube-api-access-n87xw\") pod \"multus-additional-cni-plugins-9n92x\" (UID: \"79ea5164-32e2-4505-b92a-1e50495a6492\") " pod="openshift-multus/multus-additional-cni-plugins-9n92x"
Oct 30 19:30:55.403845 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.403358    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d0036363-bf0e-44d7-b285-44546ac729e6-socket-dir\") pod \"aws-ebs-csi-driver-node-hts7f\" (UID: \"d0036363-bf0e-44d7-b285-44546ac729e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hts7f"
Oct 30 19:30:55.403845 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.403373    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-etc-systemd\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:30:55.403845 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.403404    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-host-run-multus-certs\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:30:55.403845 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.403427    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d0036363-bf0e-44d7-b285-44546ac729e6-etc-selinux\") pod \"aws-ebs-csi-driver-node-hts7f\" (UID: \"d0036363-bf0e-44d7-b285-44546ac729e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hts7f"
Oct 30 19:30:55.403845 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.403444    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1977f3d7-c7f6-44cc-8a59-9d4f35adc0f7-tmp-dir\") pod \"node-resolver-859jf\" (UID: \"1977f3d7-c7f6-44cc-8a59-9d4f35adc0f7\") " pod="openshift-dns/node-resolver-859jf"
Oct 30 19:30:55.403845 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.403461    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-host-run-ovn-kubernetes\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:30:55.403845 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.403491    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d4d142b9-22f1-427f-a374-ba4918487946-ovn-node-metrics-cert\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:30:55.403845 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.403514    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-host-var-lib-kubelet\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:30:55.403845 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.403539    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/79ea5164-32e2-4505-b92a-1e50495a6492-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9n92x\" (UID: \"79ea5164-32e2-4505-b92a-1e50495a6492\") " pod="openshift-multus/multus-additional-cni-plugins-9n92x"
Oct 30 19:30:55.403845 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.403556    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-host-slash\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:30:55.403845 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.403562    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d0036363-bf0e-44d7-b285-44546ac729e6-sys-fs\") pod \"aws-ebs-csi-driver-node-hts7f\" (UID: \"d0036363-bf0e-44d7-b285-44546ac729e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hts7f"
Oct 30 19:30:55.404397 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.403607    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/17f17805-edc3-4dc5-8e75-819350666927-konnectivity-ca\") pod \"konnectivity-agent-bx5vk\" (UID: \"17f17805-edc3-4dc5-8e75-819350666927\") " pod="kube-system/konnectivity-agent-bx5vk"
Oct 30 19:30:55.404397 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.403609    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-run-systemd\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:30:55.404397 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.403668    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-host-cni-bin\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:30:55.404397 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.403705    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-host-run-ovn-kubernetes\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:30:55.404397 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.404057    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-systemd-units\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:30:55.404397 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.404066    2578 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 19:30:55.404397 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.404152    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d4d142b9-22f1-427f-a374-ba4918487946-ovnkube-script-lib\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:30:55.404397 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:30:55.404231    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 19:30:55.404397 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.404261    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-run-openvswitch\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:30:55.404397 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.403709    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:30:55.404802 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:30:55.404588    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82384cc1-894f-4bb6-9b1d-567c72004575-metrics-certs podName:82384cc1-894f-4bb6-9b1d-567c72004575 nodeName:}" failed. No retries permitted until 2026-10-30 19:30:55.904551492 +0000 UTC m=+2.961334945 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/82384cc1-894f-4bb6-9b1d-567c72004575-metrics-certs") pod "network-metrics-daemon-vlqp7" (UID: "82384cc1-894f-4bb6-9b1d-567c72004575") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 19:30:55.404802 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.404593    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-run-openvswitch\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:30:55.404802 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.404613    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-node-log\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:30:55.404802 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.404651    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d4d142b9-22f1-427f-a374-ba4918487946-ovnkube-config\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:30:55.404802 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.404705    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-node-log\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:30:55.404802 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.404739    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-cnibin\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:30:55.404802 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.404772    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mwb6\" (UniqueName: \"kubernetes.io/projected/63678441-271f-46d7-ac1d-57a60bb78b12-kube-api-access-9mwb6\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:30:55.404802 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.404790    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/79ea5164-32e2-4505-b92a-1e50495a6492-cnibin\") pod \"multus-additional-cni-plugins-9n92x\" (UID: \"79ea5164-32e2-4505-b92a-1e50495a6492\") " pod="openshift-multus/multus-additional-cni-plugins-9n92x"
Oct 30 19:30:55.405111 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.404855    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-etc-openvswitch\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:30:55.405111 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.404890    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-run-ovn\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:30:55.405111 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.404957    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-host-cni-netd\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:30:55.405111 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.404985    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-var-lib-kubelet\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:30:55.405111 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.405006    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/17f17805-edc3-4dc5-8e75-819350666927-konnectivity-ca\") pod \"konnectivity-agent-bx5vk\" (UID: \"17f17805-edc3-4dc5-8e75-819350666927\") " pod="kube-system/konnectivity-agent-bx5vk"
Oct 30 19:30:55.405111 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.405013    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-system-cni-dir\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:30:55.405111 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.405042    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxjvg\" (UniqueName: \"kubernetes.io/projected/d0036363-bf0e-44d7-b285-44546ac729e6-kube-api-access-pxjvg\") pod \"aws-ebs-csi-driver-node-hts7f\" (UID: \"d0036363-bf0e-44d7-b285-44546ac729e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hts7f"
Oct 30 19:30:55.405111 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.405084    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-host-cni-netd\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:30:55.405111 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.405082    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-run-ovn\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:30:55.405457 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.405116    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-etc-openvswitch\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:30:55.405457 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.405140    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4x29s\" (UniqueName: \"kubernetes.io/projected/1977f3d7-c7f6-44cc-8a59-9d4f35adc0f7-kube-api-access-4x29s\") pod \"node-resolver-859jf\" (UID: \"1977f3d7-c7f6-44cc-8a59-9d4f35adc0f7\") " pod="openshift-dns/node-resolver-859jf"
Oct 30 19:30:55.405457 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.405179    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d4d142b9-22f1-427f-a374-ba4918487946-ovnkube-config\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:30:55.405457 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.405214    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-var-lib-openvswitch\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:30:55.405457 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.405247    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-var-lib-openvswitch\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:30:55.405457 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.405270    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-log-socket\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:30:55.405457 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.405307    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-log-socket\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:30:55.405457 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.405352    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/63678441-271f-46d7-ac1d-57a60bb78b12-multus-daemon-config\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:30:55.405457 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.405442    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/141cf3eb-d99b-4876-b65a-a2847584ff5d-dbus\") pod \"global-pull-secret-syncer-526rb\" (UID: \"141cf3eb-d99b-4876-b65a-a2847584ff5d\") " pod="kube-system/global-pull-secret-syncer-526rb"
Oct 30 19:30:55.405873 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.405469    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/141cf3eb-d99b-4876-b65a-a2847584ff5d-original-pull-secret\") pod \"global-pull-secret-syncer-526rb\" (UID: \"141cf3eb-d99b-4876-b65a-a2847584ff5d\") " pod="kube-system/global-pull-secret-syncer-526rb"
Oct 30 19:30:55.405873 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.405592    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-host-kubelet\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:30:55.405873 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.405642    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-host-kubelet\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:30:55.405873 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.405665    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d4d142b9-22f1-427f-a374-ba4918487946-env-overrides\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:30:55.405873 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.405706    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-etc-modprobe-d\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:30:55.405873 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.405793    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-etc-kubernetes\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:30:55.405873 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.405811    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-tmp\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:30:55.405873 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.405870    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0100ec89-6366-4946-99ce-4a254850c8fd-host-slash\") pod \"iptables-alerter-5jq48\" (UID: \"0100ec89-6366-4946-99ce-4a254850c8fd\") " pod="openshift-network-operator/iptables-alerter-5jq48"
Oct 30 19:30:55.406255 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.405914    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/141cf3eb-d99b-4876-b65a-a2847584ff5d-kubelet-config\") pod \"global-pull-secret-syncer-526rb\" (UID: \"141cf3eb-d99b-4876-b65a-a2847584ff5d\") " pod="kube-system/global-pull-secret-syncer-526rb"
Oct 30 19:30:55.406255 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.405970    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-etc-sysctl-conf\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:30:55.406255 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.405998    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6eca1a1e-101a-43bb-9fde-505c3182c10a-serviceca\") pod \"node-ca-j4cdl\" (UID: \"6eca1a1e-101a-43bb-9fde-505c3182c10a\") " pod="openshift-image-registry/node-ca-j4cdl"
Oct 30 19:30:55.406255 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.406023    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/79ea5164-32e2-4505-b92a-1e50495a6492-system-cni-dir\") pod \"multus-additional-cni-plugins-9n92x\" (UID: \"79ea5164-32e2-4505-b92a-1e50495a6492\") " pod="openshift-multus/multus-additional-cni-plugins-9n92x"
Oct 30 19:30:55.406255 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.406032    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d4d142b9-22f1-427f-a374-ba4918487946-env-overrides\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:30:55.406255 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.406071    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/79ea5164-32e2-4505-b92a-1e50495a6492-cni-binary-copy\") pod \"multus-additional-cni-plugins-9n92x\" (UID: \"79ea5164-32e2-4505-b92a-1e50495a6492\") " pod="openshift-multus/multus-additional-cni-plugins-9n92x"
Oct 30 19:30:55.406255 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.406103    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/79ea5164-32e2-4505-b92a-1e50495a6492-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9n92x\" (UID: \"79ea5164-32e2-4505-b92a-1e50495a6492\") " pod="openshift-multus/multus-additional-cni-plugins-9n92x"
Oct 30 19:30:55.406255 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.406136    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gsb6w\" (UniqueName: \"kubernetes.io/projected/0100ec89-6366-4946-99ce-4a254850c8fd-kube-api-access-gsb6w\") pod \"iptables-alerter-5jq48\" (UID: \"0100ec89-6366-4946-99ce-4a254850c8fd\") " pod="openshift-network-operator/iptables-alerter-5jq48"
Oct 30 19:30:55.406255 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.406172    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-host-run-netns\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:30:55.406255 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.406196    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-host-run-netns\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:30:55.406255 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.406212    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-etc-sysconfig\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:30:55.408218 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.408196    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d4d142b9-22f1-427f-a374-ba4918487946-ovn-node-metrics-cert\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:30:55.408315 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.408233    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/17f17805-edc3-4dc5-8e75-819350666927-agent-certs\") pod \"konnectivity-agent-bx5vk\" (UID: \"17f17805-edc3-4dc5-8e75-819350666927\") " pod="kube-system/konnectivity-agent-bx5vk"
Oct 30 19:30:55.418449 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.418427    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gt5qd\" (UniqueName: \"kubernetes.io/projected/d4d142b9-22f1-427f-a374-ba4918487946-kube-api-access-gt5qd\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:30:55.418732 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.418700    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fs44x\" (UniqueName: \"kubernetes.io/projected/82384cc1-894f-4bb6-9b1d-567c72004575-kube-api-access-fs44x\") pod \"network-metrics-daemon-vlqp7\" (UID: \"82384cc1-894f-4bb6-9b1d-567c72004575\") " pod="openshift-multus/network-metrics-daemon-vlqp7"
Oct 30 19:30:55.422696 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.422653    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-42.us-east-2.compute.internal" event={"ID":"c73221e6d3bc811e32d70f92ba61e280","Type":"ContainerStarted","Data":"4134771bb05a59b201e34b75eb0925ec7b9e192de9248a9f0161110e6dc89a32"}
Oct 30 19:30:55.423912 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.423889    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-42.us-east-2.compute.internal" event={"ID":"cb07204d0709daabd91966727e3665a5","Type":"ContainerStarted","Data":"df0e67a4af24841460982f934d29e2f26264261184ee6a69067718ae926c739f"}
Oct 30 19:30:55.425471 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.425438    2578 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 19:30:55.506739 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.506712    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-tmp\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:30:55.506739 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.506744    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0100ec89-6366-4946-99ce-4a254850c8fd-host-slash\") pod \"iptables-alerter-5jq48\" (UID: \"0100ec89-6366-4946-99ce-4a254850c8fd\") " pod="openshift-network-operator/iptables-alerter-5jq48"
Oct 30 19:30:55.507023 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.506761    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/141cf3eb-d99b-4876-b65a-a2847584ff5d-kubelet-config\") pod \"global-pull-secret-syncer-526rb\" (UID: \"141cf3eb-d99b-4876-b65a-a2847584ff5d\") " pod="kube-system/global-pull-secret-syncer-526rb"
Oct 30 19:30:55.507023 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.506776    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-etc-sysctl-conf\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:30:55.507023 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.506872    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/141cf3eb-d99b-4876-b65a-a2847584ff5d-kubelet-config\") pod \"global-pull-secret-syncer-526rb\" (UID: \"141cf3eb-d99b-4876-b65a-a2847584ff5d\") " pod="kube-system/global-pull-secret-syncer-526rb"
Oct 30 19:30:55.507023 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.506938    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0100ec89-6366-4946-99ce-4a254850c8fd-host-slash\") pod \"iptables-alerter-5jq48\" (UID: \"0100ec89-6366-4946-99ce-4a254850c8fd\") " pod="openshift-network-operator/iptables-alerter-5jq48"
Oct 30 19:30:55.507023 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.506975    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6eca1a1e-101a-43bb-9fde-505c3182c10a-serviceca\") pod \"node-ca-j4cdl\" (UID: \"6eca1a1e-101a-43bb-9fde-505c3182c10a\") " pod="openshift-image-registry/node-ca-j4cdl"
Oct 30 19:30:55.507023 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.506990    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-etc-sysctl-conf\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:30:55.507023 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.507002    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/79ea5164-32e2-4505-b92a-1e50495a6492-system-cni-dir\") pod \"multus-additional-cni-plugins-9n92x\" (UID: \"79ea5164-32e2-4505-b92a-1e50495a6492\") " pod="openshift-multus/multus-additional-cni-plugins-9n92x"
Oct 30 19:30:55.507328 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.507027    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/79ea5164-32e2-4505-b92a-1e50495a6492-cni-binary-copy\") pod \"multus-additional-cni-plugins-9n92x\" (UID: \"79ea5164-32e2-4505-b92a-1e50495a6492\") " pod="openshift-multus/multus-additional-cni-plugins-9n92x"
Oct 30 19:30:55.507328 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.507052    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/79ea5164-32e2-4505-b92a-1e50495a6492-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9n92x\" (UID: \"79ea5164-32e2-4505-b92a-1e50495a6492\") " pod="openshift-multus/multus-additional-cni-plugins-9n92x"
Oct 30 19:30:55.507328 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.507074    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsb6w\" (UniqueName: \"kubernetes.io/projected/0100ec89-6366-4946-99ce-4a254850c8fd-kube-api-access-gsb6w\") pod \"iptables-alerter-5jq48\" (UID: \"0100ec89-6366-4946-99ce-4a254850c8fd\") " pod="openshift-network-operator/iptables-alerter-5jq48"
Oct 30 19:30:55.507328 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.507099    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-etc-sysconfig\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:30:55.507328 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.507084    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/79ea5164-32e2-4505-b92a-1e50495a6492-system-cni-dir\") pod \"multus-additional-cni-plugins-9n92x\" (UID: \"79ea5164-32e2-4505-b92a-1e50495a6492\") " pod="openshift-multus/multus-additional-cni-plugins-9n92x"
Oct 30 19:30:55.507328 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.507127    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/63678441-271f-46d7-ac1d-57a60bb78b12-cni-binary-copy\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:30:55.507328 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.507151    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-host-var-lib-cni-multus\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:30:55.507328 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.507198    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-host-var-lib-cni-multus\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:30:55.507560 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.507397    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6eca1a1e-101a-43bb-9fde-505c3182c10a-serviceca\") pod \"node-ca-j4cdl\" (UID: \"6eca1a1e-101a-43bb-9fde-505c3182c10a\") " pod="openshift-image-registry/node-ca-j4cdl"
Oct 30 19:30:55.507560 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.507506    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-etc-sysconfig\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:30:55.507560 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.507542    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-multus-conf-dir\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:30:55.507652 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.507576    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-etc-kubernetes\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:30:55.507652 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.507593    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/79ea5164-32e2-4505-b92a-1e50495a6492-cni-binary-copy\") pod \"multus-additional-cni-plugins-9n92x\" (UID: \"79ea5164-32e2-4505-b92a-1e50495a6492\") " pod="openshift-multus/multus-additional-cni-plugins-9n92x"
Oct 30 19:30:55.507652 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.507600    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0100ec89-6366-4946-99ce-4a254850c8fd-iptables-alerter-script\") pod \"iptables-alerter-5jq48\" (UID: \"0100ec89-6366-4946-99ce-4a254850c8fd\") " pod="openshift-network-operator/iptables-alerter-5jq48"
Oct 30 19:30:55.507652 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.507595    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/79ea5164-32e2-4505-b92a-1e50495a6492-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9n92x\" (UID: \"79ea5164-32e2-4505-b92a-1e50495a6492\") " pod="openshift-multus/multus-additional-cni-plugins-9n92x"
Oct 30 19:30:55.507652 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.507644    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-etc-sysctl-d\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:30:55.507853 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.507658    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-multus-conf-dir\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:30:55.507853 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.507671    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-multus-socket-dir-parent\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:30:55.507853 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.507684    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-etc-kubernetes\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:30:55.507853 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.507699    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d0036363-bf0e-44d7-b285-44546ac729e6-registration-dir\") pod \"aws-ebs-csi-driver-node-hts7f\" (UID: \"d0036363-bf0e-44d7-b285-44546ac729e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hts7f"
Oct 30 19:30:55.507853 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.507724    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d0036363-bf0e-44d7-b285-44546ac729e6-device-dir\") pod \"aws-ebs-csi-driver-node-hts7f\" (UID: \"d0036363-bf0e-44d7-b285-44546ac729e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hts7f"
Oct 30 19:30:55.507853 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.507749    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhsf4\" (UniqueName: \"kubernetes.io/projected/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-kube-api-access-mhsf4\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:30:55.507853 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.507772    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6eca1a1e-101a-43bb-9fde-505c3182c10a-host\") pod \"node-ca-j4cdl\" (UID: \"6eca1a1e-101a-43bb-9fde-505c3182c10a\") " pod="openshift-image-registry/node-ca-j4cdl"
Oct 30 19:30:55.507853 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.507779    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-multus-socket-dir-parent\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:30:55.507853 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.507797    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-multus-cni-dir\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:30:55.507853 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.507803    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d0036363-bf0e-44d7-b285-44546ac729e6-registration-dir\") pod \"aws-ebs-csi-driver-node-hts7f\" (UID: \"d0036363-bf0e-44d7-b285-44546ac729e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hts7f"
Oct 30 19:30:55.508297 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.507860    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6eca1a1e-101a-43bb-9fde-505c3182c10a-host\") pod \"node-ca-j4cdl\" (UID: \"6eca1a1e-101a-43bb-9fde-505c3182c10a\") " pod="openshift-image-registry/node-ca-j4cdl"
Oct 30 19:30:55.508297 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.507862    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1977f3d7-c7f6-44cc-8a59-9d4f35adc0f7-hosts-file\") pod \"node-resolver-859jf\" (UID: \"1977f3d7-c7f6-44cc-8a59-9d4f35adc0f7\") " pod="openshift-dns/node-resolver-859jf"
Oct 30 19:30:55.508297 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.507903    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1977f3d7-c7f6-44cc-8a59-9d4f35adc0f7-hosts-file\") pod \"node-resolver-859jf\" (UID: \"1977f3d7-c7f6-44cc-8a59-9d4f35adc0f7\") " pod="openshift-dns/node-resolver-859jf"
Oct 30 19:30:55.508297 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.507904    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nn6bt\" (UniqueName: \"kubernetes.io/projected/7ff4ab79-1d7c-49ac-9d63-31772ad68b42-kube-api-access-nn6bt\") pod \"network-check-target-4f5sl\" (UID: \"7ff4ab79-1d7c-49ac-9d63-31772ad68b42\") " pod="openshift-network-diagnostics/network-check-target-4f5sl"
Oct 30 19:30:55.508297 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.507938    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-multus-cni-dir\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:30:55.508297 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.507945    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-host-run-netns\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:30:55.508297 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.507982    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-host-run-netns\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:30:55.508297 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.508003    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/79ea5164-32e2-4505-b92a-1e50495a6492-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9n92x\" (UID: \"79ea5164-32e2-4505-b92a-1e50495a6492\") " pod="openshift-multus/multus-additional-cni-plugins-9n92x"
Oct 30 19:30:55.508297 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.508048    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/63678441-271f-46d7-ac1d-57a60bb78b12-cni-binary-copy\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:30:55.508297 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.508051    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d0036363-bf0e-44d7-b285-44546ac729e6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hts7f\" (UID: \"d0036363-bf0e-44d7-b285-44546ac729e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hts7f"
Oct 30 19:30:55.508297 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.507811    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-etc-sysctl-d\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:30:55.508297 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.508154    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0100ec89-6366-4946-99ce-4a254850c8fd-iptables-alerter-script\") pod \"iptables-alerter-5jq48\" (UID: \"0100ec89-6366-4946-99ce-4a254850c8fd\") " pod="openshift-network-operator/iptables-alerter-5jq48"
Oct 30 19:30:55.508297 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.508174    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d0036363-bf0e-44d7-b285-44546ac729e6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hts7f\" (UID: \"d0036363-bf0e-44d7-b285-44546ac729e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hts7f"
Oct 30 19:30:55.508297 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.508213    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bls4\" (UniqueName: \"kubernetes.io/projected/6eca1a1e-101a-43bb-9fde-505c3182c10a-kube-api-access-8bls4\") pod \"node-ca-j4cdl\" (UID: \"6eca1a1e-101a-43bb-9fde-505c3182c10a\") " pod="openshift-image-registry/node-ca-j4cdl"
Oct 30 19:30:55.508297 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.508234    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d0036363-bf0e-44d7-b285-44546ac729e6-device-dir\") pod \"aws-ebs-csi-driver-node-hts7f\" (UID: \"d0036363-bf0e-44d7-b285-44546ac729e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hts7f"
Oct 30 19:30:55.508297 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.508241    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-run\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:30:55.508297 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.508272    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-lib-modules\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:30:55.508297 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.508297    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-hostroot\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:30:55.509185 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.508323    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-sys\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:30:55.509185 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.508275    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-run\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:30:55.509185 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.508347    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-host\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:30:55.509185 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.508405    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-sys\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:30:55.509185 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.508415    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-host\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:30:55.509185 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.508430    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-etc-tuned\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:30:55.509185 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.508437    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-lib-modules\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:30:55.509185 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.508469    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-hostroot\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:30:55.509185 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.508501    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/79ea5164-32e2-4505-b92a-1e50495a6492-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9n92x\" (UID: \"79ea5164-32e2-4505-b92a-1e50495a6492\") " pod="openshift-multus/multus-additional-cni-plugins-9n92x"
Oct 30 19:30:55.509185 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.508515    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-host-run-k8s-cni-cncf-io\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:30:55.509185 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.508557    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-host-var-lib-cni-bin\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:30:55.509185 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.508637    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-host-var-lib-cni-bin\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:30:55.509848 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.509548    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n87xw\" (UniqueName: \"kubernetes.io/projected/79ea5164-32e2-4505-b92a-1e50495a6492-kube-api-access-n87xw\") pod \"multus-additional-cni-plugins-9n92x\" (UID: \"79ea5164-32e2-4505-b92a-1e50495a6492\") " pod="openshift-multus/multus-additional-cni-plugins-9n92x"
Oct 30 19:30:55.509895 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.509880    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d0036363-bf0e-44d7-b285-44546ac729e6-socket-dir\") pod \"aws-ebs-csi-driver-node-hts7f\" (UID: \"d0036363-bf0e-44d7-b285-44546ac729e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hts7f"
Oct 30 19:30:55.509927 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.509911    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-etc-systemd\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:30:55.509958 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.509934    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-host-run-multus-certs\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:30:55.509958 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.509950    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d0036363-bf0e-44d7-b285-44546ac729e6-etc-selinux\") pod \"aws-ebs-csi-driver-node-hts7f\" (UID: \"d0036363-bf0e-44d7-b285-44546ac729e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hts7f"
Oct 30 19:30:55.510017 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.509972    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1977f3d7-c7f6-44cc-8a59-9d4f35adc0f7-tmp-dir\") pod \"node-resolver-859jf\" (UID: \"1977f3d7-c7f6-44cc-8a59-9d4f35adc0f7\") " pod="openshift-dns/node-resolver-859jf"
Oct 30 19:30:55.510017 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.509993    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-host-var-lib-kubelet\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:30:55.510071 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.510017    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/79ea5164-32e2-4505-b92a-1e50495a6492-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9n92x\" (UID: \"79ea5164-32e2-4505-b92a-1e50495a6492\") " pod="openshift-multus/multus-additional-cni-plugins-9n92x"
Oct 30 19:30:55.510071 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.510036    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d0036363-bf0e-44d7-b285-44546ac729e6-sys-fs\") pod \"aws-ebs-csi-driver-node-hts7f\" (UID: \"d0036363-bf0e-44d7-b285-44546ac729e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hts7f"
Oct 30 19:30:55.510071 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.510065    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-cnibin\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:30:55.510155 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.510087    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mwb6\" (UniqueName: \"kubernetes.io/projected/63678441-271f-46d7-ac1d-57a60bb78b12-kube-api-access-9mwb6\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:30:55.510155 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.510105    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/79ea5164-32e2-4505-b92a-1e50495a6492-cnibin\") pod \"multus-additional-cni-plugins-9n92x\" (UID: \"79ea5164-32e2-4505-b92a-1e50495a6492\") " pod="openshift-multus/multus-additional-cni-plugins-9n92x"
Oct 30 19:30:55.510155 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.510132    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-var-lib-kubelet\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:30:55.510155 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.510147    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-system-cni-dir\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:30:55.510261 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.510166    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxjvg\" (UniqueName: \"kubernetes.io/projected/d0036363-bf0e-44d7-b285-44546ac729e6-kube-api-access-pxjvg\") pod \"aws-ebs-csi-driver-node-hts7f\" (UID: \"d0036363-bf0e-44d7-b285-44546ac729e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hts7f"
Oct 30 19:30:55.510261 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.510188    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4x29s\" (UniqueName: \"kubernetes.io/projected/1977f3d7-c7f6-44cc-8a59-9d4f35adc0f7-kube-api-access-4x29s\") pod \"node-resolver-859jf\" (UID: \"1977f3d7-c7f6-44cc-8a59-9d4f35adc0f7\") " pod="openshift-dns/node-resolver-859jf"
Oct 30 19:30:55.510261 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.510211    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/63678441-271f-46d7-ac1d-57a60bb78b12-multus-daemon-config\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:30:55.510261 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.510230    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/141cf3eb-d99b-4876-b65a-a2847584ff5d-dbus\") pod \"global-pull-secret-syncer-526rb\" (UID: \"141cf3eb-d99b-4876-b65a-a2847584ff5d\") " pod="kube-system/global-pull-secret-syncer-526rb"
Oct 30 19:30:55.510261 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.510247    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/141cf3eb-d99b-4876-b65a-a2847584ff5d-original-pull-secret\") pod \"global-pull-secret-syncer-526rb\" (UID: \"141cf3eb-d99b-4876-b65a-a2847584ff5d\") " pod="kube-system/global-pull-secret-syncer-526rb"
Oct 30 19:30:55.510396 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.510267    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-etc-modprobe-d\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:30:55.510396 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.510285    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-etc-kubernetes\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:30:55.510450 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.510396    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-etc-kubernetes\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:30:55.510524 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.510493    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d0036363-bf0e-44d7-b285-44546ac729e6-socket-dir\") pod \"aws-ebs-csi-driver-node-hts7f\" (UID: \"d0036363-bf0e-44d7-b285-44546ac729e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hts7f"
Oct 30 19:30:55.510604 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.510580    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-etc-systemd\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:30:55.510655 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.510633    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-host-run-multus-certs\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:30:55.510746 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.510729    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d0036363-bf0e-44d7-b285-44546ac729e6-etc-selinux\") pod \"aws-ebs-csi-driver-node-hts7f\" (UID: \"d0036363-bf0e-44d7-b285-44546ac729e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hts7f"
Oct 30 19:30:55.511214 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.511012    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-var-lib-kubelet\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:30:55.511214 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.511099    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-host-var-lib-kubelet\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:30:55.514183 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.511552    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-system-cni-dir\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:30:55.514183 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.511768    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1977f3d7-c7f6-44cc-8a59-9d4f35adc0f7-tmp-dir\") pod \"node-resolver-859jf\" (UID: \"1977f3d7-c7f6-44cc-8a59-9d4f35adc0f7\") " pod="openshift-dns/node-resolver-859jf"
Oct 30 19:30:55.514183 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.511867    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/79ea5164-32e2-4505-b92a-1e50495a6492-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9n92x\" (UID: \"79ea5164-32e2-4505-b92a-1e50495a6492\") " pod="openshift-multus/multus-additional-cni-plugins-9n92x"
Oct 30 19:30:55.514183 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.511905    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/63678441-271f-46d7-ac1d-57a60bb78b12-multus-daemon-config\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:30:55.514183 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.511953    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d0036363-bf0e-44d7-b285-44546ac729e6-sys-fs\") pod \"aws-ebs-csi-driver-node-hts7f\" (UID: \"d0036363-bf0e-44d7-b285-44546ac729e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hts7f"
Oct 30 19:30:55.514183 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.512018    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/79ea5164-32e2-4505-b92a-1e50495a6492-cnibin\") pod \"multus-additional-cni-plugins-9n92x\" (UID: \"79ea5164-32e2-4505-b92a-1e50495a6492\") " pod="openshift-multus/multus-additional-cni-plugins-9n92x"
Oct 30 19:30:55.514183 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.512094    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/141cf3eb-d99b-4876-b65a-a2847584ff5d-dbus\") pod \"global-pull-secret-syncer-526rb\" (UID: \"141cf3eb-d99b-4876-b65a-a2847584ff5d\") " pod="kube-system/global-pull-secret-syncer-526rb"
Oct 30 19:30:55.514183 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:30:55.512159    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 19:30:55.514183 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:30:55.512227    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/141cf3eb-d99b-4876-b65a-a2847584ff5d-original-pull-secret podName:141cf3eb-d99b-4876-b65a-a2847584ff5d nodeName:}" failed. No retries permitted until 2026-10-30 19:30:56.012202535 +0000 UTC m=+3.068985985 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/141cf3eb-d99b-4876-b65a-a2847584ff5d-original-pull-secret") pod "global-pull-secret-syncer-526rb" (UID: "141cf3eb-d99b-4876-b65a-a2847584ff5d") : object "kube-system"/"original-pull-secret" not registered
Oct 30 19:30:55.514183 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.512249    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-etc-modprobe-d\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:30:55.514183 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.512159    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-cnibin\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:30:55.514183 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.508549    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-host-run-k8s-cni-cncf-io\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:30:55.514820 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.514367    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-tmp\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:30:55.516818 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.516796    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-etc-tuned\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:30:55.519335 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:30:55.519288    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 19:30:55.519335 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:30:55.519317    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 19:30:55.519335 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:30:55.519332    2578 projected.go:196] Error preparing data for projected volume kube-api-access-nn6bt for pod openshift-network-diagnostics/network-check-target-4f5sl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 19:30:55.519551 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:30:55.519421    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7ff4ab79-1d7c-49ac-9d63-31772ad68b42-kube-api-access-nn6bt podName:7ff4ab79-1d7c-49ac-9d63-31772ad68b42 nodeName:}" failed. No retries permitted until 2026-10-30 19:30:56.019402692 +0000 UTC m=+3.076186147 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-nn6bt" (UniqueName: "kubernetes.io/projected/7ff4ab79-1d7c-49ac-9d63-31772ad68b42-kube-api-access-nn6bt") pod "network-check-target-4f5sl" (UID: "7ff4ab79-1d7c-49ac-9d63-31772ad68b42") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 19:30:55.520755 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.520713    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gsb6w\" (UniqueName: \"kubernetes.io/projected/0100ec89-6366-4946-99ce-4a254850c8fd-kube-api-access-gsb6w\") pod \"iptables-alerter-5jq48\" (UID: \"0100ec89-6366-4946-99ce-4a254850c8fd\") " pod="openshift-network-operator/iptables-alerter-5jq48"
Oct 30 19:30:55.524635 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.522747    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhsf4\" (UniqueName: \"kubernetes.io/projected/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-kube-api-access-mhsf4\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:30:55.524635 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.523902    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bls4\" (UniqueName: \"kubernetes.io/projected/6eca1a1e-101a-43bb-9fde-505c3182c10a-kube-api-access-8bls4\") pod \"node-ca-j4cdl\" (UID: \"6eca1a1e-101a-43bb-9fde-505c3182c10a\") " pod="openshift-image-registry/node-ca-j4cdl"
Oct 30 19:30:55.525624 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.525583    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxjvg\" (UniqueName: \"kubernetes.io/projected/d0036363-bf0e-44d7-b285-44546ac729e6-kube-api-access-pxjvg\") pod \"aws-ebs-csi-driver-node-hts7f\" (UID: \"d0036363-bf0e-44d7-b285-44546ac729e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hts7f"
Oct 30 19:30:55.525955 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.525912    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n87xw\" (UniqueName: \"kubernetes.io/projected/79ea5164-32e2-4505-b92a-1e50495a6492-kube-api-access-n87xw\") pod \"multus-additional-cni-plugins-9n92x\" (UID: \"79ea5164-32e2-4505-b92a-1e50495a6492\") " pod="openshift-multus/multus-additional-cni-plugins-9n92x"
Oct 30 19:30:55.529216 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.529164    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mwb6\" (UniqueName: \"kubernetes.io/projected/63678441-271f-46d7-ac1d-57a60bb78b12-kube-api-access-9mwb6\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:30:55.530529 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.530511    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4x29s\" (UniqueName: \"kubernetes.io/projected/1977f3d7-c7f6-44cc-8a59-9d4f35adc0f7-kube-api-access-4x29s\") pod \"node-resolver-859jf\" (UID: \"1977f3d7-c7f6-44cc-8a59-9d4f35adc0f7\") " pod="openshift-dns/node-resolver-859jf"
Oct 30 19:30:55.602765 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.602734    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-bx5vk"
Oct 30 19:30:55.613849 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:55.613822    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17f17805_edc3_4dc5_8e75_819350666927.slice/crio-e2ef6d2d62c1316cad7cb0a0c15c8d10909426f11d5ce3aa1ace901fda021b50 WatchSource:0}: Error finding container e2ef6d2d62c1316cad7cb0a0c15c8d10909426f11d5ce3aa1ace901fda021b50: Status 404 returned error can't find the container with id e2ef6d2d62c1316cad7cb0a0c15c8d10909426f11d5ce3aa1ace901fda021b50
Oct 30 19:30:55.631251 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.631223    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-5jq48"
Oct 30 19:30:55.641543 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:55.641515    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0100ec89_6366_4946_99ce_4a254850c8fd.slice/crio-a78bea986ecd7304024f9866af6d58af75a3209bb96fe5f1644b4e1c6e2d07cb WatchSource:0}: Error finding container a78bea986ecd7304024f9866af6d58af75a3209bb96fe5f1644b4e1c6e2d07cb: Status 404 returned error can't find the container with id a78bea986ecd7304024f9866af6d58af75a3209bb96fe5f1644b4e1c6e2d07cb
Oct 30 19:30:55.642440 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.642420    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:30:55.645583 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.645559    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-859jf"
Oct 30 19:30:55.651753 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.651734    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9n92x"
Oct 30 19:30:55.652047 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:55.652024    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4d142b9_22f1_427f_a374_ba4918487946.slice/crio-ffc43bb54a4e85852ad1661bee7e08beb0d0b0ce2c01f742abd0c63e16bb9525 WatchSource:0}: Error finding container ffc43bb54a4e85852ad1661bee7e08beb0d0b0ce2c01f742abd0c63e16bb9525: Status 404 returned error can't find the container with id ffc43bb54a4e85852ad1661bee7e08beb0d0b0ce2c01f742abd0c63e16bb9525
Oct 30 19:30:55.653917 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.653817    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:30:55.655543 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:55.655520    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1977f3d7_c7f6_44cc_8a59_9d4f35adc0f7.slice/crio-86c49726545dad2c141d39e101d778e9462326e082fa11f86d0353997aab0e7b WatchSource:0}: Error finding container 86c49726545dad2c141d39e101d778e9462326e082fa11f86d0353997aab0e7b: Status 404 returned error can't find the container with id 86c49726545dad2c141d39e101d778e9462326e082fa11f86d0353997aab0e7b
Oct 30 19:30:55.656288 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.656257    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hts7f"
Oct 30 19:30:55.659500 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.659468    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-j4cdl"
Oct 30 19:30:55.662199 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.661975    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-97df4"
Oct 30 19:30:55.665682 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:55.665659    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79ea5164_32e2_4505_b92a_1e50495a6492.slice/crio-dc6a849d14e5d5769caf5ac8d09449b75867ed2072639589d9d4bcf42e9eff11 WatchSource:0}: Error finding container dc6a849d14e5d5769caf5ac8d09449b75867ed2072639589d9d4bcf42e9eff11: Status 404 returned error can't find the container with id dc6a849d14e5d5769caf5ac8d09449b75867ed2072639589d9d4bcf42e9eff11
Oct 30 19:30:55.671742 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:55.671717    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0036363_bf0e_44d7_b285_44546ac729e6.slice/crio-622e7a0cce2a941e66b9921201ac3a833f7d138ac2a11a7a730d0851aa315991 WatchSource:0}: Error finding container 622e7a0cce2a941e66b9921201ac3a833f7d138ac2a11a7a730d0851aa315991: Status 404 returned error can't find the container with id 622e7a0cce2a941e66b9921201ac3a833f7d138ac2a11a7a730d0851aa315991
Oct 30 19:30:55.676402 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:30:55.676378    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6eca1a1e_101a_43bb_9fde_505c3182c10a.slice/crio-cc9f277b2b9cf4478544dfdbbe924957afc897a82e1b27ae0133c7a94cc6ce3e WatchSource:0}: Error finding container cc9f277b2b9cf4478544dfdbbe924957afc897a82e1b27ae0133c7a94cc6ce3e: Status 404 returned error can't find the container with id cc9f277b2b9cf4478544dfdbbe924957afc897a82e1b27ae0133c7a94cc6ce3e
Oct 30 19:30:55.913627 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:55.913592    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/82384cc1-894f-4bb6-9b1d-567c72004575-metrics-certs\") pod \"network-metrics-daemon-vlqp7\" (UID: \"82384cc1-894f-4bb6-9b1d-567c72004575\") " pod="openshift-multus/network-metrics-daemon-vlqp7"
Oct 30 19:30:55.913775 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:30:55.913753    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 19:30:55.913855 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:30:55.913826    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82384cc1-894f-4bb6-9b1d-567c72004575-metrics-certs podName:82384cc1-894f-4bb6-9b1d-567c72004575 nodeName:}" failed. No retries permitted until 2026-10-30 19:30:56.913806364 +0000 UTC m=+3.970589803 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/82384cc1-894f-4bb6-9b1d-567c72004575-metrics-certs") pod "network-metrics-daemon-vlqp7" (UID: "82384cc1-894f-4bb6-9b1d-567c72004575") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 19:30:56.014649 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:56.014617    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/141cf3eb-d99b-4876-b65a-a2847584ff5d-original-pull-secret\") pod \"global-pull-secret-syncer-526rb\" (UID: \"141cf3eb-d99b-4876-b65a-a2847584ff5d\") " pod="kube-system/global-pull-secret-syncer-526rb"
Oct 30 19:30:56.014793 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:30:56.014755    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 19:30:56.014828 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:30:56.014815    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/141cf3eb-d99b-4876-b65a-a2847584ff5d-original-pull-secret podName:141cf3eb-d99b-4876-b65a-a2847584ff5d nodeName:}" failed. No retries permitted until 2026-10-30 19:30:57.014801377 +0000 UTC m=+4.071584830 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/141cf3eb-d99b-4876-b65a-a2847584ff5d-original-pull-secret") pod "global-pull-secret-syncer-526rb" (UID: "141cf3eb-d99b-4876-b65a-a2847584ff5d") : object "kube-system"/"original-pull-secret" not registered
Oct 30 19:30:56.115213 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:56.115038    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nn6bt\" (UniqueName: \"kubernetes.io/projected/7ff4ab79-1d7c-49ac-9d63-31772ad68b42-kube-api-access-nn6bt\") pod \"network-check-target-4f5sl\" (UID: \"7ff4ab79-1d7c-49ac-9d63-31772ad68b42\") " pod="openshift-network-diagnostics/network-check-target-4f5sl"
Oct 30 19:30:56.115278 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:30:56.115210    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 19:30:56.115278 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:30:56.115236    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 19:30:56.115278 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:30:56.115250    2578 projected.go:196] Error preparing data for projected volume kube-api-access-nn6bt for pod openshift-network-diagnostics/network-check-target-4f5sl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 19:30:56.115416 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:30:56.115303    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7ff4ab79-1d7c-49ac-9d63-31772ad68b42-kube-api-access-nn6bt podName:7ff4ab79-1d7c-49ac-9d63-31772ad68b42 nodeName:}" failed. No retries permitted until 2026-10-30 19:30:57.115285228 +0000 UTC m=+4.172068670 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-nn6bt" (UniqueName: "kubernetes.io/projected/7ff4ab79-1d7c-49ac-9d63-31772ad68b42-kube-api-access-nn6bt") pod "network-check-target-4f5sl" (UID: "7ff4ab79-1d7c-49ac-9d63-31772ad68b42") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 19:30:56.343858 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:56.343742    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 19:25:55 +0000 UTC" deadline="2028-04-20 22:17:47.030343501 +0000 UTC"
Oct 30 19:30:56.343858 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:56.343798    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12914h46m50.68654893s"
Oct 30 19:30:56.429390 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:56.429298    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hts7f" event={"ID":"d0036363-bf0e-44d7-b285-44546ac729e6","Type":"ContainerStarted","Data":"622e7a0cce2a941e66b9921201ac3a833f7d138ac2a11a7a730d0851aa315991"}
Oct 30 19:30:56.435889 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:56.435815    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-859jf" event={"ID":"1977f3d7-c7f6-44cc-8a59-9d4f35adc0f7","Type":"ContainerStarted","Data":"86c49726545dad2c141d39e101d778e9462326e082fa11f86d0353997aab0e7b"}
Oct 30 19:30:56.443167 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:56.443104    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-bx5vk" event={"ID":"17f17805-edc3-4dc5-8e75-819350666927","Type":"ContainerStarted","Data":"e2ef6d2d62c1316cad7cb0a0c15c8d10909426f11d5ce3aa1ace901fda021b50"}
Oct 30 19:30:56.449275 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:56.449194    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-j4cdl" event={"ID":"6eca1a1e-101a-43bb-9fde-505c3182c10a","Type":"ContainerStarted","Data":"cc9f277b2b9cf4478544dfdbbe924957afc897a82e1b27ae0133c7a94cc6ce3e"}
Oct 30 19:30:56.459990 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:56.459912    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-g4c6r" event={"ID":"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e","Type":"ContainerStarted","Data":"d54eada4817d21788217ce5317a0858beae67944013b1ff75418f8c061a5854e"}
Oct 30 19:30:56.489427 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:56.489190    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9n92x" event={"ID":"79ea5164-32e2-4505-b92a-1e50495a6492","Type":"ContainerStarted","Data":"dc6a849d14e5d5769caf5ac8d09449b75867ed2072639589d9d4bcf42e9eff11"}
Oct 30 19:30:56.503563 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:56.503531    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-74lps" event={"ID":"d4d142b9-22f1-427f-a374-ba4918487946","Type":"ContainerStarted","Data":"ffc43bb54a4e85852ad1661bee7e08beb0d0b0ce2c01f742abd0c63e16bb9525"}
Oct 30 19:30:56.512860 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:56.508503    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-5jq48" event={"ID":"0100ec89-6366-4946-99ce-4a254850c8fd","Type":"ContainerStarted","Data":"a78bea986ecd7304024f9866af6d58af75a3209bb96fe5f1644b4e1c6e2d07cb"}
Oct 30 19:30:56.515459 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:56.515429    2578 generic.go:356] "Generic (PLEG): container finished" podID="c73221e6d3bc811e32d70f92ba61e280" containerID="fe5d517e6465dec695facc2299abdf8f8b576c4000b73562db9ae5441a2c5b1c" exitCode=0
Oct 30 19:30:56.515572 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:56.515523    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-42.us-east-2.compute.internal" event={"ID":"c73221e6d3bc811e32d70f92ba61e280","Type":"ContainerDied","Data":"fe5d517e6465dec695facc2299abdf8f8b576c4000b73562db9ae5441a2c5b1c"}
Oct 30 19:30:56.532321 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:56.532296    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-42.us-east-2.compute.internal" event={"ID":"cb07204d0709daabd91966727e3665a5","Type":"ContainerStarted","Data":"5a7d3c6d6b1a4185e00b14684f86ae2d2b8ffb7a2b04deb308868529763314df"}
Oct 30 19:30:56.553068 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:56.553034    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-97df4" event={"ID":"63678441-271f-46d7-ac1d-57a60bb78b12","Type":"ContainerStarted","Data":"24754be54bc89f364cfe432df10c2c6d7d47a52f7a9313105640ce82a6e5a54e"}
Oct 30 19:30:56.556515 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:56.556430    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-42.us-east-2.compute.internal" podStartSLOduration=2.556414804 podStartE2EDuration="2.556414804s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:30:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:56.555568141 +0000 UTC m=+3.612351597" watchObservedRunningTime="2026-10-30 19:30:56.556414804 +0000 UTC m=+3.613198263"
Oct 30 19:30:56.922150 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:56.921467    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/82384cc1-894f-4bb6-9b1d-567c72004575-metrics-certs\") pod \"network-metrics-daemon-vlqp7\" (UID: \"82384cc1-894f-4bb6-9b1d-567c72004575\") " pod="openshift-multus/network-metrics-daemon-vlqp7"
Oct 30 19:30:56.922150 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:30:56.921685    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 19:30:56.922150 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:30:56.921747    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82384cc1-894f-4bb6-9b1d-567c72004575-metrics-certs podName:82384cc1-894f-4bb6-9b1d-567c72004575 nodeName:}" failed. No retries permitted until 2026-10-30 19:30:58.921728673 +0000 UTC m=+5.978512126 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/82384cc1-894f-4bb6-9b1d-567c72004575-metrics-certs") pod "network-metrics-daemon-vlqp7" (UID: "82384cc1-894f-4bb6-9b1d-567c72004575") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 19:30:57.022610 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:57.022495    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/141cf3eb-d99b-4876-b65a-a2847584ff5d-original-pull-secret\") pod \"global-pull-secret-syncer-526rb\" (UID: \"141cf3eb-d99b-4876-b65a-a2847584ff5d\") " pod="kube-system/global-pull-secret-syncer-526rb"
Oct 30 19:30:57.022773 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:30:57.022721    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 19:30:57.022840 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:30:57.022780    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/141cf3eb-d99b-4876-b65a-a2847584ff5d-original-pull-secret podName:141cf3eb-d99b-4876-b65a-a2847584ff5d nodeName:}" failed. No retries permitted until 2026-10-30 19:30:59.022762277 +0000 UTC m=+6.079545728 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/141cf3eb-d99b-4876-b65a-a2847584ff5d-original-pull-secret") pod "global-pull-secret-syncer-526rb" (UID: "141cf3eb-d99b-4876-b65a-a2847584ff5d") : object "kube-system"/"original-pull-secret" not registered
Oct 30 19:30:57.123664 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:57.123626    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nn6bt\" (UniqueName: \"kubernetes.io/projected/7ff4ab79-1d7c-49ac-9d63-31772ad68b42-kube-api-access-nn6bt\") pod \"network-check-target-4f5sl\" (UID: \"7ff4ab79-1d7c-49ac-9d63-31772ad68b42\") " pod="openshift-network-diagnostics/network-check-target-4f5sl"
Oct 30 19:30:57.123890 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:30:57.123875    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 19:30:57.123961 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:30:57.123897    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 19:30:57.123961 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:30:57.123910    2578 projected.go:196] Error preparing data for projected volume kube-api-access-nn6bt for pod openshift-network-diagnostics/network-check-target-4f5sl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 19:30:57.124063 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:30:57.123965    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7ff4ab79-1d7c-49ac-9d63-31772ad68b42-kube-api-access-nn6bt podName:7ff4ab79-1d7c-49ac-9d63-31772ad68b42 nodeName:}" failed. No retries permitted until 2026-10-30 19:30:59.12394879 +0000 UTC m=+6.180732227 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-nn6bt" (UniqueName: "kubernetes.io/projected/7ff4ab79-1d7c-49ac-9d63-31772ad68b42-kube-api-access-nn6bt") pod "network-check-target-4f5sl" (UID: "7ff4ab79-1d7c-49ac-9d63-31772ad68b42") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 19:30:57.192294 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:57.192259    2578 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 19:30:57.344319 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:57.344176    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 19:25:55 +0000 UTC" deadline="2028-03-26 22:35:23.086604745 +0000 UTC"
Oct 30 19:30:57.344319 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:57.344235    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12315h4m25.742374731s"
Oct 30 19:30:57.422049 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:57.422017    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vlqp7"
Oct 30 19:30:57.422200 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:30:57.422164    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vlqp7" podUID="82384cc1-894f-4bb6-9b1d-567c72004575"
Oct 30 19:30:57.422601 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:57.422580    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4f5sl"
Oct 30 19:30:57.422707 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:30:57.422679    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4f5sl" podUID="7ff4ab79-1d7c-49ac-9d63-31772ad68b42"
Oct 30 19:30:57.422768 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:57.422741    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-526rb"
Oct 30 19:30:57.422843 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:30:57.422813    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-526rb" podUID="141cf3eb-d99b-4876-b65a-a2847584ff5d"
Oct 30 19:30:57.568850 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:57.568811    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-42.us-east-2.compute.internal" event={"ID":"c73221e6d3bc811e32d70f92ba61e280","Type":"ContainerStarted","Data":"47d27f3fb2ed66891a28fba8ffe3f81bc65634aad05104cba1f0678caa5b2171"}
Oct 30 19:30:58.579586 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:58.579550    2578 generic.go:356] "Generic (PLEG): container finished" podID="79ea5164-32e2-4505-b92a-1e50495a6492" containerID="5f9af21001b3bd286e77635d95130794e36b2aedad876c3f3206c4b306002b04" exitCode=0
Oct 30 19:30:58.580661 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:58.580628    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9n92x" event={"ID":"79ea5164-32e2-4505-b92a-1e50495a6492","Type":"ContainerDied","Data":"5f9af21001b3bd286e77635d95130794e36b2aedad876c3f3206c4b306002b04"}
Oct 30 19:30:58.612125 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:58.611752    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-42.us-east-2.compute.internal" podStartSLOduration=4.611733313 podStartE2EDuration="4.611733313s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:30:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:57.590586942 +0000 UTC m=+4.647370402" watchObservedRunningTime="2026-10-30 19:30:58.611733313 +0000 UTC m=+5.668516772"
Oct 30 19:30:58.936972 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:58.936934    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/82384cc1-894f-4bb6-9b1d-567c72004575-metrics-certs\") pod \"network-metrics-daemon-vlqp7\" (UID: \"82384cc1-894f-4bb6-9b1d-567c72004575\") " pod="openshift-multus/network-metrics-daemon-vlqp7"
Oct 30 19:30:58.937178 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:30:58.937161    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 19:30:58.937249 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:30:58.937229    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82384cc1-894f-4bb6-9b1d-567c72004575-metrics-certs podName:82384cc1-894f-4bb6-9b1d-567c72004575 nodeName:}" failed. No retries permitted until 2026-10-30 19:31:02.937211864 +0000 UTC m=+9.993995315 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/82384cc1-894f-4bb6-9b1d-567c72004575-metrics-certs") pod "network-metrics-daemon-vlqp7" (UID: "82384cc1-894f-4bb6-9b1d-567c72004575") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 19:30:59.037568 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:59.037530    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/141cf3eb-d99b-4876-b65a-a2847584ff5d-original-pull-secret\") pod \"global-pull-secret-syncer-526rb\" (UID: \"141cf3eb-d99b-4876-b65a-a2847584ff5d\") " pod="kube-system/global-pull-secret-syncer-526rb"
Oct 30 19:30:59.037745 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:30:59.037725    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 19:30:59.037824 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:30:59.037789    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/141cf3eb-d99b-4876-b65a-a2847584ff5d-original-pull-secret podName:141cf3eb-d99b-4876-b65a-a2847584ff5d nodeName:}" failed. No retries permitted until 2026-10-30 19:31:03.037772359 +0000 UTC m=+10.094555798 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/141cf3eb-d99b-4876-b65a-a2847584ff5d-original-pull-secret") pod "global-pull-secret-syncer-526rb" (UID: "141cf3eb-d99b-4876-b65a-a2847584ff5d") : object "kube-system"/"original-pull-secret" not registered
Oct 30 19:30:59.138598 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:59.138561    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nn6bt\" (UniqueName: \"kubernetes.io/projected/7ff4ab79-1d7c-49ac-9d63-31772ad68b42-kube-api-access-nn6bt\") pod \"network-check-target-4f5sl\" (UID: \"7ff4ab79-1d7c-49ac-9d63-31772ad68b42\") " pod="openshift-network-diagnostics/network-check-target-4f5sl"
Oct 30 19:30:59.138775 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:30:59.138742    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 19:30:59.138775 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:30:59.138764    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 19:30:59.138887 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:30:59.138779    2578 projected.go:196] Error preparing data for projected volume kube-api-access-nn6bt for pod openshift-network-diagnostics/network-check-target-4f5sl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 19:30:59.138887 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:30:59.138837    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7ff4ab79-1d7c-49ac-9d63-31772ad68b42-kube-api-access-nn6bt podName:7ff4ab79-1d7c-49ac-9d63-31772ad68b42 nodeName:}" failed. No retries permitted until 2026-10-30 19:31:03.138819119 +0000 UTC m=+10.195602573 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-nn6bt" (UniqueName: "kubernetes.io/projected/7ff4ab79-1d7c-49ac-9d63-31772ad68b42-kube-api-access-nn6bt") pod "network-check-target-4f5sl" (UID: "7ff4ab79-1d7c-49ac-9d63-31772ad68b42") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 19:30:59.418823 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:59.418687    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vlqp7"
Oct 30 19:30:59.418906 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:30:59.418821    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vlqp7" podUID="82384cc1-894f-4bb6-9b1d-567c72004575"
Oct 30 19:30:59.418906 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:59.418886    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4f5sl"
Oct 30 19:30:59.419026 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:30:59.418983    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4f5sl" podUID="7ff4ab79-1d7c-49ac-9d63-31772ad68b42"
Oct 30 19:30:59.419081 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:30:59.419029    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-526rb"
Oct 30 19:30:59.419132 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:30:59.419097    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-526rb" podUID="141cf3eb-d99b-4876-b65a-a2847584ff5d"
Oct 30 19:31:00.589711 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:00.589668    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-bx5vk" event={"ID":"17f17805-edc3-4dc5-8e75-819350666927","Type":"ContainerStarted","Data":"c014899b1d87f5006f7b646fe1db50d21037be39918695ae67dcbf85db976d14"}
Oct 30 19:31:00.611839 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:00.611778    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-bx5vk" podStartSLOduration=3.815432573 podStartE2EDuration="7.611760316s" totalImagesPullingTime="3.796327743s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:31:00.611309853 +0000 UTC m=+7.668093361" watchObservedRunningTime="2026-10-30 19:31:00.611760316 +0000 UTC m=+7.668543774"
Oct 30 19:31:01.424559 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:01.424429    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4f5sl"
Oct 30 19:31:01.425612 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:01.424915    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4f5sl" podUID="7ff4ab79-1d7c-49ac-9d63-31772ad68b42"
Oct 30 19:31:01.425612 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:01.425302    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-526rb"
Oct 30 19:31:01.425612 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:01.425397    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-526rb" podUID="141cf3eb-d99b-4876-b65a-a2847584ff5d"
Oct 30 19:31:01.425612 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:01.425463    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vlqp7"
Oct 30 19:31:01.425612 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:01.425574    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vlqp7" podUID="82384cc1-894f-4bb6-9b1d-567c72004575"
Oct 30 19:31:01.596703 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:01.596670    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-j4cdl" event={"ID":"6eca1a1e-101a-43bb-9fde-505c3182c10a","Type":"ContainerStarted","Data":"69a59cb5387abb412d50f7d23448ef421dd3da522fce4858bea00aa96189622f"}
Oct 30 19:31:01.615918 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:01.615851    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-j4cdl" podStartSLOduration=3.785778805 podStartE2EDuration="8.615833103s" totalImagesPullingTime="4.830054298s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:30:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:31:01.615301557 +0000 UTC m=+8.672085016" watchObservedRunningTime="2026-10-30 19:31:01.615833103 +0000 UTC m=+8.672616563"
Oct 30 19:31:02.601649 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:02.601567    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hts7f" event={"ID":"d0036363-bf0e-44d7-b285-44546ac729e6","Type":"ContainerStarted","Data":"d805b3fb32cef9778671de3d347c96f3dcc34f98e6f799ef7d961be5d736ab57"}
Oct 30 19:31:02.972003 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:02.971926    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/82384cc1-894f-4bb6-9b1d-567c72004575-metrics-certs\") pod \"network-metrics-daemon-vlqp7\" (UID: \"82384cc1-894f-4bb6-9b1d-567c72004575\") " pod="openshift-multus/network-metrics-daemon-vlqp7"
Oct 30 19:31:02.972148 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:02.972073    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 19:31:02.972148 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:02.972136    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82384cc1-894f-4bb6-9b1d-567c72004575-metrics-certs podName:82384cc1-894f-4bb6-9b1d-567c72004575 nodeName:}" failed. No retries permitted until 2026-10-30 19:31:10.972118536 +0000 UTC m=+18.028901989 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/82384cc1-894f-4bb6-9b1d-567c72004575-metrics-certs") pod "network-metrics-daemon-vlqp7" (UID: "82384cc1-894f-4bb6-9b1d-567c72004575") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 19:31:03.072695 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:03.072665    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/141cf3eb-d99b-4876-b65a-a2847584ff5d-original-pull-secret\") pod \"global-pull-secret-syncer-526rb\" (UID: \"141cf3eb-d99b-4876-b65a-a2847584ff5d\") " pod="kube-system/global-pull-secret-syncer-526rb"
Oct 30 19:31:03.072845 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:03.072806    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 19:31:03.072910 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:03.072862    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/141cf3eb-d99b-4876-b65a-a2847584ff5d-original-pull-secret podName:141cf3eb-d99b-4876-b65a-a2847584ff5d nodeName:}" failed. No retries permitted until 2026-10-30 19:31:11.072845316 +0000 UTC m=+18.129628755 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/141cf3eb-d99b-4876-b65a-a2847584ff5d-original-pull-secret") pod "global-pull-secret-syncer-526rb" (UID: "141cf3eb-d99b-4876-b65a-a2847584ff5d") : object "kube-system"/"original-pull-secret" not registered
Oct 30 19:31:03.154404 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:03.154368    2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-bx5vk"
Oct 30 19:31:03.173159 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:03.173123    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nn6bt\" (UniqueName: \"kubernetes.io/projected/7ff4ab79-1d7c-49ac-9d63-31772ad68b42-kube-api-access-nn6bt\") pod \"network-check-target-4f5sl\" (UID: \"7ff4ab79-1d7c-49ac-9d63-31772ad68b42\") " pod="openshift-network-diagnostics/network-check-target-4f5sl"
Oct 30 19:31:03.173814 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:03.173357    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 19:31:03.173814 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:03.173384    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 19:31:03.173814 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:03.173397    2578 projected.go:196] Error preparing data for projected volume kube-api-access-nn6bt for pod openshift-network-diagnostics/network-check-target-4f5sl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 19:31:03.173814 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:03.173459    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7ff4ab79-1d7c-49ac-9d63-31772ad68b42-kube-api-access-nn6bt podName:7ff4ab79-1d7c-49ac-9d63-31772ad68b42 nodeName:}" failed. No retries permitted until 2026-10-30 19:31:11.173440741 +0000 UTC m=+18.230224194 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-nn6bt" (UniqueName: "kubernetes.io/projected/7ff4ab79-1d7c-49ac-9d63-31772ad68b42-kube-api-access-nn6bt") pod "network-check-target-4f5sl" (UID: "7ff4ab79-1d7c-49ac-9d63-31772ad68b42") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 19:31:03.227213 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:03.227042    2578 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-bx5vk"
Oct 30 19:31:03.227968 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:03.227946    2578 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-bx5vk"
Oct 30 19:31:03.396469 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:03.396402    2578 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 19:31:03.419252 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:03.418737    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vlqp7"
Oct 30 19:31:03.419252 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:03.418885    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vlqp7" podUID="82384cc1-894f-4bb6-9b1d-567c72004575"
Oct 30 19:31:03.419252 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:03.419157    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4f5sl"
Oct 30 19:31:03.419252 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:03.419216    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4f5sl" podUID="7ff4ab79-1d7c-49ac-9d63-31772ad68b42"
Oct 30 19:31:03.419631 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:03.419514    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-526rb"
Oct 30 19:31:03.419631 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:03.419584    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-526rb" podUID="141cf3eb-d99b-4876-b65a-a2847584ff5d"
Oct 30 19:31:03.605818 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:03.605781    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-97df4" event={"ID":"63678441-271f-46d7-ac1d-57a60bb78b12","Type":"ContainerStarted","Data":"5392fbf7eb8cd1fc841bf808875cd0dddd0389a78839f6d7f538deadf62a0c08"}
Oct 30 19:31:03.608294 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:03.608266    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hts7f" event={"ID":"d0036363-bf0e-44d7-b285-44546ac729e6","Type":"ContainerStarted","Data":"b0e194faa8d0d4a5eda4f30bcea6d144011326d58f9ad12735ff36c699a1385d"}
Oct 30 19:31:03.611252 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:03.611225    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-859jf" event={"ID":"1977f3d7-c7f6-44cc-8a59-9d4f35adc0f7","Type":"ContainerStarted","Data":"72d8a0c738c274653810c0823d9cb5fe5e5ddadf935b04e58a522d4732acc42e"}
Oct 30 19:31:03.611736 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:03.611715    2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-bx5vk"
Oct 30 19:31:03.633510 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:03.633449    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-97df4" podStartSLOduration=3.324862664 podStartE2EDuration="10.633435963s" totalImagesPullingTime="7.308573299s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:30:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:31:03.633238268 +0000 UTC m=+10.690021728" watchObservedRunningTime="2026-10-30 19:31:03.633435963 +0000 UTC m=+10.690219421"
Oct 30 19:31:03.653104 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:03.653049    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-859jf" podStartSLOduration=3.038671475 podStartE2EDuration="10.653029354s" totalImagesPullingTime="7.614357879s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:30:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:31:03.652335506 +0000 UTC m=+10.709118981" watchObservedRunningTime="2026-10-30 19:31:03.653029354 +0000 UTC m=+10.709812814"
Oct 30 19:31:04.355094 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:04.354418    2578 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T19:31:03.396418192Z","UUID":"77669f75-5a8a-4e25-b133-7f124aec082b","Handler":null,"Name":"","Endpoint":""}
Oct 30 19:31:04.356069 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:04.356046    2578 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 19:31:04.356165 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:04.356080    2578 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 19:31:04.615714 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:04.615625    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hts7f" event={"ID":"d0036363-bf0e-44d7-b285-44546ac729e6","Type":"ContainerStarted","Data":"b95b672d7a9a268dce557446831efc01c97fcb5e10c25fef072868bbc7324d13"}
Oct 30 19:31:04.619551 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:04.619520    2578 generic.go:356] "Generic (PLEG): container finished" podID="79ea5164-32e2-4505-b92a-1e50495a6492" containerID="1e81737bca963e643ff03b6a198cde56fcd2c6976197077e33e4b6d2266b1b83" exitCode=0
Oct 30 19:31:04.619683 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:04.619605    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9n92x" event={"ID":"79ea5164-32e2-4505-b92a-1e50495a6492","Type":"ContainerDied","Data":"1e81737bca963e643ff03b6a198cde56fcd2c6976197077e33e4b6d2266b1b83"}
Oct 30 19:31:04.639578 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:04.639529    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hts7f" podStartSLOduration=3.259387778 podStartE2EDuration="11.639513909s" totalImagesPullingTime="8.380126131s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:30:53 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:31:04.63946311 +0000 UTC m=+11.696246570" watchObservedRunningTime="2026-10-30 19:31:04.639513909 +0000 UTC m=+11.696297370"
Oct 30 19:31:05.418647 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:05.418612    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-526rb"
Oct 30 19:31:05.418824 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:05.418754    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-526rb" podUID="141cf3eb-d99b-4876-b65a-a2847584ff5d"
Oct 30 19:31:05.419178 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:05.419158    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4f5sl"
Oct 30 19:31:05.419285 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:05.419264    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4f5sl" podUID="7ff4ab79-1d7c-49ac-9d63-31772ad68b42"
Oct 30 19:31:05.419421 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:05.419407    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vlqp7"
Oct 30 19:31:05.419547 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:05.419530    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vlqp7" podUID="82384cc1-894f-4bb6-9b1d-567c72004575"
Oct 30 19:31:05.623300 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:05.623204    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-5jq48" event={"ID":"0100ec89-6366-4946-99ce-4a254850c8fd","Type":"ContainerStarted","Data":"7a72edeb111111256c443e22c7f602ac4317503d635fdd7749182c0b8fb8f52d"}
Oct 30 19:31:05.643575 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:05.643526    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-5jq48" podStartSLOduration=4.591838676 podStartE2EDuration="12.643512008s" totalImagesPullingTime="8.051673332s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:30:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:31:05.6434069 +0000 UTC m=+12.700190359" watchObservedRunningTime="2026-10-30 19:31:05.643512008 +0000 UTC m=+12.700295466"
Oct 30 19:31:06.627620 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:06.627580    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-g4c6r" event={"ID":"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e","Type":"ContainerStarted","Data":"d91e5275553886eac997f61e851c3e585a389850a490955a4f1906826edc31aa"}
Oct 30 19:31:06.629938 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:06.629903    2578 generic.go:356] "Generic (PLEG): container finished" podID="79ea5164-32e2-4505-b92a-1e50495a6492" containerID="c88f94df906706a3e9dd404f1a8c10215763b241ced15c9a44778d060e0b3067" exitCode=0
Oct 30 19:31:06.630076 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:06.629984    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9n92x" event={"ID":"79ea5164-32e2-4505-b92a-1e50495a6492","Type":"ContainerDied","Data":"c88f94df906706a3e9dd404f1a8c10215763b241ced15c9a44778d060e0b3067"}
Oct 30 19:31:06.652923 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:06.652882    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-g4c6r" podStartSLOduration=3.419072206 podStartE2EDuration="13.652870174s" totalImagesPullingTime="10.233797968s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:30:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:31:06.65251626 +0000 UTC m=+13.709299719" watchObservedRunningTime="2026-10-30 19:31:06.652870174 +0000 UTC m=+13.709653632"
Oct 30 19:31:07.419962 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:07.419939    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-526rb"
Oct 30 19:31:07.420052 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:07.420035    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-526rb" podUID="141cf3eb-d99b-4876-b65a-a2847584ff5d"
Oct 30 19:31:07.420312 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:07.420298    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vlqp7"
Oct 30 19:31:07.420381 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:07.420367    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vlqp7" podUID="82384cc1-894f-4bb6-9b1d-567c72004575"
Oct 30 19:31:07.420538 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:07.420517    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4f5sl"
Oct 30 19:31:07.420637 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:07.420618    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4f5sl" podUID="7ff4ab79-1d7c-49ac-9d63-31772ad68b42"
Oct 30 19:31:07.635338 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:07.635299    2578 generic.go:356] "Generic (PLEG): container finished" podID="79ea5164-32e2-4505-b92a-1e50495a6492" containerID="92ab9b79e4d2dca1f83a300cf372440b5f0ba7742b259371d4ad309627f9738b" exitCode=0
Oct 30 19:31:07.635996 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:07.635535    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9n92x" event={"ID":"79ea5164-32e2-4505-b92a-1e50495a6492","Type":"ContainerDied","Data":"92ab9b79e4d2dca1f83a300cf372440b5f0ba7742b259371d4ad309627f9738b"}
Oct 30 19:31:09.418426 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:09.418235    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4f5sl"
Oct 30 19:31:09.418778 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:09.418235    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-526rb"
Oct 30 19:31:09.418778 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:09.418550    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4f5sl" podUID="7ff4ab79-1d7c-49ac-9d63-31772ad68b42"
Oct 30 19:31:09.418778 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:09.418236    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vlqp7"
Oct 30 19:31:09.418778 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:09.418628    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-526rb" podUID="141cf3eb-d99b-4876-b65a-a2847584ff5d"
Oct 30 19:31:09.418778 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:09.418677    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vlqp7" podUID="82384cc1-894f-4bb6-9b1d-567c72004575"
Oct 30 19:31:09.642497 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:09.642453    2578 generic.go:356] "Generic (PLEG): container finished" podID="79ea5164-32e2-4505-b92a-1e50495a6492" containerID="59e177e0ff64c1a680dc22a75215aeb1640a97b49d204789f9ec8276c744ce07" exitCode=0
Oct 30 19:31:09.642614 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:09.642521    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9n92x" event={"ID":"79ea5164-32e2-4505-b92a-1e50495a6492","Type":"ContainerDied","Data":"59e177e0ff64c1a680dc22a75215aeb1640a97b49d204789f9ec8276c744ce07"}
Oct 30 19:31:10.647844 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:10.647554    2578 generic.go:356] "Generic (PLEG): container finished" podID="79ea5164-32e2-4505-b92a-1e50495a6492" containerID="17c529387bfe794eedc3eb734d277de132f57cc48c4ecc0cffbf3dc3c1f6c057" exitCode=0
Oct 30 19:31:10.647844 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:10.647615    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9n92x" event={"ID":"79ea5164-32e2-4505-b92a-1e50495a6492","Type":"ContainerDied","Data":"17c529387bfe794eedc3eb734d277de132f57cc48c4ecc0cffbf3dc3c1f6c057"}
Oct 30 19:31:10.651070 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:10.651049    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-74lps" event={"ID":"d4d142b9-22f1-427f-a374-ba4918487946","Type":"ContainerStarted","Data":"471082ab7443ae1374f5747e633036824fe3fbd8b38419b906573452ad4117d0"}
Oct 30 19:31:10.651148 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:10.651121    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-74lps" event={"ID":"d4d142b9-22f1-427f-a374-ba4918487946","Type":"ContainerStarted","Data":"08c24435452e92185c1652fb300defdddd519cdf1f99e719ffc2d6b75b748d38"}
Oct 30 19:31:10.651182 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:10.651162    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-74lps" event={"ID":"d4d142b9-22f1-427f-a374-ba4918487946","Type":"ContainerStarted","Data":"c18e9f033e3967f8eed852cc14396418f47bd8583255db779b532b0d1a0b5af2"}
Oct 30 19:31:10.651221 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:10.651191    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-74lps" event={"ID":"d4d142b9-22f1-427f-a374-ba4918487946","Type":"ContainerStarted","Data":"94653e450cc8a60ec377a3635429ad8098035633a42a8d616b4f3560076daebf"}
Oct 30 19:31:10.651221 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:10.651219    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-74lps" event={"ID":"d4d142b9-22f1-427f-a374-ba4918487946","Type":"ContainerStarted","Data":"104f96dfacc468878d8328896b4e89294a4961ee76eac6d117d58a080c0e8c41"}
Oct 30 19:31:10.651283 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:10.651245    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-74lps" event={"ID":"d4d142b9-22f1-427f-a374-ba4918487946","Type":"ContainerStarted","Data":"29f6650eae6461e4d3e46e06cdaa9978e9d1735f90086e8920aa818797c93ea7"}
Oct 30 19:31:11.035796 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:11.035717    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/82384cc1-894f-4bb6-9b1d-567c72004575-metrics-certs\") pod \"network-metrics-daemon-vlqp7\" (UID: \"82384cc1-894f-4bb6-9b1d-567c72004575\") " pod="openshift-multus/network-metrics-daemon-vlqp7"
Oct 30 19:31:11.035945 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:11.035817    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 19:31:11.035945 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:11.035867    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82384cc1-894f-4bb6-9b1d-567c72004575-metrics-certs podName:82384cc1-894f-4bb6-9b1d-567c72004575 nodeName:}" failed. No retries permitted until 2026-10-30 19:31:27.035852694 +0000 UTC m=+34.092636130 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/82384cc1-894f-4bb6-9b1d-567c72004575-metrics-certs") pod "network-metrics-daemon-vlqp7" (UID: "82384cc1-894f-4bb6-9b1d-567c72004575") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 19:31:11.136503 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:11.136468    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/141cf3eb-d99b-4876-b65a-a2847584ff5d-original-pull-secret\") pod \"global-pull-secret-syncer-526rb\" (UID: \"141cf3eb-d99b-4876-b65a-a2847584ff5d\") " pod="kube-system/global-pull-secret-syncer-526rb"
Oct 30 19:31:11.136612 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:11.136599    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 19:31:11.136659 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:11.136650    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/141cf3eb-d99b-4876-b65a-a2847584ff5d-original-pull-secret podName:141cf3eb-d99b-4876-b65a-a2847584ff5d nodeName:}" failed. No retries permitted until 2026-10-30 19:31:27.136636212 +0000 UTC m=+34.193419657 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/141cf3eb-d99b-4876-b65a-a2847584ff5d-original-pull-secret") pod "global-pull-secret-syncer-526rb" (UID: "141cf3eb-d99b-4876-b65a-a2847584ff5d") : object "kube-system"/"original-pull-secret" not registered
Oct 30 19:31:11.237206 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:11.237175    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nn6bt\" (UniqueName: \"kubernetes.io/projected/7ff4ab79-1d7c-49ac-9d63-31772ad68b42-kube-api-access-nn6bt\") pod \"network-check-target-4f5sl\" (UID: \"7ff4ab79-1d7c-49ac-9d63-31772ad68b42\") " pod="openshift-network-diagnostics/network-check-target-4f5sl"
Oct 30 19:31:11.237345 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:11.237327    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 19:31:11.237401 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:11.237351    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 19:31:11.237401 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:11.237361    2578 projected.go:196] Error preparing data for projected volume kube-api-access-nn6bt for pod openshift-network-diagnostics/network-check-target-4f5sl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 19:31:11.237401 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:11.237400    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7ff4ab79-1d7c-49ac-9d63-31772ad68b42-kube-api-access-nn6bt podName:7ff4ab79-1d7c-49ac-9d63-31772ad68b42 nodeName:}" failed. No retries permitted until 2026-10-30 19:31:27.237387753 +0000 UTC m=+34.294171188 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-nn6bt" (UniqueName: "kubernetes.io/projected/7ff4ab79-1d7c-49ac-9d63-31772ad68b42-kube-api-access-nn6bt") pod "network-check-target-4f5sl" (UID: "7ff4ab79-1d7c-49ac-9d63-31772ad68b42") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 19:31:11.421962 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:11.421893    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4f5sl"
Oct 30 19:31:11.422082 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:11.421993    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vlqp7"
Oct 30 19:31:11.422082 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:11.422074    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-526rb"
Oct 30 19:31:11.422165 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:11.422078    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4f5sl" podUID="7ff4ab79-1d7c-49ac-9d63-31772ad68b42"
Oct 30 19:31:11.422165 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:11.422143    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vlqp7" podUID="82384cc1-894f-4bb6-9b1d-567c72004575"
Oct 30 19:31:11.422242 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:11.422220    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-526rb" podUID="141cf3eb-d99b-4876-b65a-a2847584ff5d"
Oct 30 19:31:11.656493 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:11.656445    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9n92x" event={"ID":"79ea5164-32e2-4505-b92a-1e50495a6492","Type":"ContainerStarted","Data":"92953f44370ba99cb33a7cab72cd9ef1cdbb7999be4c50235e57b026456f2a88"}
Oct 30 19:31:11.691928 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:11.691847    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-9n92x" podStartSLOduration=7.490654791 podStartE2EDuration="18.691834778s" totalImagesPullingTime="11.201179987s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:30:53 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:31:11.691834907 +0000 UTC m=+18.748618365" watchObservedRunningTime="2026-10-30 19:31:11.691834778 +0000 UTC m=+18.748618233"
Oct 30 19:31:12.661654 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:12.661621    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-74lps" event={"ID":"d4d142b9-22f1-427f-a374-ba4918487946","Type":"ContainerStarted","Data":"e1145072a12196da4280b2849606e4b628f336ee13ca44f3725936bdb9d901dc"}
Oct 30 19:31:13.421026 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:13.420997    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4f5sl"
Oct 30 19:31:13.421159 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:13.421085    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vlqp7"
Oct 30 19:31:13.421159 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:13.421104    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-526rb"
Oct 30 19:31:13.421237 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:13.421184    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4f5sl" podUID="7ff4ab79-1d7c-49ac-9d63-31772ad68b42"
Oct 30 19:31:13.421331 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:13.421313    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vlqp7" podUID="82384cc1-894f-4bb6-9b1d-567c72004575"
Oct 30 19:31:13.421386 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:13.421372    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-526rb" podUID="141cf3eb-d99b-4876-b65a-a2847584ff5d"
Oct 30 19:31:14.672824 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:14.672641    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-74lps" event={"ID":"d4d142b9-22f1-427f-a374-ba4918487946","Type":"ContainerStarted","Data":"0272e11cc07675e7037d78efd8f2d7ce721abe6432ed5eca32faf126acffff43"}
Oct 30 19:31:14.673516 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:14.672914    2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:31:14.673516 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:14.672962    2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:31:14.691044 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:14.691008    2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:31:14.691175 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:14.691133    2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:31:14.713399 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:14.713349    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-74lps" podStartSLOduration=7.277517795 podStartE2EDuration="21.713337078s" totalImagesPullingTime="14.435819283s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:30:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:31:14.712574949 +0000 UTC m=+21.769358398" watchObservedRunningTime="2026-10-30 19:31:14.713337078 +0000 UTC m=+21.770120560"
Oct 30 19:31:15.419123 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:15.419095    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vlqp7"
Oct 30 19:31:15.419277 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:15.419156    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4f5sl"
Oct 30 19:31:15.419277 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:15.419159    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-526rb"
Oct 30 19:31:15.419277 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:15.419249    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4f5sl" podUID="7ff4ab79-1d7c-49ac-9d63-31772ad68b42"
Oct 30 19:31:15.419406 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:15.419309    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-526rb" podUID="141cf3eb-d99b-4876-b65a-a2847584ff5d"
Oct 30 19:31:15.419406 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:15.419388    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vlqp7" podUID="82384cc1-894f-4bb6-9b1d-567c72004575"
Oct 30 19:31:15.674663 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:15.674590    2578 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 19:31:16.490520 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:16.490443    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4f5sl"]
Oct 30 19:31:16.490640 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:16.490565    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4f5sl"
Oct 30 19:31:16.490674 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:16.490650    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4f5sl" podUID="7ff4ab79-1d7c-49ac-9d63-31772ad68b42"
Oct 30 19:31:16.493929 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:16.493904    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-526rb"]
Oct 30 19:31:16.494435 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:16.494416    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-526rb"
Oct 30 19:31:16.494561 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:16.494538    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-526rb" podUID="141cf3eb-d99b-4876-b65a-a2847584ff5d"
Oct 30 19:31:16.494684 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:16.494668    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vlqp7"]
Oct 30 19:31:16.494755 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:16.494743    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vlqp7"
Oct 30 19:31:16.494841 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:16.494824    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vlqp7" podUID="82384cc1-894f-4bb6-9b1d-567c72004575"
Oct 30 19:31:16.676966 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:16.676942    2578 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 19:31:18.418496 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:18.418461    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-526rb"
Oct 30 19:31:18.418965 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:18.418500    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vlqp7"
Oct 30 19:31:18.418965 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:18.418566    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4f5sl"
Oct 30 19:31:18.418965 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:18.418577    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-526rb" podUID="141cf3eb-d99b-4876-b65a-a2847584ff5d"
Oct 30 19:31:18.418965 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:18.418642    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vlqp7" podUID="82384cc1-894f-4bb6-9b1d-567c72004575"
Oct 30 19:31:18.418965 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:18.418749    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4f5sl" podUID="7ff4ab79-1d7c-49ac-9d63-31772ad68b42"
Oct 30 19:31:18.765062 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:18.765004    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-859jf_1977f3d7-c7f6-44cc-8a59-9d4f35adc0f7/dns-node-resolver/0.log"
Oct 30 19:31:19.924970 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:19.924948    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-j4cdl_6eca1a1e-101a-43bb-9fde-505c3182c10a/node-ca/0.log"
Oct 30 19:31:20.304899 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:20.304831    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bx5vk_17f17805-edc3-4dc5-8e75-819350666927/konnectivity-agent/0.log" limit=1024
Oct 30 19:31:20.418799 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:20.418768    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vlqp7"
Oct 30 19:31:20.418959 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:20.418803    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4f5sl"
Oct 30 19:31:20.418959 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:20.418782    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-526rb"
Oct 30 19:31:20.418959 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:20.418894    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vlqp7" podUID="82384cc1-894f-4bb6-9b1d-567c72004575"
Oct 30 19:31:20.418959 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:20.418948    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-526rb" podUID="141cf3eb-d99b-4876-b65a-a2847584ff5d"
Oct 30 19:31:20.419099 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:20.419034    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4f5sl" podUID="7ff4ab79-1d7c-49ac-9d63-31772ad68b42"
Oct 30 19:31:20.509320 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:20.509294    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bx5vk_17f17805-edc3-4dc5-8e75-819350666927/konnectivity-agent/0.log" limit=1024
Oct 30 19:31:21.088689 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:21.088661    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bx5vk_17f17805-edc3-4dc5-8e75-819350666927/konnectivity-agent/0.log" limit=1024
Oct 30 19:31:21.252117 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:21.252072    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bx5vk_17f17805-edc3-4dc5-8e75-819350666927/konnectivity-agent/0.log" limit=1024
Oct 30 19:31:22.048681 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:22.048653    2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:31:22.048914 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:22.048899    2578 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 19:31:22.059750 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:22.059556    2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:31:22.418873 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:22.418784    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4f5sl"
Oct 30 19:31:22.419223 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:22.418890    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vlqp7"
Oct 30 19:31:22.419223 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:22.418895    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4f5sl" podUID="7ff4ab79-1d7c-49ac-9d63-31772ad68b42"
Oct 30 19:31:22.419223 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:22.418921    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-526rb"
Oct 30 19:31:22.419223 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:22.419003    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vlqp7" podUID="82384cc1-894f-4bb6-9b1d-567c72004575"
Oct 30 19:31:22.419223 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:22.419056    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-526rb" podUID="141cf3eb-d99b-4876-b65a-a2847584ff5d"
Oct 30 19:31:24.418637 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:24.418609    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4f5sl"
Oct 30 19:31:24.419094 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:24.418700    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-526rb"
Oct 30 19:31:24.419094 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:24.418727    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vlqp7"
Oct 30 19:31:24.419094 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:24.418701    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4f5sl" podUID="7ff4ab79-1d7c-49ac-9d63-31772ad68b42"
Oct 30 19:31:24.419094 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:24.418818    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-526rb" podUID="141cf3eb-d99b-4876-b65a-a2847584ff5d"
Oct 30 19:31:24.419094 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:24.418895    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vlqp7" podUID="82384cc1-894f-4bb6-9b1d-567c72004575"
Oct 30 19:31:26.418615 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:26.418580    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4f5sl"
Oct 30 19:31:26.418615 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:26.418608    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-526rb"
Oct 30 19:31:26.418615 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:26.418619    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vlqp7"
Oct 30 19:31:26.419155 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:26.418699    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4f5sl" podUID="7ff4ab79-1d7c-49ac-9d63-31772ad68b42"
Oct 30 19:31:26.419155 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:26.418808    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-526rb" podUID="141cf3eb-d99b-4876-b65a-a2847584ff5d"
Oct 30 19:31:26.419155 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:26.418878    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vlqp7" podUID="82384cc1-894f-4bb6-9b1d-567c72004575"
Oct 30 19:31:27.052510 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:27.052415    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/82384cc1-894f-4bb6-9b1d-567c72004575-metrics-certs\") pod \"network-metrics-daemon-vlqp7\" (UID: \"82384cc1-894f-4bb6-9b1d-567c72004575\") " pod="openshift-multus/network-metrics-daemon-vlqp7"
Oct 30 19:31:27.052657 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:27.052554    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 19:31:27.052657 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:27.052607    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82384cc1-894f-4bb6-9b1d-567c72004575-metrics-certs podName:82384cc1-894f-4bb6-9b1d-567c72004575 nodeName:}" failed. No retries permitted until 2026-10-30 19:31:59.052589512 +0000 UTC m=+66.109372956 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/82384cc1-894f-4bb6-9b1d-567c72004575-metrics-certs") pod "network-metrics-daemon-vlqp7" (UID: "82384cc1-894f-4bb6-9b1d-567c72004575") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 19:31:27.152681 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:27.152657    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/141cf3eb-d99b-4876-b65a-a2847584ff5d-original-pull-secret\") pod \"global-pull-secret-syncer-526rb\" (UID: \"141cf3eb-d99b-4876-b65a-a2847584ff5d\") " pod="kube-system/global-pull-secret-syncer-526rb"
Oct 30 19:31:27.152835 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:27.152748    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 19:31:27.152835 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:27.152793    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/141cf3eb-d99b-4876-b65a-a2847584ff5d-original-pull-secret podName:141cf3eb-d99b-4876-b65a-a2847584ff5d nodeName:}" failed. No retries permitted until 2026-10-30 19:31:59.152779785 +0000 UTC m=+66.209563221 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/141cf3eb-d99b-4876-b65a-a2847584ff5d-original-pull-secret") pod "global-pull-secret-syncer-526rb" (UID: "141cf3eb-d99b-4876-b65a-a2847584ff5d") : object "kube-system"/"original-pull-secret" not registered
Oct 30 19:31:27.252906 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:27.252878    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nn6bt\" (UniqueName: \"kubernetes.io/projected/7ff4ab79-1d7c-49ac-9d63-31772ad68b42-kube-api-access-nn6bt\") pod \"network-check-target-4f5sl\" (UID: \"7ff4ab79-1d7c-49ac-9d63-31772ad68b42\") " pod="openshift-network-diagnostics/network-check-target-4f5sl"
Oct 30 19:31:27.253032 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:27.252991    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 19:31:27.253032 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:27.253006    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 19:31:27.253032 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:27.253014    2578 projected.go:196] Error preparing data for projected volume kube-api-access-nn6bt for pod openshift-network-diagnostics/network-check-target-4f5sl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 19:31:27.253126 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:27.253054    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7ff4ab79-1d7c-49ac-9d63-31772ad68b42-kube-api-access-nn6bt podName:7ff4ab79-1d7c-49ac-9d63-31772ad68b42 nodeName:}" failed. No retries permitted until 2026-10-30 19:31:59.253042433 +0000 UTC m=+66.309825882 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-nn6bt" (UniqueName: "kubernetes.io/projected/7ff4ab79-1d7c-49ac-9d63-31772ad68b42-kube-api-access-nn6bt") pod "network-check-target-4f5sl" (UID: "7ff4ab79-1d7c-49ac-9d63-31772ad68b42") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 19:31:28.418520 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:28.418472    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4f5sl"
Oct 30 19:31:28.418520 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:28.418515    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-526rb"
Oct 30 19:31:28.418973 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:28.418603    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4f5sl" podUID="7ff4ab79-1d7c-49ac-9d63-31772ad68b42"
Oct 30 19:31:28.418973 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:28.418660    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vlqp7"
Oct 30 19:31:28.418973 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:28.418741    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-526rb" podUID="141cf3eb-d99b-4876-b65a-a2847584ff5d"
Oct 30 19:31:28.418973 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:28.418830    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vlqp7" podUID="82384cc1-894f-4bb6-9b1d-567c72004575"
Oct 30 19:31:30.419246 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:30.419212    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4f5sl"
Oct 30 19:31:30.419644 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:30.419331    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-526rb"
Oct 30 19:31:30.419644 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:30.419357    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vlqp7"
Oct 30 19:31:30.419644 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:30.419350    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4f5sl" podUID="7ff4ab79-1d7c-49ac-9d63-31772ad68b42"
Oct 30 19:31:30.419644 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:30.419424    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-526rb" podUID="141cf3eb-d99b-4876-b65a-a2847584ff5d"
Oct 30 19:31:30.419644 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:30.419503    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vlqp7" podUID="82384cc1-894f-4bb6-9b1d-567c72004575"
Oct 30 19:31:32.418169 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:32.418135    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vlqp7"
Oct 30 19:31:32.418169 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:32.418171    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4f5sl"
Oct 30 19:31:32.418706 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:32.418266    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vlqp7" podUID="82384cc1-894f-4bb6-9b1d-567c72004575"
Oct 30 19:31:32.418706 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:32.418310    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-526rb"
Oct 30 19:31:32.418706 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:32.418391    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4f5sl" podUID="7ff4ab79-1d7c-49ac-9d63-31772ad68b42"
Oct 30 19:31:32.418706 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:32.418453    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-526rb" podUID="141cf3eb-d99b-4876-b65a-a2847584ff5d"
Oct 30 19:31:34.418475 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:34.418447    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4f5sl"
Oct 30 19:31:34.418869 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:34.418543    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vlqp7"
Oct 30 19:31:34.418869 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:34.418569    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-526rb"
Oct 30 19:31:34.418869 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:34.418680    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vlqp7" podUID="82384cc1-894f-4bb6-9b1d-567c72004575"
Oct 30 19:31:34.418869 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:34.418694    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-526rb" podUID="141cf3eb-d99b-4876-b65a-a2847584ff5d"
Oct 30 19:31:34.418869 ip-10-0-141-42 kubenswrapper[2578]: E1030 19:31:34.418766    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4f5sl" podUID="7ff4ab79-1d7c-49ac-9d63-31772ad68b42"
Oct 30 19:31:35.763282 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:35.763208    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-42.us-east-2.compute.internal" event="NodeReady"
Oct 30 19:31:35.763665 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:35.763317    2578 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Oct 30 19:31:35.830554 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:35.830525    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-8dn55"]
Oct 30 19:31:35.855130 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:35.855097    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-qdkpz"]
Oct 30 19:31:35.855266 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:35.855218    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8dn55"
Oct 30 19:31:35.871240 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:35.871218    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8dn55"]
Oct 30 19:31:35.871370 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:35.871265    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qdkpz"]
Oct 30 19:31:35.871370 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:35.871288    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-8hjtk"]
Oct 30 19:31:35.871370 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:35.871345    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qdkpz"
Oct 30 19:31:35.888280 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:35.885455    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-8hjtk"]
Oct 30 19:31:35.888280 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:35.885605    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8hjtk"
Oct 30 19:31:35.893617 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:35.893556    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 19:31:35.893617 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:35.893574    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 19:31:35.893723 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:35.893624    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 19:31:35.893723 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:35.893646    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 19:31:35.893723 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:35.893695    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 19:31:35.893723 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:35.893716    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 19:31:35.895472 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:35.895453    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-h46gl\""
Oct 30 19:31:35.895472 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:35.895456    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 19:31:35.895592 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:35.895460    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-f2mgm\""
Oct 30 19:31:35.895703 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:35.895690    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 19:31:35.895836 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:35.895824    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 19:31:35.895869 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:35.895845    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-scxl6\""
Oct 30 19:31:36.018281 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:36.018208    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jx8qx\" (UniqueName: \"kubernetes.io/projected/7bb33e32-f8ac-44a2-95a4-cb949b08710f-kube-api-access-jx8qx\") pod \"dns-default-8dn55\" (UID: \"7bb33e32-f8ac-44a2-95a4-cb949b08710f\") " pod="openshift-dns/dns-default-8dn55"
Oct 30 19:31:36.018281 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:36.018256    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/559fa506-ed9a-4c61-9316-edc85d6c5399-crio-socket\") pod \"insights-runtime-extractor-qdkpz\" (UID: \"559fa506-ed9a-4c61-9316-edc85d6c5399\") " pod="openshift-insights/insights-runtime-extractor-qdkpz"
Oct 30 19:31:36.018281 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:36.018279    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7bb33e32-f8ac-44a2-95a4-cb949b08710f-config-volume\") pod \"dns-default-8dn55\" (UID: \"7bb33e32-f8ac-44a2-95a4-cb949b08710f\") " pod="openshift-dns/dns-default-8dn55"
Oct 30 19:31:36.018610 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:36.018362    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7bb33e32-f8ac-44a2-95a4-cb949b08710f-metrics-tls\") pod \"dns-default-8dn55\" (UID: \"7bb33e32-f8ac-44a2-95a4-cb949b08710f\") " pod="openshift-dns/dns-default-8dn55"
Oct 30 19:31:36.018610 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:36.018404    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/559fa506-ed9a-4c61-9316-edc85d6c5399-data-volume\") pod \"insights-runtime-extractor-qdkpz\" (UID: \"559fa506-ed9a-4c61-9316-edc85d6c5399\") " pod="openshift-insights/insights-runtime-extractor-qdkpz"
Oct 30 19:31:36.018610 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:36.018421    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/517d6b00-0305-40cc-95c3-bba4adb2f849-cert\") pod \"ingress-canary-8hjtk\" (UID: \"517d6b00-0305-40cc-95c3-bba4adb2f849\") " pod="openshift-ingress-canary/ingress-canary-8hjtk"
Oct 30 19:31:36.018610 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:36.018439    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tv2lm\" (UniqueName: \"kubernetes.io/projected/517d6b00-0305-40cc-95c3-bba4adb2f849-kube-api-access-tv2lm\") pod \"ingress-canary-8hjtk\" (UID: \"517d6b00-0305-40cc-95c3-bba4adb2f849\") " pod="openshift-ingress-canary/ingress-canary-8hjtk"
Oct 30 19:31:36.018610 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:36.018543    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7bb33e32-f8ac-44a2-95a4-cb949b08710f-tmp-dir\") pod \"dns-default-8dn55\" (UID: \"7bb33e32-f8ac-44a2-95a4-cb949b08710f\") " pod="openshift-dns/dns-default-8dn55"
Oct 30 19:31:36.018610 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:36.018578    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/559fa506-ed9a-4c61-9316-edc85d6c5399-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qdkpz\" (UID: \"559fa506-ed9a-4c61-9316-edc85d6c5399\") " pod="openshift-insights/insights-runtime-extractor-qdkpz"
Oct 30 19:31:36.018836 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:36.018615    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/559fa506-ed9a-4c61-9316-edc85d6c5399-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qdkpz\" (UID: \"559fa506-ed9a-4c61-9316-edc85d6c5399\") " pod="openshift-insights/insights-runtime-extractor-qdkpz"
Oct 30 19:31:36.018836 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:36.018640    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bh7cj\" (UniqueName: \"kubernetes.io/projected/559fa506-ed9a-4c61-9316-edc85d6c5399-kube-api-access-bh7cj\") pod \"insights-runtime-extractor-qdkpz\" (UID: \"559fa506-ed9a-4c61-9316-edc85d6c5399\") " pod="openshift-insights/insights-runtime-extractor-qdkpz"
Oct 30 19:31:36.119549 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:36.119522    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tv2lm\" (UniqueName: \"kubernetes.io/projected/517d6b00-0305-40cc-95c3-bba4adb2f849-kube-api-access-tv2lm\") pod \"ingress-canary-8hjtk\" (UID: \"517d6b00-0305-40cc-95c3-bba4adb2f849\") " pod="openshift-ingress-canary/ingress-canary-8hjtk"
Oct 30 19:31:36.119718 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:36.119552    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7bb33e32-f8ac-44a2-95a4-cb949b08710f-tmp-dir\") pod \"dns-default-8dn55\" (UID: \"7bb33e32-f8ac-44a2-95a4-cb949b08710f\") " pod="openshift-dns/dns-default-8dn55"
Oct 30 19:31:36.119718 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:36.119578    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/559fa506-ed9a-4c61-9316-edc85d6c5399-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qdkpz\" (UID: \"559fa506-ed9a-4c61-9316-edc85d6c5399\") " pod="openshift-insights/insights-runtime-extractor-qdkpz"
Oct 30 19:31:36.119718 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:36.119690    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/559fa506-ed9a-4c61-9316-edc85d6c5399-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qdkpz\" (UID: \"559fa506-ed9a-4c61-9316-edc85d6c5399\") " pod="openshift-insights/insights-runtime-extractor-qdkpz"
Oct 30 19:31:36.119866 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:36.119719    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bh7cj\" (UniqueName: \"kubernetes.io/projected/559fa506-ed9a-4c61-9316-edc85d6c5399-kube-api-access-bh7cj\") pod \"insights-runtime-extractor-qdkpz\" (UID: \"559fa506-ed9a-4c61-9316-edc85d6c5399\") " pod="openshift-insights/insights-runtime-extractor-qdkpz"
Oct 30 19:31:36.119866 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:36.119769    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jx8qx\" (UniqueName: \"kubernetes.io/projected/7bb33e32-f8ac-44a2-95a4-cb949b08710f-kube-api-access-jx8qx\") pod \"dns-default-8dn55\" (UID: \"7bb33e32-f8ac-44a2-95a4-cb949b08710f\") " pod="openshift-dns/dns-default-8dn55"
Oct 30 19:31:36.119866 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:36.119821    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/559fa506-ed9a-4c61-9316-edc85d6c5399-crio-socket\") pod \"insights-runtime-extractor-qdkpz\" (UID: \"559fa506-ed9a-4c61-9316-edc85d6c5399\") " pod="openshift-insights/insights-runtime-extractor-qdkpz"
Oct 30 19:31:36.119866 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:36.119836    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7bb33e32-f8ac-44a2-95a4-cb949b08710f-tmp-dir\") pod \"dns-default-8dn55\" (UID: \"7bb33e32-f8ac-44a2-95a4-cb949b08710f\") " pod="openshift-dns/dns-default-8dn55"
Oct 30 19:31:36.119866 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:36.119854    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7bb33e32-f8ac-44a2-95a4-cb949b08710f-config-volume\") pod \"dns-default-8dn55\" (UID: \"7bb33e32-f8ac-44a2-95a4-cb949b08710f\") " pod="openshift-dns/dns-default-8dn55"
Oct 30 19:31:36.120100 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:36.119924    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7bb33e32-f8ac-44a2-95a4-cb949b08710f-metrics-tls\") pod \"dns-default-8dn55\" (UID: \"7bb33e32-f8ac-44a2-95a4-cb949b08710f\") " pod="openshift-dns/dns-default-8dn55"
Oct 30 19:31:36.120100 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:36.119976    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/559fa506-ed9a-4c61-9316-edc85d6c5399-data-volume\") pod \"insights-runtime-extractor-qdkpz\" (UID: \"559fa506-ed9a-4c61-9316-edc85d6c5399\") " pod="openshift-insights/insights-runtime-extractor-qdkpz"
Oct 30 19:31:36.120100 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:36.120010    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/517d6b00-0305-40cc-95c3-bba4adb2f849-cert\") pod \"ingress-canary-8hjtk\" (UID: \"517d6b00-0305-40cc-95c3-bba4adb2f849\") " pod="openshift-ingress-canary/ingress-canary-8hjtk"
Oct 30 19:31:36.120100 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:36.120027    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/559fa506-ed9a-4c61-9316-edc85d6c5399-crio-socket\") pod \"insights-runtime-extractor-qdkpz\" (UID: \"559fa506-ed9a-4c61-9316-edc85d6c5399\") " pod="openshift-insights/insights-runtime-extractor-qdkpz"
Oct 30 19:31:36.120295 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:36.120268    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/559fa506-ed9a-4c61-9316-edc85d6c5399-data-volume\") pod \"insights-runtime-extractor-qdkpz\" (UID: \"559fa506-ed9a-4c61-9316-edc85d6c5399\") " pod="openshift-insights/insights-runtime-extractor-qdkpz"
Oct 30 19:31:36.120376 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:36.120355    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/559fa506-ed9a-4c61-9316-edc85d6c5399-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qdkpz\" (UID: \"559fa506-ed9a-4c61-9316-edc85d6c5399\") " pod="openshift-insights/insights-runtime-extractor-qdkpz"
Oct 30 19:31:36.120498 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:36.120457    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7bb33e32-f8ac-44a2-95a4-cb949b08710f-config-volume\") pod \"dns-default-8dn55\" (UID: \"7bb33e32-f8ac-44a2-95a4-cb949b08710f\") " pod="openshift-dns/dns-default-8dn55"
Oct 30 19:31:36.123573 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:36.123555    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/517d6b00-0305-40cc-95c3-bba4adb2f849-cert\") pod \"ingress-canary-8hjtk\" (UID: \"517d6b00-0305-40cc-95c3-bba4adb2f849\") " pod="openshift-ingress-canary/ingress-canary-8hjtk"
Oct 30 19:31:36.123573 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:36.123564    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7bb33e32-f8ac-44a2-95a4-cb949b08710f-metrics-tls\") pod \"dns-default-8dn55\" (UID: \"7bb33e32-f8ac-44a2-95a4-cb949b08710f\") " pod="openshift-dns/dns-default-8dn55"
Oct 30 19:31:36.123673 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:36.123564    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/559fa506-ed9a-4c61-9316-edc85d6c5399-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qdkpz\" (UID: \"559fa506-ed9a-4c61-9316-edc85d6c5399\") " pod="openshift-insights/insights-runtime-extractor-qdkpz"
Oct 30 19:31:36.133595 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:36.133571    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jx8qx\" (UniqueName: \"kubernetes.io/projected/7bb33e32-f8ac-44a2-95a4-cb949b08710f-kube-api-access-jx8qx\") pod \"dns-default-8dn55\" (UID: \"7bb33e32-f8ac-44a2-95a4-cb949b08710f\") " pod="openshift-dns/dns-default-8dn55"
Oct 30 19:31:36.133765 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:36.133745    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bh7cj\" (UniqueName: \"kubernetes.io/projected/559fa506-ed9a-4c61-9316-edc85d6c5399-kube-api-access-bh7cj\") pod \"insights-runtime-extractor-qdkpz\" (UID: \"559fa506-ed9a-4c61-9316-edc85d6c5399\") " pod="openshift-insights/insights-runtime-extractor-qdkpz"
Oct 30 19:31:36.133863 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:36.133847    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tv2lm\" (UniqueName: \"kubernetes.io/projected/517d6b00-0305-40cc-95c3-bba4adb2f849-kube-api-access-tv2lm\") pod \"ingress-canary-8hjtk\" (UID: \"517d6b00-0305-40cc-95c3-bba4adb2f849\") " pod="openshift-ingress-canary/ingress-canary-8hjtk"
Oct 30 19:31:36.191663 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:36.191638    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8dn55"
Oct 30 19:31:36.228720 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:36.228700    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qdkpz"
Oct 30 19:31:36.239928 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:36.239908    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8hjtk"
Oct 30 19:31:36.418404 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:36.418365    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vlqp7"
Oct 30 19:31:36.418404 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:36.418389    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-526rb"
Oct 30 19:31:36.418596 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:36.418495    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4f5sl"
Oct 30 19:31:36.427776 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:36.427753    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 19:31:36.427776 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:36.427761    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 19:31:36.429980 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:36.429958    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-rpkw7\""
Oct 30 19:31:36.430085 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:36.430002    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 19:31:36.430085 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:36.430008    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-jznq8\""
Oct 30 19:31:36.430085 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:36.430004    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 19:31:36.430085 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:36.430047    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-wr97x\""
Oct 30 19:31:36.553204 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:36.553036    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8dn55"]
Oct 30 19:31:36.556627 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:36.556595    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-8hjtk"]
Oct 30 19:31:36.557452 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:36.557427    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qdkpz"]
Oct 30 19:31:36.560908 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:31:36.560886    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod559fa506_ed9a_4c61_9316_edc85d6c5399.slice/crio-aa8ff2e85b70fb63a568d8b38871125373cc5857c308306bd7c35426b68d32f2 WatchSource:0}: Error finding container aa8ff2e85b70fb63a568d8b38871125373cc5857c308306bd7c35426b68d32f2: Status 404 returned error can't find the container with id aa8ff2e85b70fb63a568d8b38871125373cc5857c308306bd7c35426b68d32f2
Oct 30 19:31:36.561351 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:31:36.561330    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod517d6b00_0305_40cc_95c3_bba4adb2f849.slice/crio-aed45bc206861537fda3348ae1fa90ce9ae2788424d46356d5e92ff68acc77a4 WatchSource:0}: Error finding container aed45bc206861537fda3348ae1fa90ce9ae2788424d46356d5e92ff68acc77a4: Status 404 returned error can't find the container with id aed45bc206861537fda3348ae1fa90ce9ae2788424d46356d5e92ff68acc77a4
Oct 30 19:31:36.716174 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:36.716143    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-8hjtk" event={"ID":"517d6b00-0305-40cc-95c3-bba4adb2f849","Type":"ContainerStarted","Data":"aed45bc206861537fda3348ae1fa90ce9ae2788424d46356d5e92ff68acc77a4"}
Oct 30 19:31:36.717531 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:36.717506    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qdkpz" event={"ID":"559fa506-ed9a-4c61-9316-edc85d6c5399","Type":"ContainerStarted","Data":"9a43b74c0f6abd0345ffcd5d0731477065a0b203a0d4ba83e880ddd5947135ae"}
Oct 30 19:31:36.717638 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:36.717558    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qdkpz" event={"ID":"559fa506-ed9a-4c61-9316-edc85d6c5399","Type":"ContainerStarted","Data":"aa8ff2e85b70fb63a568d8b38871125373cc5857c308306bd7c35426b68d32f2"}
Oct 30 19:31:36.718537 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:36.718517    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8dn55" event={"ID":"7bb33e32-f8ac-44a2-95a4-cb949b08710f","Type":"ContainerStarted","Data":"2313f16d736a5221d08e1f95e5b8ab2af5d3d250f681e6f0d6c58416e0502cc4"}
Oct 30 19:31:37.143074 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:37.143041    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-859jf_1977f3d7-c7f6-44cc-8a59-9d4f35adc0f7/dns-node-resolver/0.log"
Oct 30 19:31:37.723707 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:37.723609    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qdkpz" event={"ID":"559fa506-ed9a-4c61-9316-edc85d6c5399","Type":"ContainerStarted","Data":"dd754c046d68647f3d0434e01b1e0fc0cae18ded7408ede97e0e46b8398eddb9"}
Oct 30 19:31:38.383224 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:38.382252    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-7c4f845644-d8mc5"]
Oct 30 19:31:38.384457 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:38.384430    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c4f845644-d8mc5"
Oct 30 19:31:38.393102 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:38.393082    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Oct 30 19:31:38.397516 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:38.396790    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-f6m98\""
Oct 30 19:31:38.397516 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:38.396988    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Oct 30 19:31:38.398998 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:38.397775    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Oct 30 19:31:38.398998 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:38.397889    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Oct 30 19:31:38.398998 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:38.398131    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Oct 30 19:31:38.398998 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:38.398186    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Oct 30 19:31:38.406939 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:38.406896    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Oct 30 19:31:38.407270 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:38.407249    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c4f845644-d8mc5"]
Oct 30 19:31:38.407428 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:38.407325    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Oct 30 19:31:38.439715 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:38.439686    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-console-oauth-config\") pod \"console-7c4f845644-d8mc5\" (UID: \"c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3\") " pod="openshift-console/console-7c4f845644-d8mc5"
Oct 30 19:31:38.439715 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:38.439719    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-console-serving-cert\") pod \"console-7c4f845644-d8mc5\" (UID: \"c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3\") " pod="openshift-console/console-7c4f845644-d8mc5"
Oct 30 19:31:38.439907 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:38.439743    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-tmp\") pod \"console-7c4f845644-d8mc5\" (UID: \"c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3\") " pod="openshift-console/console-7c4f845644-d8mc5"
Oct 30 19:31:38.439907 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:38.439876    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9wcq\" (UniqueName: \"kubernetes.io/projected/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-kube-api-access-f9wcq\") pod \"console-7c4f845644-d8mc5\" (UID: \"c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3\") " pod="openshift-console/console-7c4f845644-d8mc5"
Oct 30 19:31:38.440007 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:38.439935    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-trusted-ca-bundle\") pod \"console-7c4f845644-d8mc5\" (UID: \"c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3\") " pod="openshift-console/console-7c4f845644-d8mc5"
Oct 30 19:31:38.440007 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:38.439960    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-oauth-serving-cert\") pod \"console-7c4f845644-d8mc5\" (UID: \"c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3\") " pod="openshift-console/console-7c4f845644-d8mc5"
Oct 30 19:31:38.440007 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:38.439987    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-console-config\") pod \"console-7c4f845644-d8mc5\" (UID: \"c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3\") " pod="openshift-console/console-7c4f845644-d8mc5"
Oct 30 19:31:38.440157 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:38.440011    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-service-ca\") pod \"console-7c4f845644-d8mc5\" (UID: \"c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3\") " pod="openshift-console/console-7c4f845644-d8mc5"
Oct 30 19:31:38.541200 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:38.541172    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-console-oauth-config\") pod \"console-7c4f845644-d8mc5\" (UID: \"c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3\") " pod="openshift-console/console-7c4f845644-d8mc5"
Oct 30 19:31:38.541290 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:38.541211    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-console-serving-cert\") pod \"console-7c4f845644-d8mc5\" (UID: \"c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3\") " pod="openshift-console/console-7c4f845644-d8mc5"
Oct 30 19:31:38.541290 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:38.541228    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-tmp\") pod \"console-7c4f845644-d8mc5\" (UID: \"c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3\") " pod="openshift-console/console-7c4f845644-d8mc5"
Oct 30 19:31:38.541290 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:38.541273    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9wcq\" (UniqueName: \"kubernetes.io/projected/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-kube-api-access-f9wcq\") pod \"console-7c4f845644-d8mc5\" (UID: \"c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3\") " pod="openshift-console/console-7c4f845644-d8mc5"
Oct 30 19:31:38.541454 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:38.541302    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-trusted-ca-bundle\") pod \"console-7c4f845644-d8mc5\" (UID: \"c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3\") " pod="openshift-console/console-7c4f845644-d8mc5"
Oct 30 19:31:38.541454 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:38.541331    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-oauth-serving-cert\") pod \"console-7c4f845644-d8mc5\" (UID: \"c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3\") " pod="openshift-console/console-7c4f845644-d8mc5"
Oct 30 19:31:38.541454 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:38.541356    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-console-config\") pod \"console-7c4f845644-d8mc5\" (UID: \"c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3\") " pod="openshift-console/console-7c4f845644-d8mc5"
Oct 30 19:31:38.541454 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:38.541376    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-service-ca\") pod \"console-7c4f845644-d8mc5\" (UID: \"c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3\") " pod="openshift-console/console-7c4f845644-d8mc5"
Oct 30 19:31:38.541983 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:38.541959    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-service-ca\") pod \"console-7c4f845644-d8mc5\" (UID: \"c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3\") " pod="openshift-console/console-7c4f845644-d8mc5"
Oct 30 19:31:38.542704 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:38.542679    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-tmp\") pod \"console-7c4f845644-d8mc5\" (UID: \"c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3\") " pod="openshift-console/console-7c4f845644-d8mc5"
Oct 30 19:31:38.543214 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:38.543186    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-oauth-serving-cert\") pod \"console-7c4f845644-d8mc5\" (UID: \"c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3\") " pod="openshift-console/console-7c4f845644-d8mc5"
Oct 30 19:31:38.543596 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:38.543572    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-trusted-ca-bundle\") pod \"console-7c4f845644-d8mc5\" (UID: \"c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3\") " pod="openshift-console/console-7c4f845644-d8mc5"
Oct 30 19:31:38.543678 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:38.543622    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-console-config\") pod \"console-7c4f845644-d8mc5\" (UID: \"c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3\") " pod="openshift-console/console-7c4f845644-d8mc5"
Oct 30 19:31:38.544509 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:38.544470    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-j4cdl_6eca1a1e-101a-43bb-9fde-505c3182c10a/node-ca/0.log"
Oct 30 19:31:38.545122 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:38.545100    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-console-oauth-config\") pod \"console-7c4f845644-d8mc5\" (UID: \"c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3\") " pod="openshift-console/console-7c4f845644-d8mc5"
Oct 30 19:31:38.547141 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:38.547117    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-console-serving-cert\") pod \"console-7c4f845644-d8mc5\" (UID: \"c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3\") " pod="openshift-console/console-7c4f845644-d8mc5"
Oct 30 19:31:38.556830 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:38.556799    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9wcq\" (UniqueName: \"kubernetes.io/projected/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-kube-api-access-f9wcq\") pod \"console-7c4f845644-d8mc5\" (UID: \"c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3\") " pod="openshift-console/console-7c4f845644-d8mc5"
Oct 30 19:31:38.696618 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:38.696584    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c4f845644-d8mc5"
Oct 30 19:31:38.729621 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:38.729581    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8dn55" event={"ID":"7bb33e32-f8ac-44a2-95a4-cb949b08710f","Type":"ContainerStarted","Data":"6f4021cef5b950b49254c7f006cab21ce42450f8b31b6285b50411c48ef3648b"}
Oct 30 19:31:38.729781 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:38.729652    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8dn55" event={"ID":"7bb33e32-f8ac-44a2-95a4-cb949b08710f","Type":"ContainerStarted","Data":"836aca45f50c7ebb31401c76191859fc5ff4070e1b8b2b7492a37e947989bc37"}
Oct 30 19:31:38.730118 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:38.730049    2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-8dn55"
Oct 30 19:31:38.978713 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:38.976928    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/dns-default-8dn55" podStartSLOduration=2.079685492 podStartE2EDuration="3.976907941s" totalImagesPullingTime="1.897222449s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:31:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:31:38.762456881 +0000 UTC m=+45.819240341" watchObservedRunningTime="2026-10-30 19:31:38.976907941 +0000 UTC m=+46.033691400"
Oct 30 19:31:38.978713 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:38.978049    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c4f845644-d8mc5"]
Oct 30 19:31:38.990527 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:31:38.990499    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc59434a3_f0f2_4807_bd5b_65b9b2bbc2e3.slice/crio-54b8f021991e26a526673f6dff681f418709cea4abb400a0abbdfec24cf722be WatchSource:0}: Error finding container 54b8f021991e26a526673f6dff681f418709cea4abb400a0abbdfec24cf722be: Status 404 returned error can't find the container with id 54b8f021991e26a526673f6dff681f418709cea4abb400a0abbdfec24cf722be
Oct 30 19:31:39.734683 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:39.734627    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-8hjtk" event={"ID":"517d6b00-0305-40cc-95c3-bba4adb2f849","Type":"ContainerStarted","Data":"5cd38aaa80309d59415b5ac6039e943171cdeb0113c4d00ca78723c2f5c8a7c8"}
Oct 30 19:31:39.736929 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:39.736894    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qdkpz" event={"ID":"559fa506-ed9a-4c61-9316-edc85d6c5399","Type":"ContainerStarted","Data":"0e36fbff755a30b1f03aa8edfaed2042c895c7d23c194f65963df8134052ed76"}
Oct 30 19:31:39.738227 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:39.738199    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c4f845644-d8mc5" event={"ID":"c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3","Type":"ContainerStarted","Data":"54b8f021991e26a526673f6dff681f418709cea4abb400a0abbdfec24cf722be"}
Oct 30 19:31:39.757981 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:39.757941    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-8hjtk" podStartSLOduration=2.041491717 podStartE2EDuration="4.757928509s" totalImagesPullingTime="2.716436792s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:31:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:31:39.757665201 +0000 UTC m=+46.814448661" watchObservedRunningTime="2026-10-30 19:31:39.757928509 +0000 UTC m=+46.814711967"
Oct 30 19:31:39.789973 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:39.789929    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-qdkpz" podStartSLOduration=2.532684207 podStartE2EDuration="4.789913346s" totalImagesPullingTime="2.257229139s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:31:35 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:31:39.789199722 +0000 UTC m=+46.845983181" watchObservedRunningTime="2026-10-30 19:31:39.789913346 +0000 UTC m=+46.846696803"
Oct 30 19:31:41.745091 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:41.745052    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c4f845644-d8mc5" event={"ID":"c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3","Type":"ContainerStarted","Data":"fca9ef27e3b7963e624d67c1297ec7b88e24a10df49955387f9a99896554fbc7"}
Oct 30 19:31:41.772763 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:41.772718    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-7c4f845644-d8mc5" podStartSLOduration=1.178858269 podStartE2EDuration="3.772704149s" totalImagesPullingTime="2.59384588s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:31:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:31:41.772061018 +0000 UTC m=+48.828844476" watchObservedRunningTime="2026-10-30 19:31:41.772704149 +0000 UTC m=+48.829487607"
Oct 30 19:31:42.406974 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:42.406937    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-nq89t"]
Oct 30 19:31:42.408896 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:42.408874    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-nq89t"
Oct 30 19:31:42.419520 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:42.419500    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 30 19:31:42.421399 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:42.421384    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 19:31:42.421714 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:42.421692    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-md7hj\""
Oct 30 19:31:42.421839 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:42.421734    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Oct 30 19:31:42.421839 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:42.421775    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 30 19:31:42.421839 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:42.421790    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Oct 30 19:31:42.423217 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:42.423194    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-nq89t"]
Oct 30 19:31:42.426191 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:42.426170    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-sx8t4"]
Oct 30 19:31:42.428349 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:42.428332    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-sx8t4"
Oct 30 19:31:42.473064 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:42.473037    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8cde7f30-1d5b-4bea-ad21-2264b567ca1d-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-nq89t\" (UID: \"8cde7f30-1d5b-4bea-ad21-2264b567ca1d\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-nq89t"
Oct 30 19:31:42.473064 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:42.473066    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd-sys\") pod \"node-exporter-sx8t4\" (UID: \"0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd\") " pod="openshift-monitoring/node-exporter-sx8t4"
Oct 30 19:31:42.473233 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:42.473087    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sx8t4\" (UID: \"0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd\") " pod="openshift-monitoring/node-exporter-sx8t4"
Oct 30 19:31:42.473233 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:42.473148    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd-node-exporter-textfile\") pod \"node-exporter-sx8t4\" (UID: \"0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd\") " pod="openshift-monitoring/node-exporter-sx8t4"
Oct 30 19:31:42.473233 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:42.473176    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8cde7f30-1d5b-4bea-ad21-2264b567ca1d-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-nq89t\" (UID: \"8cde7f30-1d5b-4bea-ad21-2264b567ca1d\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-nq89t"
Oct 30 19:31:42.473233 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:42.473193    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd-root\") pod \"node-exporter-sx8t4\" (UID: \"0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd\") " pod="openshift-monitoring/node-exporter-sx8t4"
Oct 30 19:31:42.473394 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:42.473280    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lphsg\" (UniqueName: \"kubernetes.io/projected/0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd-kube-api-access-lphsg\") pod \"node-exporter-sx8t4\" (UID: \"0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd\") " pod="openshift-monitoring/node-exporter-sx8t4"
Oct 30 19:31:42.473394 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:42.473313    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8cde7f30-1d5b-4bea-ad21-2264b567ca1d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-nq89t\" (UID: \"8cde7f30-1d5b-4bea-ad21-2264b567ca1d\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-nq89t"
Oct 30 19:31:42.473394 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:42.473349    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd-node-exporter-accelerators-collector-config\") pod \"node-exporter-sx8t4\" (UID: \"0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd\") " pod="openshift-monitoring/node-exporter-sx8t4"
Oct 30 19:31:42.473394 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:42.473377    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd-node-exporter-tls\") pod \"node-exporter-sx8t4\" (UID: \"0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd\") " pod="openshift-monitoring/node-exporter-sx8t4"
Oct 30 19:31:42.473550 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:42.473401    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd-node-exporter-wtmp\") pod \"node-exporter-sx8t4\" (UID: \"0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd\") " pod="openshift-monitoring/node-exporter-sx8t4"
Oct 30 19:31:42.473550 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:42.473506    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd-metrics-client-ca\") pod \"node-exporter-sx8t4\" (UID: \"0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd\") " pod="openshift-monitoring/node-exporter-sx8t4"
Oct 30 19:31:42.473618 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:42.473555    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5j8qc\" (UniqueName: \"kubernetes.io/projected/8cde7f30-1d5b-4bea-ad21-2264b567ca1d-kube-api-access-5j8qc\") pod \"openshift-state-metrics-7bd7489957-nq89t\" (UID: \"8cde7f30-1d5b-4bea-ad21-2264b567ca1d\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-nq89t"
Oct 30 19:31:42.493663 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:42.493641    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 30 19:31:42.493812 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:42.493789    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 30 19:31:42.493867 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:42.493824    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-vzztc\""
Oct 30 19:31:42.519881 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:42.519859    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 30 19:31:42.574254 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:42.574232    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd-node-exporter-tls\") pod \"node-exporter-sx8t4\" (UID: \"0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd\") " pod="openshift-monitoring/node-exporter-sx8t4"
Oct 30 19:31:42.574346 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:42.574267    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd-node-exporter-wtmp\") pod \"node-exporter-sx8t4\" (UID: \"0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd\") " pod="openshift-monitoring/node-exporter-sx8t4"
Oct 30 19:31:42.574346 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:42.574306    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd-metrics-client-ca\") pod \"node-exporter-sx8t4\" (UID: \"0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd\") " pod="openshift-monitoring/node-exporter-sx8t4"
Oct 30 19:31:42.574346 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:42.574332    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5j8qc\" (UniqueName: \"kubernetes.io/projected/8cde7f30-1d5b-4bea-ad21-2264b567ca1d-kube-api-access-5j8qc\") pod \"openshift-state-metrics-7bd7489957-nq89t\" (UID: \"8cde7f30-1d5b-4bea-ad21-2264b567ca1d\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-nq89t"
Oct 30 19:31:42.574466 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:42.574371    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8cde7f30-1d5b-4bea-ad21-2264b567ca1d-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-nq89t\" (UID: \"8cde7f30-1d5b-4bea-ad21-2264b567ca1d\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-nq89t"
Oct 30 19:31:42.574466 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:42.574394    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd-sys\") pod \"node-exporter-sx8t4\" (UID: \"0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd\") " pod="openshift-monitoring/node-exporter-sx8t4"
Oct 30 19:31:42.574466 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:42.574421    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sx8t4\" (UID: \"0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd\") " pod="openshift-monitoring/node-exporter-sx8t4"
Oct 30 19:31:42.574466 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:42.574439    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd-node-exporter-wtmp\") pod \"node-exporter-sx8t4\" (UID: \"0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd\") " pod="openshift-monitoring/node-exporter-sx8t4"
Oct 30 19:31:42.574671 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:42.574508    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd-sys\") pod \"node-exporter-sx8t4\" (UID: \"0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd\") " pod="openshift-monitoring/node-exporter-sx8t4"
Oct 30 19:31:42.574671 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:42.574514    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd-node-exporter-textfile\") pod \"node-exporter-sx8t4\" (UID: \"0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd\") " pod="openshift-monitoring/node-exporter-sx8t4"
Oct 30 19:31:42.574671 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:42.574595    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8cde7f30-1d5b-4bea-ad21-2264b567ca1d-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-nq89t\" (UID: \"8cde7f30-1d5b-4bea-ad21-2264b567ca1d\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-nq89t"
Oct 30 19:31:42.574671 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:42.574625    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd-root\") pod \"node-exporter-sx8t4\" (UID: \"0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd\") " pod="openshift-monitoring/node-exporter-sx8t4"
Oct 30 19:31:42.574671 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:42.574653    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lphsg\" (UniqueName: \"kubernetes.io/projected/0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd-kube-api-access-lphsg\") pod \"node-exporter-sx8t4\" (UID: \"0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd\") " pod="openshift-monitoring/node-exporter-sx8t4"
Oct 30 19:31:42.574904 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:42.574688    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8cde7f30-1d5b-4bea-ad21-2264b567ca1d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-nq89t\" (UID: \"8cde7f30-1d5b-4bea-ad21-2264b567ca1d\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-nq89t"
Oct 30 19:31:42.574904 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:42.574730    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd-node-exporter-accelerators-collector-config\") pod \"node-exporter-sx8t4\" (UID: \"0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd\") " pod="openshift-monitoring/node-exporter-sx8t4"
Oct 30 19:31:42.574904 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:42.574813    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd-node-exporter-textfile\") pod \"node-exporter-sx8t4\" (UID: \"0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd\") " pod="openshift-monitoring/node-exporter-sx8t4"
Oct 30 19:31:42.575046 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:42.575003    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd-metrics-client-ca\") pod \"node-exporter-sx8t4\" (UID: \"0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd\") " pod="openshift-monitoring/node-exporter-sx8t4"
Oct 30 19:31:42.575097 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:42.575074    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd-root\") pod \"node-exporter-sx8t4\" (UID: \"0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd\") " pod="openshift-monitoring/node-exporter-sx8t4"
Oct 30 19:31:42.575196 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:42.575174    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8cde7f30-1d5b-4bea-ad21-2264b567ca1d-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-nq89t\" (UID: \"8cde7f30-1d5b-4bea-ad21-2264b567ca1d\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-nq89t"
Oct 30 19:31:42.575353 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:42.575308    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd-node-exporter-accelerators-collector-config\") pod \"node-exporter-sx8t4\" (UID: \"0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd\") " pod="openshift-monitoring/node-exporter-sx8t4"
Oct 30 19:31:42.577102 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:42.577079    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sx8t4\" (UID: \"0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd\") " pod="openshift-monitoring/node-exporter-sx8t4"
Oct 30 19:31:42.577196 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:42.577174    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8cde7f30-1d5b-4bea-ad21-2264b567ca1d-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-nq89t\" (UID: \"8cde7f30-1d5b-4bea-ad21-2264b567ca1d\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-nq89t"
Oct 30 19:31:42.577395 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:42.577376    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd-node-exporter-tls\") pod \"node-exporter-sx8t4\" (UID: \"0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd\") " pod="openshift-monitoring/node-exporter-sx8t4"
Oct 30 19:31:42.577511 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:42.577496    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8cde7f30-1d5b-4bea-ad21-2264b567ca1d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-nq89t\" (UID: \"8cde7f30-1d5b-4bea-ad21-2264b567ca1d\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-nq89t"
Oct 30 19:31:42.588702 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:42.588679    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5j8qc\" (UniqueName: \"kubernetes.io/projected/8cde7f30-1d5b-4bea-ad21-2264b567ca1d-kube-api-access-5j8qc\") pod \"openshift-state-metrics-7bd7489957-nq89t\" (UID: \"8cde7f30-1d5b-4bea-ad21-2264b567ca1d\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-nq89t"
Oct 30 19:31:42.588835 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:42.588817    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lphsg\" (UniqueName: \"kubernetes.io/projected/0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd-kube-api-access-lphsg\") pod \"node-exporter-sx8t4\" (UID: \"0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd\") " pod="openshift-monitoring/node-exporter-sx8t4"
Oct 30 19:31:42.720608 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:42.720541    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-nq89t"
Oct 30 19:31:42.740854 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:42.740829    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-sx8t4"
Oct 30 19:31:42.750761 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:31:42.750732    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b3b1fc9_d34f_46dd_a354_ef07ab2b37fd.slice/crio-4f542d1caf4aeb13175f8810167ccf8611f0265dbcf47e71bdebf1c575d22e7f WatchSource:0}: Error finding container 4f542d1caf4aeb13175f8810167ccf8611f0265dbcf47e71bdebf1c575d22e7f: Status 404 returned error can't find the container with id 4f542d1caf4aeb13175f8810167ccf8611f0265dbcf47e71bdebf1c575d22e7f
Oct 30 19:31:43.009522 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:43.009497    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-nq89t"]
Oct 30 19:31:43.011942 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:31:43.011911    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cde7f30_1d5b_4bea_ad21_2264b567ca1d.slice/crio-f1ea5994ad4b3958ca822900d5f83d172b0e2279bbea9cf345f46ef1abf4f082 WatchSource:0}: Error finding container f1ea5994ad4b3958ca822900d5f83d172b0e2279bbea9cf345f46ef1abf4f082: Status 404 returned error can't find the container with id f1ea5994ad4b3958ca822900d5f83d172b0e2279bbea9cf345f46ef1abf4f082
Oct 30 19:31:43.754166 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:43.754126    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-nq89t" event={"ID":"8cde7f30-1d5b-4bea-ad21-2264b567ca1d","Type":"ContainerStarted","Data":"27a034c655360b84f52d055ff6df2458518b430f028c12826a3825ab9765fd17"}
Oct 30 19:31:43.754657 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:43.754212    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-nq89t" event={"ID":"8cde7f30-1d5b-4bea-ad21-2264b567ca1d","Type":"ContainerStarted","Data":"68447e2e2a47a4fb880b3c427ddbcc2d529fbd63975240e146bc01661dc1b511"}
Oct 30 19:31:43.754657 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:43.754250    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-nq89t" event={"ID":"8cde7f30-1d5b-4bea-ad21-2264b567ca1d","Type":"ContainerStarted","Data":"f1ea5994ad4b3958ca822900d5f83d172b0e2279bbea9cf345f46ef1abf4f082"}
Oct 30 19:31:43.755471 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:43.755439    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sx8t4" event={"ID":"0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd","Type":"ContainerStarted","Data":"4f542d1caf4aeb13175f8810167ccf8611f0265dbcf47e71bdebf1c575d22e7f"}
Oct 30 19:31:44.545350 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:44.545316    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-996bc4dbd-wr65k"]
Oct 30 19:31:44.547511 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:44.547492    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-996bc4dbd-wr65k"
Oct 30 19:31:44.553076 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:44.553051    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Oct 30 19:31:44.553076 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:44.553063    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Oct 30 19:31:44.554869 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:44.554846    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-pkmls\""
Oct 30 19:31:44.554990 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:44.554873    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Oct 30 19:31:44.554990 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:44.554875    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Oct 30 19:31:44.554990 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:44.554904    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-55h96gg718jqk\""
Oct 30 19:31:44.554990 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:44.554906    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Oct 30 19:31:44.565756 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:44.565737    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-996bc4dbd-wr65k"]
Oct 30 19:31:44.592406 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:44.592376    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/94ecba26-0809-414d-b084-8de0a693795e-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-996bc4dbd-wr65k\" (UID: \"94ecba26-0809-414d-b084-8de0a693795e\") " pod="openshift-monitoring/thanos-querier-996bc4dbd-wr65k"
Oct 30 19:31:44.592525 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:44.592410    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/94ecba26-0809-414d-b084-8de0a693795e-secret-grpc-tls\") pod \"thanos-querier-996bc4dbd-wr65k\" (UID: \"94ecba26-0809-414d-b084-8de0a693795e\") " pod="openshift-monitoring/thanos-querier-996bc4dbd-wr65k"
Oct 30 19:31:44.592525 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:44.592442    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/94ecba26-0809-414d-b084-8de0a693795e-tmp\") pod \"thanos-querier-996bc4dbd-wr65k\" (UID: \"94ecba26-0809-414d-b084-8de0a693795e\") " pod="openshift-monitoring/thanos-querier-996bc4dbd-wr65k"
Oct 30 19:31:44.592613 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:44.592554    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/94ecba26-0809-414d-b084-8de0a693795e-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-996bc4dbd-wr65k\" (UID: \"94ecba26-0809-414d-b084-8de0a693795e\") " pod="openshift-monitoring/thanos-querier-996bc4dbd-wr65k"
Oct 30 19:31:44.592672 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:44.592650    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/94ecba26-0809-414d-b084-8de0a693795e-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-996bc4dbd-wr65k\" (UID: \"94ecba26-0809-414d-b084-8de0a693795e\") " pod="openshift-monitoring/thanos-querier-996bc4dbd-wr65k"
Oct 30 19:31:44.592720 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:44.592703    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x98xh\" (UniqueName: \"kubernetes.io/projected/94ecba26-0809-414d-b084-8de0a693795e-kube-api-access-x98xh\") pod \"thanos-querier-996bc4dbd-wr65k\" (UID: \"94ecba26-0809-414d-b084-8de0a693795e\") " pod="openshift-monitoring/thanos-querier-996bc4dbd-wr65k"
Oct 30 19:31:44.592783 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:44.592767    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/94ecba26-0809-414d-b084-8de0a693795e-metrics-client-ca\") pod \"thanos-querier-996bc4dbd-wr65k\" (UID: \"94ecba26-0809-414d-b084-8de0a693795e\") " pod="openshift-monitoring/thanos-querier-996bc4dbd-wr65k"
Oct 30 19:31:44.592816 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:44.592801    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/94ecba26-0809-414d-b084-8de0a693795e-secret-thanos-querier-tls\") pod \"thanos-querier-996bc4dbd-wr65k\" (UID: \"94ecba26-0809-414d-b084-8de0a693795e\") " pod="openshift-monitoring/thanos-querier-996bc4dbd-wr65k"
Oct 30 19:31:44.592845 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:44.592828    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/94ecba26-0809-414d-b084-8de0a693795e-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-996bc4dbd-wr65k\" (UID: \"94ecba26-0809-414d-b084-8de0a693795e\") " pod="openshift-monitoring/thanos-querier-996bc4dbd-wr65k"
Oct 30 19:31:44.693385 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:44.693355    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/94ecba26-0809-414d-b084-8de0a693795e-tmp\") pod \"thanos-querier-996bc4dbd-wr65k\" (UID: \"94ecba26-0809-414d-b084-8de0a693795e\") " pod="openshift-monitoring/thanos-querier-996bc4dbd-wr65k"
Oct 30 19:31:44.693568 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:44.693397    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/94ecba26-0809-414d-b084-8de0a693795e-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-996bc4dbd-wr65k\" (UID: \"94ecba26-0809-414d-b084-8de0a693795e\") " pod="openshift-monitoring/thanos-querier-996bc4dbd-wr65k"
Oct 30 19:31:44.693568 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:44.693434    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/94ecba26-0809-414d-b084-8de0a693795e-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-996bc4dbd-wr65k\" (UID: \"94ecba26-0809-414d-b084-8de0a693795e\") " pod="openshift-monitoring/thanos-querier-996bc4dbd-wr65k"
Oct 30 19:31:44.693568 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:44.693519    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x98xh\" (UniqueName: \"kubernetes.io/projected/94ecba26-0809-414d-b084-8de0a693795e-kube-api-access-x98xh\") pod \"thanos-querier-996bc4dbd-wr65k\" (UID: \"94ecba26-0809-414d-b084-8de0a693795e\") " pod="openshift-monitoring/thanos-querier-996bc4dbd-wr65k"
Oct 30 19:31:44.693732 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:44.693579    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/94ecba26-0809-414d-b084-8de0a693795e-metrics-client-ca\") pod \"thanos-querier-996bc4dbd-wr65k\" (UID: \"94ecba26-0809-414d-b084-8de0a693795e\") " pod="openshift-monitoring/thanos-querier-996bc4dbd-wr65k"
Oct 30 19:31:44.693732 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:44.693604    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/94ecba26-0809-414d-b084-8de0a693795e-secret-thanos-querier-tls\") pod \"thanos-querier-996bc4dbd-wr65k\" (UID: \"94ecba26-0809-414d-b084-8de0a693795e\") " pod="openshift-monitoring/thanos-querier-996bc4dbd-wr65k"
Oct 30 19:31:44.693732 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:44.693635    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/94ecba26-0809-414d-b084-8de0a693795e-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-996bc4dbd-wr65k\" (UID: \"94ecba26-0809-414d-b084-8de0a693795e\") " pod="openshift-monitoring/thanos-querier-996bc4dbd-wr65k"
Oct 30 19:31:44.693732 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:44.693670    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/94ecba26-0809-414d-b084-8de0a693795e-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-996bc4dbd-wr65k\" (UID: \"94ecba26-0809-414d-b084-8de0a693795e\") " pod="openshift-monitoring/thanos-querier-996bc4dbd-wr65k"
Oct 30 19:31:44.693732 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:44.693689    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/94ecba26-0809-414d-b084-8de0a693795e-secret-grpc-tls\") pod \"thanos-querier-996bc4dbd-wr65k\" (UID: \"94ecba26-0809-414d-b084-8de0a693795e\") " pod="openshift-monitoring/thanos-querier-996bc4dbd-wr65k"
Oct 30 19:31:44.693973 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:44.693875    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/94ecba26-0809-414d-b084-8de0a693795e-tmp\") pod \"thanos-querier-996bc4dbd-wr65k\" (UID: \"94ecba26-0809-414d-b084-8de0a693795e\") " pod="openshift-monitoring/thanos-querier-996bc4dbd-wr65k"
Oct 30 19:31:44.694354 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:44.694323    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/94ecba26-0809-414d-b084-8de0a693795e-metrics-client-ca\") pod \"thanos-querier-996bc4dbd-wr65k\" (UID: \"94ecba26-0809-414d-b084-8de0a693795e\") " pod="openshift-monitoring/thanos-querier-996bc4dbd-wr65k"
Oct 30 19:31:44.695886 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:44.695865    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/94ecba26-0809-414d-b084-8de0a693795e-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-996bc4dbd-wr65k\" (UID: \"94ecba26-0809-414d-b084-8de0a693795e\") " pod="openshift-monitoring/thanos-querier-996bc4dbd-wr65k"
Oct 30 19:31:44.696025 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:44.696003    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/94ecba26-0809-414d-b084-8de0a693795e-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-996bc4dbd-wr65k\" (UID: \"94ecba26-0809-414d-b084-8de0a693795e\") " pod="openshift-monitoring/thanos-querier-996bc4dbd-wr65k"
Oct 30 19:31:44.696524 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:44.696453    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/94ecba26-0809-414d-b084-8de0a693795e-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-996bc4dbd-wr65k\" (UID: \"94ecba26-0809-414d-b084-8de0a693795e\") " pod="openshift-monitoring/thanos-querier-996bc4dbd-wr65k"
Oct 30 19:31:44.696605 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:44.696556    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/94ecba26-0809-414d-b084-8de0a693795e-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-996bc4dbd-wr65k\" (UID: \"94ecba26-0809-414d-b084-8de0a693795e\") " pod="openshift-monitoring/thanos-querier-996bc4dbd-wr65k"
Oct 30 19:31:44.696605 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:44.696601    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/94ecba26-0809-414d-b084-8de0a693795e-secret-thanos-querier-tls\") pod \"thanos-querier-996bc4dbd-wr65k\" (UID: \"94ecba26-0809-414d-b084-8de0a693795e\") " pod="openshift-monitoring/thanos-querier-996bc4dbd-wr65k"
Oct 30 19:31:44.696806 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:44.696789    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/94ecba26-0809-414d-b084-8de0a693795e-secret-grpc-tls\") pod \"thanos-querier-996bc4dbd-wr65k\" (UID: \"94ecba26-0809-414d-b084-8de0a693795e\") " pod="openshift-monitoring/thanos-querier-996bc4dbd-wr65k"
Oct 30 19:31:44.707086 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:44.707066    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x98xh\" (UniqueName: \"kubernetes.io/projected/94ecba26-0809-414d-b084-8de0a693795e-kube-api-access-x98xh\") pod \"thanos-querier-996bc4dbd-wr65k\" (UID: \"94ecba26-0809-414d-b084-8de0a693795e\") " pod="openshift-monitoring/thanos-querier-996bc4dbd-wr65k"
Oct 30 19:31:44.760509 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:44.760462    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-nq89t" event={"ID":"8cde7f30-1d5b-4bea-ad21-2264b567ca1d","Type":"ContainerStarted","Data":"733cd1ad8ffc90930e333571137d8f1dfba2fb81cce6b9f1c44379e6892f7fb3"}
Oct 30 19:31:44.762015 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:44.761994    2578 generic.go:356] "Generic (PLEG): container finished" podID="0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd" containerID="7bffe48f200a9e527f3f41a2e90681541f98710b76ce283d380d916857687dc4" exitCode=0
Oct 30 19:31:44.762097 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:44.762038    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sx8t4" event={"ID":"0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd","Type":"ContainerDied","Data":"7bffe48f200a9e527f3f41a2e90681541f98710b76ce283d380d916857687dc4"}
Oct 30 19:31:44.786156 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:44.786118    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-nq89t" podStartSLOduration=1.8591289309999999 podStartE2EDuration="2.786106635s" totalImagesPullingTime="926.977704ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:31:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:31:44.785408273 +0000 UTC m=+51.842191732" watchObservedRunningTime="2026-10-30 19:31:44.786106635 +0000 UTC m=+51.842890093"
Oct 30 19:31:44.862949 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:44.862928    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-996bc4dbd-wr65k"
Oct 30 19:31:45.175115 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:45.175081    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-996bc4dbd-wr65k"]
Oct 30 19:31:45.182301 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:31:45.182276    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94ecba26_0809_414d_b084_8de0a693795e.slice/crio-5959efc147c29abce752d39e65fffae0a2335eec259a1e3de2618a12d358e116 WatchSource:0}: Error finding container 5959efc147c29abce752d39e65fffae0a2335eec259a1e3de2618a12d358e116: Status 404 returned error can't find the container with id 5959efc147c29abce752d39e65fffae0a2335eec259a1e3de2618a12d358e116
Oct 30 19:31:45.767726 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:45.767416    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sx8t4" event={"ID":"0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd","Type":"ContainerStarted","Data":"f2f209be70ab15e77bd0728d3c86a14609e256c709a736e44f9beb93ec77c11b"}
Oct 30 19:31:45.767726 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:45.767499    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sx8t4" event={"ID":"0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd","Type":"ContainerStarted","Data":"1e6d6688bd46dc84ddb7db34594c4206b01dce0755d8ae00410ed696bb7efe0a"}
Oct 30 19:31:45.768813 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:45.768787    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-996bc4dbd-wr65k" event={"ID":"94ecba26-0809-414d-b084-8de0a693795e","Type":"ContainerStarted","Data":"5959efc147c29abce752d39e65fffae0a2335eec259a1e3de2618a12d358e116"}
Oct 30 19:31:45.797074 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:45.797018    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-sx8t4" podStartSLOduration=2.676035836 podStartE2EDuration="3.797000142s" totalImagesPullingTime="1.045334821s" totalInitContainerRuntime="75.629485ms" isStatefulPod=true podCreationTimestamp="2026-10-30 19:31:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:31:45.794920086 +0000 UTC m=+52.851703542" watchObservedRunningTime="2026-10-30 19:31:45.797000142 +0000 UTC m=+52.853783602"
Oct 30 19:31:46.886674 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:46.886630    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5cb8677769-45w2n"]
Oct 30 19:31:46.888919 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:46.888892    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5cb8677769-45w2n"
Oct 30 19:31:46.898425 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:46.898385    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Oct 30 19:31:46.898573 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:46.898465    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Oct 30 19:31:46.904450 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:46.904392    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5cb8677769-45w2n"]
Oct 30 19:31:46.912400 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:46.912379    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Oct 30 19:31:46.912644 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:46.912623    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Oct 30 19:31:46.912754 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:46.912674    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-4l7fj\""
Oct 30 19:31:46.914449 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:46.914426    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-cgn2r6prpq3cp\""
Oct 30 19:31:46.915818 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:46.915794    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb-audit-log\") pod \"metrics-server-5cb8677769-45w2n\" (UID: \"136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb\") " pod="openshift-monitoring/metrics-server-5cb8677769-45w2n"
Oct 30 19:31:46.915925 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:46.915844    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb-secret-metrics-server-client-certs\") pod \"metrics-server-5cb8677769-45w2n\" (UID: \"136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb\") " pod="openshift-monitoring/metrics-server-5cb8677769-45w2n"
Oct 30 19:31:46.915925 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:46.915881    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb-secret-metrics-server-tls\") pod \"metrics-server-5cb8677769-45w2n\" (UID: \"136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb\") " pod="openshift-monitoring/metrics-server-5cb8677769-45w2n"
Oct 30 19:31:46.916013 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:46.915933    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb-metrics-server-audit-profiles\") pod \"metrics-server-5cb8677769-45w2n\" (UID: \"136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb\") " pod="openshift-monitoring/metrics-server-5cb8677769-45w2n"
Oct 30 19:31:46.916057 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:46.916007    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5cb8677769-45w2n\" (UID: \"136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb\") " pod="openshift-monitoring/metrics-server-5cb8677769-45w2n"
Oct 30 19:31:46.916057 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:46.916052    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdcn4\" (UniqueName: \"kubernetes.io/projected/136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb-kube-api-access-wdcn4\") pod \"metrics-server-5cb8677769-45w2n\" (UID: \"136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb\") " pod="openshift-monitoring/metrics-server-5cb8677769-45w2n"
Oct 30 19:31:46.916130 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:46.916103    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb-client-ca-bundle\") pod \"metrics-server-5cb8677769-45w2n\" (UID: \"136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb\") " pod="openshift-monitoring/metrics-server-5cb8677769-45w2n"
Oct 30 19:31:47.016616 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:47.016455    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb-audit-log\") pod \"metrics-server-5cb8677769-45w2n\" (UID: \"136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb\") " pod="openshift-monitoring/metrics-server-5cb8677769-45w2n"
Oct 30 19:31:47.016616 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:47.016532    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb-secret-metrics-server-client-certs\") pod \"metrics-server-5cb8677769-45w2n\" (UID: \"136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb\") " pod="openshift-monitoring/metrics-server-5cb8677769-45w2n"
Oct 30 19:31:47.016616 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:47.016585    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb-secret-metrics-server-tls\") pod \"metrics-server-5cb8677769-45w2n\" (UID: \"136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb\") " pod="openshift-monitoring/metrics-server-5cb8677769-45w2n"
Oct 30 19:31:47.016780 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:47.016614    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb-metrics-server-audit-profiles\") pod \"metrics-server-5cb8677769-45w2n\" (UID: \"136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb\") " pod="openshift-monitoring/metrics-server-5cb8677769-45w2n"
Oct 30 19:31:47.016780 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:47.016642    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5cb8677769-45w2n\" (UID: \"136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb\") " pod="openshift-monitoring/metrics-server-5cb8677769-45w2n"
Oct 30 19:31:47.016780 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:47.016678    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdcn4\" (UniqueName: \"kubernetes.io/projected/136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb-kube-api-access-wdcn4\") pod \"metrics-server-5cb8677769-45w2n\" (UID: \"136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb\") " pod="openshift-monitoring/metrics-server-5cb8677769-45w2n"
Oct 30 19:31:47.016780 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:47.016762    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb-client-ca-bundle\") pod \"metrics-server-5cb8677769-45w2n\" (UID: \"136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb\") " pod="openshift-monitoring/metrics-server-5cb8677769-45w2n"
Oct 30 19:31:47.016983 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:47.016825    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb-audit-log\") pod \"metrics-server-5cb8677769-45w2n\" (UID: \"136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb\") " pod="openshift-monitoring/metrics-server-5cb8677769-45w2n"
Oct 30 19:31:47.017729 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:47.017705    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb-metrics-server-audit-profiles\") pod \"metrics-server-5cb8677769-45w2n\" (UID: \"136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb\") " pod="openshift-monitoring/metrics-server-5cb8677769-45w2n"
Oct 30 19:31:47.018230 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:47.018207    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5cb8677769-45w2n\" (UID: \"136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb\") " pod="openshift-monitoring/metrics-server-5cb8677769-45w2n"
Oct 30 19:31:47.019050 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:47.019028    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb-secret-metrics-server-client-certs\") pod \"metrics-server-5cb8677769-45w2n\" (UID: \"136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb\") " pod="openshift-monitoring/metrics-server-5cb8677769-45w2n"
Oct 30 19:31:47.019439 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:47.019416    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb-secret-metrics-server-tls\") pod \"metrics-server-5cb8677769-45w2n\" (UID: \"136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb\") " pod="openshift-monitoring/metrics-server-5cb8677769-45w2n"
Oct 30 19:31:47.019895 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:47.019872    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb-client-ca-bundle\") pod \"metrics-server-5cb8677769-45w2n\" (UID: \"136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb\") " pod="openshift-monitoring/metrics-server-5cb8677769-45w2n"
Oct 30 19:31:47.046751 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:47.046722    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdcn4\" (UniqueName: \"kubernetes.io/projected/136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb-kube-api-access-wdcn4\") pod \"metrics-server-5cb8677769-45w2n\" (UID: \"136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb\") " pod="openshift-monitoring/metrics-server-5cb8677769-45w2n"
Oct 30 19:31:47.202504 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:47.202420    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5cb8677769-45w2n"
Oct 30 19:31:47.243372 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:47.243341    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-ph577"]
Oct 30 19:31:47.246019 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:47.245997    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-ph577"
Oct 30 19:31:47.262322 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:47.262218    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-j67kr\""
Oct 30 19:31:47.262470 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:47.262452    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Oct 30 19:31:47.262470 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:47.262228    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-ph577"]
Oct 30 19:31:47.319951 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:47.319916    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmz8l\" (UniqueName: \"kubernetes.io/projected/1cc9603f-1c61-43e4-b987-0e185cf26252-kube-api-access-bmz8l\") pod \"monitoring-plugin-77f4f76594-ph577\" (UID: \"1cc9603f-1c61-43e4-b987-0e185cf26252\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-ph577"
Oct 30 19:31:47.320096 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:47.319979    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1cc9603f-1c61-43e4-b987-0e185cf26252-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-ph577\" (UID: \"1cc9603f-1c61-43e4-b987-0e185cf26252\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-ph577"
Oct 30 19:31:47.420596 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:47.420565    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmz8l\" (UniqueName: \"kubernetes.io/projected/1cc9603f-1c61-43e4-b987-0e185cf26252-kube-api-access-bmz8l\") pod \"monitoring-plugin-77f4f76594-ph577\" (UID: \"1cc9603f-1c61-43e4-b987-0e185cf26252\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-ph577"
Oct 30 19:31:47.420743 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:47.420628    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1cc9603f-1c61-43e4-b987-0e185cf26252-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-ph577\" (UID: \"1cc9603f-1c61-43e4-b987-0e185cf26252\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-ph577"
Oct 30 19:31:47.423457 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:47.423432    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1cc9603f-1c61-43e4-b987-0e185cf26252-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-ph577\" (UID: \"1cc9603f-1c61-43e4-b987-0e185cf26252\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-ph577"
Oct 30 19:31:47.435134 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:47.435115    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmz8l\" (UniqueName: \"kubernetes.io/projected/1cc9603f-1c61-43e4-b987-0e185cf26252-kube-api-access-bmz8l\") pod \"monitoring-plugin-77f4f76594-ph577\" (UID: \"1cc9603f-1c61-43e4-b987-0e185cf26252\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-ph577"
Oct 30 19:31:47.475221 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:47.475168    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5cb8677769-45w2n"]
Oct 30 19:31:47.477694 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:31:47.477662    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod136dc928_99f8_48d6_ae3b_7b7cf5d6bfeb.slice/crio-8371052014d19ca2a7b06e031c1c5aebd406a27e12a242c1773973b46689359a WatchSource:0}: Error finding container 8371052014d19ca2a7b06e031c1c5aebd406a27e12a242c1773973b46689359a: Status 404 returned error can't find the container with id 8371052014d19ca2a7b06e031c1c5aebd406a27e12a242c1773973b46689359a
Oct 30 19:31:47.558892 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:47.558863    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-ph577"
Oct 30 19:31:47.704197 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:47.704160    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-6d56c5d745-9xkj5"]
Oct 30 19:31:47.706702 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:47.706680    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6d56c5d745-9xkj5"
Oct 30 19:31:47.714414 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:47.714219    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Oct 30 19:31:47.714735 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:47.714681    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Oct 30 19:31:47.717050 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:47.717033    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-z26kr\""
Oct 30 19:31:47.726998 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:47.726944    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Oct 30 19:31:47.732714 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:47.732691    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-28r105lbnph2c\""
Oct 30 19:31:47.738136 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:47.738104    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6d56c5d745-9xkj5"]
Oct 30 19:31:47.777197 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:47.777153    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-996bc4dbd-wr65k" event={"ID":"94ecba26-0809-414d-b084-8de0a693795e","Type":"ContainerStarted","Data":"87b9ce0537d4a76cfcafe87acfe06d0f7bb25193ea136694025de39479a2e1d5"}
Oct 30 19:31:47.777375 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:47.777228    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-996bc4dbd-wr65k" event={"ID":"94ecba26-0809-414d-b084-8de0a693795e","Type":"ContainerStarted","Data":"9f719a7b79f2af5388a68a40f6b4534b95e3d4de2e4f482b815c4b89c0a752e5"}
Oct 30 19:31:47.777375 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:47.777269    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-996bc4dbd-wr65k" event={"ID":"94ecba26-0809-414d-b084-8de0a693795e","Type":"ContainerStarted","Data":"e8ac1d24f194e891cd7af8f62400232a4e8b0f8fcdcce4936f8bf95d484885db"}
Oct 30 19:31:47.778430 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:47.778406    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5cb8677769-45w2n" event={"ID":"136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb","Type":"ContainerStarted","Data":"8371052014d19ca2a7b06e031c1c5aebd406a27e12a242c1773973b46689359a"}
Oct 30 19:31:47.814135 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:47.814115    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Oct 30 19:31:47.814261 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:47.814209    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Oct 30 19:31:47.823730 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:47.823691    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/bfd79074-21c8-4caf-8c63-a2c5ff45e26d-telemeter-client-tls\") pod \"telemeter-client-6d56c5d745-9xkj5\" (UID: \"bfd79074-21c8-4caf-8c63-a2c5ff45e26d\") " pod="openshift-monitoring/telemeter-client-6d56c5d745-9xkj5"
Oct 30 19:31:47.823826 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:47.823734    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bfd79074-21c8-4caf-8c63-a2c5ff45e26d-metrics-client-ca\") pod \"telemeter-client-6d56c5d745-9xkj5\" (UID: \"bfd79074-21c8-4caf-8c63-a2c5ff45e26d\") " pod="openshift-monitoring/telemeter-client-6d56c5d745-9xkj5"
Oct 30 19:31:47.823863 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:47.823827    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bfd79074-21c8-4caf-8c63-a2c5ff45e26d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6d56c5d745-9xkj5\" (UID: \"bfd79074-21c8-4caf-8c63-a2c5ff45e26d\") " pod="openshift-monitoring/telemeter-client-6d56c5d745-9xkj5"
Oct 30 19:31:47.823863 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:47.823849    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlnsz\" (UniqueName: \"kubernetes.io/projected/bfd79074-21c8-4caf-8c63-a2c5ff45e26d-kube-api-access-xlnsz\") pod \"telemeter-client-6d56c5d745-9xkj5\" (UID: \"bfd79074-21c8-4caf-8c63-a2c5ff45e26d\") " pod="openshift-monitoring/telemeter-client-6d56c5d745-9xkj5"
Oct 30 19:31:47.823928 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:47.823869    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bfd79074-21c8-4caf-8c63-a2c5ff45e26d-serving-certs-ca-bundle\") pod \"telemeter-client-6d56c5d745-9xkj5\" (UID: \"bfd79074-21c8-4caf-8c63-a2c5ff45e26d\") " pod="openshift-monitoring/telemeter-client-6d56c5d745-9xkj5"
Oct 30 19:31:47.824005 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:47.823985    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/bfd79074-21c8-4caf-8c63-a2c5ff45e26d-federate-client-tls\") pod \"telemeter-client-6d56c5d745-9xkj5\" (UID: \"bfd79074-21c8-4caf-8c63-a2c5ff45e26d\") " pod="openshift-monitoring/telemeter-client-6d56c5d745-9xkj5"
Oct 30 19:31:47.824037 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:47.824024    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bfd79074-21c8-4caf-8c63-a2c5ff45e26d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6d56c5d745-9xkj5\" (UID: \"bfd79074-21c8-4caf-8c63-a2c5ff45e26d\") " pod="openshift-monitoring/telemeter-client-6d56c5d745-9xkj5"
Oct 30 19:31:47.824134 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:47.824114    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/bfd79074-21c8-4caf-8c63-a2c5ff45e26d-secret-telemeter-client\") pod \"telemeter-client-6d56c5d745-9xkj5\" (UID: \"bfd79074-21c8-4caf-8c63-a2c5ff45e26d\") " pod="openshift-monitoring/telemeter-client-6d56c5d745-9xkj5"
Oct 30 19:31:47.834910 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:47.834880    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-ph577"]
Oct 30 19:31:47.837183 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:31:47.837161    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cc9603f_1c61_43e4_b987_0e185cf26252.slice/crio-a92e741ee1642fdb90a23191249ff6f142e0f8bdc25b987a15900a41b61c4dcd WatchSource:0}: Error finding container a92e741ee1642fdb90a23191249ff6f142e0f8bdc25b987a15900a41b61c4dcd: Status 404 returned error can't find the container with id a92e741ee1642fdb90a23191249ff6f142e0f8bdc25b987a15900a41b61c4dcd
Oct 30 19:31:47.924992 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:47.924968    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bfd79074-21c8-4caf-8c63-a2c5ff45e26d-serving-certs-ca-bundle\") pod \"telemeter-client-6d56c5d745-9xkj5\" (UID: \"bfd79074-21c8-4caf-8c63-a2c5ff45e26d\") " pod="openshift-monitoring/telemeter-client-6d56c5d745-9xkj5"
Oct 30 19:31:47.925334 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:47.925008    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/bfd79074-21c8-4caf-8c63-a2c5ff45e26d-federate-client-tls\") pod \"telemeter-client-6d56c5d745-9xkj5\" (UID: \"bfd79074-21c8-4caf-8c63-a2c5ff45e26d\") " pod="openshift-monitoring/telemeter-client-6d56c5d745-9xkj5"
Oct 30 19:31:47.925334 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:47.925033    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bfd79074-21c8-4caf-8c63-a2c5ff45e26d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6d56c5d745-9xkj5\" (UID: \"bfd79074-21c8-4caf-8c63-a2c5ff45e26d\") " pod="openshift-monitoring/telemeter-client-6d56c5d745-9xkj5"
Oct 30 19:31:47.925334 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:47.925085    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/bfd79074-21c8-4caf-8c63-a2c5ff45e26d-secret-telemeter-client\") pod \"telemeter-client-6d56c5d745-9xkj5\" (UID: \"bfd79074-21c8-4caf-8c63-a2c5ff45e26d\") " pod="openshift-monitoring/telemeter-client-6d56c5d745-9xkj5"
Oct 30 19:31:47.925334 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:47.925219    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/bfd79074-21c8-4caf-8c63-a2c5ff45e26d-telemeter-client-tls\") pod \"telemeter-client-6d56c5d745-9xkj5\" (UID: \"bfd79074-21c8-4caf-8c63-a2c5ff45e26d\") " pod="openshift-monitoring/telemeter-client-6d56c5d745-9xkj5"
Oct 30 19:31:47.925334 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:47.925248    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bfd79074-21c8-4caf-8c63-a2c5ff45e26d-metrics-client-ca\") pod \"telemeter-client-6d56c5d745-9xkj5\" (UID: \"bfd79074-21c8-4caf-8c63-a2c5ff45e26d\") " pod="openshift-monitoring/telemeter-client-6d56c5d745-9xkj5"
Oct 30 19:31:47.925334 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:47.925305    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bfd79074-21c8-4caf-8c63-a2c5ff45e26d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6d56c5d745-9xkj5\" (UID: \"bfd79074-21c8-4caf-8c63-a2c5ff45e26d\") " pod="openshift-monitoring/telemeter-client-6d56c5d745-9xkj5"
Oct 30 19:31:47.925334 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:47.925333    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlnsz\" (UniqueName: \"kubernetes.io/projected/bfd79074-21c8-4caf-8c63-a2c5ff45e26d-kube-api-access-xlnsz\") pod \"telemeter-client-6d56c5d745-9xkj5\" (UID: \"bfd79074-21c8-4caf-8c63-a2c5ff45e26d\") " pod="openshift-monitoring/telemeter-client-6d56c5d745-9xkj5"
Oct 30 19:31:47.925777 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:47.925751    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bfd79074-21c8-4caf-8c63-a2c5ff45e26d-serving-certs-ca-bundle\") pod \"telemeter-client-6d56c5d745-9xkj5\" (UID: \"bfd79074-21c8-4caf-8c63-a2c5ff45e26d\") " pod="openshift-monitoring/telemeter-client-6d56c5d745-9xkj5"
Oct 30 19:31:47.926124 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:47.926069    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bfd79074-21c8-4caf-8c63-a2c5ff45e26d-metrics-client-ca\") pod \"telemeter-client-6d56c5d745-9xkj5\" (UID: \"bfd79074-21c8-4caf-8c63-a2c5ff45e26d\") " pod="openshift-monitoring/telemeter-client-6d56c5d745-9xkj5"
Oct 30 19:31:47.926225 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:47.926199    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bfd79074-21c8-4caf-8c63-a2c5ff45e26d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6d56c5d745-9xkj5\" (UID: \"bfd79074-21c8-4caf-8c63-a2c5ff45e26d\") " pod="openshift-monitoring/telemeter-client-6d56c5d745-9xkj5"
Oct 30 19:31:47.928071 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:47.928049    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bfd79074-21c8-4caf-8c63-a2c5ff45e26d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6d56c5d745-9xkj5\" (UID: \"bfd79074-21c8-4caf-8c63-a2c5ff45e26d\") " pod="openshift-monitoring/telemeter-client-6d56c5d745-9xkj5"
Oct 30 19:31:47.928410 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:47.928390    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/bfd79074-21c8-4caf-8c63-a2c5ff45e26d-secret-telemeter-client\") pod \"telemeter-client-6d56c5d745-9xkj5\" (UID: \"bfd79074-21c8-4caf-8c63-a2c5ff45e26d\") " pod="openshift-monitoring/telemeter-client-6d56c5d745-9xkj5"
Oct 30 19:31:47.928451 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:47.928407    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/bfd79074-21c8-4caf-8c63-a2c5ff45e26d-federate-client-tls\") pod \"telemeter-client-6d56c5d745-9xkj5\" (UID: \"bfd79074-21c8-4caf-8c63-a2c5ff45e26d\") " pod="openshift-monitoring/telemeter-client-6d56c5d745-9xkj5"
Oct 30 19:31:47.930198 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:47.930178    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/bfd79074-21c8-4caf-8c63-a2c5ff45e26d-telemeter-client-tls\") pod \"telemeter-client-6d56c5d745-9xkj5\" (UID: \"bfd79074-21c8-4caf-8c63-a2c5ff45e26d\") " pod="openshift-monitoring/telemeter-client-6d56c5d745-9xkj5"
Oct 30 19:31:47.942703 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:47.942438    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlnsz\" (UniqueName: \"kubernetes.io/projected/bfd79074-21c8-4caf-8c63-a2c5ff45e26d-kube-api-access-xlnsz\") pod \"telemeter-client-6d56c5d745-9xkj5\" (UID: \"bfd79074-21c8-4caf-8c63-a2c5ff45e26d\") " pod="openshift-monitoring/telemeter-client-6d56c5d745-9xkj5"
Oct 30 19:31:48.020673 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.020592    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6d56c5d745-9xkj5"
Oct 30 19:31:48.192363 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.192330    2578 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c4f845644-d8mc5"]
Oct 30 19:31:48.223293 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.223268    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-6479499c8f-ptgvc"]
Oct 30 19:31:48.225756 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.225737    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6479499c8f-ptgvc"
Oct 30 19:31:48.243282 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.243258    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6479499c8f-ptgvc"]
Oct 30 19:31:48.328372 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.328339    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1f3e7563-7109-45a5-9a77-7676a0697bd1-console-serving-cert\") pod \"console-6479499c8f-ptgvc\" (UID: \"1f3e7563-7109-45a5-9a77-7676a0697bd1\") " pod="openshift-console/console-6479499c8f-ptgvc"
Oct 30 19:31:48.328562 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.328512    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1f3e7563-7109-45a5-9a77-7676a0697bd1-service-ca\") pod \"console-6479499c8f-ptgvc\" (UID: \"1f3e7563-7109-45a5-9a77-7676a0697bd1\") " pod="openshift-console/console-6479499c8f-ptgvc"
Oct 30 19:31:48.328562 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.328534    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mqpt\" (UniqueName: \"kubernetes.io/projected/1f3e7563-7109-45a5-9a77-7676a0697bd1-kube-api-access-2mqpt\") pod \"console-6479499c8f-ptgvc\" (UID: \"1f3e7563-7109-45a5-9a77-7676a0697bd1\") " pod="openshift-console/console-6479499c8f-ptgvc"
Oct 30 19:31:48.328723 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.328574    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1f3e7563-7109-45a5-9a77-7676a0697bd1-console-oauth-config\") pod \"console-6479499c8f-ptgvc\" (UID: \"1f3e7563-7109-45a5-9a77-7676a0697bd1\") " pod="openshift-console/console-6479499c8f-ptgvc"
Oct 30 19:31:48.328723 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.328692    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1f3e7563-7109-45a5-9a77-7676a0697bd1-console-config\") pod \"console-6479499c8f-ptgvc\" (UID: \"1f3e7563-7109-45a5-9a77-7676a0697bd1\") " pod="openshift-console/console-6479499c8f-ptgvc"
Oct 30 19:31:48.328802 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.328771    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1f3e7563-7109-45a5-9a77-7676a0697bd1-oauth-serving-cert\") pod \"console-6479499c8f-ptgvc\" (UID: \"1f3e7563-7109-45a5-9a77-7676a0697bd1\") " pod="openshift-console/console-6479499c8f-ptgvc"
Oct 30 19:31:48.328856 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.328810    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f3e7563-7109-45a5-9a77-7676a0697bd1-trusted-ca-bundle\") pod \"console-6479499c8f-ptgvc\" (UID: \"1f3e7563-7109-45a5-9a77-7676a0697bd1\") " pod="openshift-console/console-6479499c8f-ptgvc"
Oct 30 19:31:48.328917 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.328897    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1f3e7563-7109-45a5-9a77-7676a0697bd1-tmp\") pod \"console-6479499c8f-ptgvc\" (UID: \"1f3e7563-7109-45a5-9a77-7676a0697bd1\") " pod="openshift-console/console-6479499c8f-ptgvc"
Oct 30 19:31:48.349842 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.349814    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6d56c5d745-9xkj5"]
Oct 30 19:31:48.356144 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:31:48.356117    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfd79074_21c8_4caf_8c63_a2c5ff45e26d.slice/crio-4c388040c36db938ab6e8b9aa5380695690ebfc3798a1baba340df88847d47dc WatchSource:0}: Error finding container 4c388040c36db938ab6e8b9aa5380695690ebfc3798a1baba340df88847d47dc: Status 404 returned error can't find the container with id 4c388040c36db938ab6e8b9aa5380695690ebfc3798a1baba340df88847d47dc
Oct 30 19:31:48.429983 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.429945    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f3e7563-7109-45a5-9a77-7676a0697bd1-trusted-ca-bundle\") pod \"console-6479499c8f-ptgvc\" (UID: \"1f3e7563-7109-45a5-9a77-7676a0697bd1\") " pod="openshift-console/console-6479499c8f-ptgvc"
Oct 30 19:31:48.430214 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.430011    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1f3e7563-7109-45a5-9a77-7676a0697bd1-tmp\") pod \"console-6479499c8f-ptgvc\" (UID: \"1f3e7563-7109-45a5-9a77-7676a0697bd1\") " pod="openshift-console/console-6479499c8f-ptgvc"
Oct 30 19:31:48.430214 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.430059    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1f3e7563-7109-45a5-9a77-7676a0697bd1-console-serving-cert\") pod \"console-6479499c8f-ptgvc\" (UID: \"1f3e7563-7109-45a5-9a77-7676a0697bd1\") " pod="openshift-console/console-6479499c8f-ptgvc"
Oct 30 19:31:48.430214 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.430111    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1f3e7563-7109-45a5-9a77-7676a0697bd1-service-ca\") pod \"console-6479499c8f-ptgvc\" (UID: \"1f3e7563-7109-45a5-9a77-7676a0697bd1\") " pod="openshift-console/console-6479499c8f-ptgvc"
Oct 30 19:31:48.430214 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.430131    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mqpt\" (UniqueName: \"kubernetes.io/projected/1f3e7563-7109-45a5-9a77-7676a0697bd1-kube-api-access-2mqpt\") pod \"console-6479499c8f-ptgvc\" (UID: \"1f3e7563-7109-45a5-9a77-7676a0697bd1\") " pod="openshift-console/console-6479499c8f-ptgvc"
Oct 30 19:31:48.430214 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.430188    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1f3e7563-7109-45a5-9a77-7676a0697bd1-console-oauth-config\") pod \"console-6479499c8f-ptgvc\" (UID: \"1f3e7563-7109-45a5-9a77-7676a0697bd1\") " pod="openshift-console/console-6479499c8f-ptgvc"
Oct 30 19:31:48.430474 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.430215    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1f3e7563-7109-45a5-9a77-7676a0697bd1-console-config\") pod \"console-6479499c8f-ptgvc\" (UID: \"1f3e7563-7109-45a5-9a77-7676a0697bd1\") " pod="openshift-console/console-6479499c8f-ptgvc"
Oct 30 19:31:48.430474 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.430265    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1f3e7563-7109-45a5-9a77-7676a0697bd1-oauth-serving-cert\") pod \"console-6479499c8f-ptgvc\" (UID: \"1f3e7563-7109-45a5-9a77-7676a0697bd1\") " pod="openshift-console/console-6479499c8f-ptgvc"
Oct 30 19:31:48.430614 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.430579    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1f3e7563-7109-45a5-9a77-7676a0697bd1-tmp\") pod \"console-6479499c8f-ptgvc\" (UID: \"1f3e7563-7109-45a5-9a77-7676a0697bd1\") " pod="openshift-console/console-6479499c8f-ptgvc"
Oct 30 19:31:48.431000 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.430949    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1f3e7563-7109-45a5-9a77-7676a0697bd1-service-ca\") pod \"console-6479499c8f-ptgvc\" (UID: \"1f3e7563-7109-45a5-9a77-7676a0697bd1\") " pod="openshift-console/console-6479499c8f-ptgvc"
Oct 30 19:31:48.431124 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.431025    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f3e7563-7109-45a5-9a77-7676a0697bd1-trusted-ca-bundle\") pod \"console-6479499c8f-ptgvc\" (UID: \"1f3e7563-7109-45a5-9a77-7676a0697bd1\") " pod="openshift-console/console-6479499c8f-ptgvc"
Oct 30 19:31:48.431731 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.431702    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1f3e7563-7109-45a5-9a77-7676a0697bd1-oauth-serving-cert\") pod \"console-6479499c8f-ptgvc\" (UID: \"1f3e7563-7109-45a5-9a77-7676a0697bd1\") " pod="openshift-console/console-6479499c8f-ptgvc"
Oct 30 19:31:48.431878 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.431859    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1f3e7563-7109-45a5-9a77-7676a0697bd1-console-config\") pod \"console-6479499c8f-ptgvc\" (UID: \"1f3e7563-7109-45a5-9a77-7676a0697bd1\") " pod="openshift-console/console-6479499c8f-ptgvc"
Oct 30 19:31:48.433809 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.433785    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1f3e7563-7109-45a5-9a77-7676a0697bd1-console-oauth-config\") pod \"console-6479499c8f-ptgvc\" (UID: \"1f3e7563-7109-45a5-9a77-7676a0697bd1\") " pod="openshift-console/console-6479499c8f-ptgvc"
Oct 30 19:31:48.433809 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.433793    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1f3e7563-7109-45a5-9a77-7676a0697bd1-console-serving-cert\") pod \"console-6479499c8f-ptgvc\" (UID: \"1f3e7563-7109-45a5-9a77-7676a0697bd1\") " pod="openshift-console/console-6479499c8f-ptgvc"
Oct 30 19:31:48.445068 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.445047    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mqpt\" (UniqueName: \"kubernetes.io/projected/1f3e7563-7109-45a5-9a77-7676a0697bd1-kube-api-access-2mqpt\") pod \"console-6479499c8f-ptgvc\" (UID: \"1f3e7563-7109-45a5-9a77-7676a0697bd1\") " pod="openshift-console/console-6479499c8f-ptgvc"
Oct 30 19:31:48.537354 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.537318    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6479499c8f-ptgvc"
Oct 30 19:31:48.697019 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.696986    2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7c4f845644-d8mc5"
Oct 30 19:31:48.741207 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.741171    2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-8dn55"
Oct 30 19:31:48.799823 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.799426    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-996bc4dbd-wr65k" event={"ID":"94ecba26-0809-414d-b084-8de0a693795e","Type":"ContainerStarted","Data":"da2567c5293836ba9bba74cb6e7e4a71a16469689bea929f788f7078c8d29832"}
Oct 30 19:31:48.799823 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.799540    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-996bc4dbd-wr65k" event={"ID":"94ecba26-0809-414d-b084-8de0a693795e","Type":"ContainerStarted","Data":"01e3d4d7d3546cdf4ef45f8c5b0dee7d49b544755cd9a5d1fc69858187df7849"}
Oct 30 19:31:48.799823 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.799587    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-996bc4dbd-wr65k" event={"ID":"94ecba26-0809-414d-b084-8de0a693795e","Type":"ContainerStarted","Data":"9479c4416499d424e80d7b7104851f7cbe55a089b3efdc86682b0720b54d6c73"}
Oct 30 19:31:48.799823 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.799656    2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-996bc4dbd-wr65k"
Oct 30 19:31:48.801254 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.801194    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6d56c5d745-9xkj5" event={"ID":"bfd79074-21c8-4caf-8c63-a2c5ff45e26d","Type":"ContainerStarted","Data":"4c388040c36db938ab6e8b9aa5380695690ebfc3798a1baba340df88847d47dc"}
Oct 30 19:31:48.803704 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.803682    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-ph577" event={"ID":"1cc9603f-1c61-43e4-b987-0e185cf26252","Type":"ContainerStarted","Data":"a92e741ee1642fdb90a23191249ff6f142e0f8bdc25b987a15900a41b61c4dcd"}
Oct 30 19:31:48.823422 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.823120    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6479499c8f-ptgvc"]
Oct 30 19:31:48.825429 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:31:48.825391    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f3e7563_7109_45a5_9a77_7676a0697bd1.slice/crio-e841b85a1079cb72d30538e44a703671f61625f93f932c45d8658bb417102983 WatchSource:0}: Error finding container e841b85a1079cb72d30538e44a703671f61625f93f932c45d8658bb417102983: Status 404 returned error can't find the container with id e841b85a1079cb72d30538e44a703671f61625f93f932c45d8658bb417102983
Oct 30 19:31:48.833350 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.833298    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-996bc4dbd-wr65k" podStartSLOduration=2.375651631 podStartE2EDuration="4.833280867s" totalImagesPullingTime="2.457629236s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:31:44 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:31:48.831784546 +0000 UTC m=+55.888568006" watchObservedRunningTime="2026-10-30 19:31:48.833280867 +0000 UTC m=+55.890064327"
Oct 30 19:31:48.861116 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.861092    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 19:31:48.864116 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.864094    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:31:48.878135 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.876696    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Oct 30 19:31:48.878135 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.876775    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Oct 30 19:31:48.878135 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.876790    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Oct 30 19:31:48.878135 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.876911    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Oct 30 19:31:48.880052 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.879332    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Oct 30 19:31:48.880052 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.879393    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Oct 30 19:31:48.880052 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.879460    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Oct 30 19:31:48.880052 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.879469    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Oct 30 19:31:48.880052 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.879612    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-f6ia1krs5s708\""
Oct 30 19:31:48.880052 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.879332    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Oct 30 19:31:48.880052 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.879397    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Oct 30 19:31:48.880052 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.879684    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Oct 30 19:31:48.880052 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.879764    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Oct 30 19:31:48.886930 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.886753    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Oct 30 19:31:48.886930 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.886874    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-gxpqn\""
Oct 30 19:31:48.886930 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.886902    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Oct 30 19:31:48.891727 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.891384    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 19:31:48.936421 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.936312    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:31:48.936421 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.936394    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:31:48.936989 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.936552    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:31:48.936989 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.936590    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/06b45974-f6ef-4b3a-b354-8103d7895fcf-config-out\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:31:48.936989 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.936640    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:31:48.936989 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.936684    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-web-config\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:31:48.936989 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.936739    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:31:48.936989 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.936795    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/06b45974-f6ef-4b3a-b354-8103d7895fcf-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:31:48.936989 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.936825    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:31:48.936989 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.936850    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:31:48.936989 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.936873    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:31:48.937521 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.937030    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:31:48.937521 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.937061    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:31:48.937521 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.937086    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/06b45974-f6ef-4b3a-b354-8103d7895fcf-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:31:48.937521 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.937187    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:31:48.937521 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.937229    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:31:48.937521 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.937254    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w94gr\" (UniqueName: \"kubernetes.io/projected/06b45974-f6ef-4b3a-b354-8103d7895fcf-kube-api-access-w94gr\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:31:48.937521 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.937286    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:31:48.937521 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.937309    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:31:48.937521 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:48.937336    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-config\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:31:49.038074 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:49.038039    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:31:49.038245 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:49.038104    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/06b45974-f6ef-4b3a-b354-8103d7895fcf-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:31:49.038245 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:49.038131    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:31:49.038245 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:49.038154    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:31:49.038245 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:49.038185    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:31:49.042290 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:49.038698    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:31:49.042290 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:49.038745    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:31:49.042290 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:49.038780    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/06b45974-f6ef-4b3a-b354-8103d7895fcf-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:31:49.042290 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:49.038826    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:31:49.042290 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:49.038922    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:31:49.042290 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:49.038952    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w94gr\" (UniqueName: \"kubernetes.io/projected/06b45974-f6ef-4b3a-b354-8103d7895fcf-kube-api-access-w94gr\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:31:49.042290 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:49.038984    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:31:49.042290 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:49.039007    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:31:49.042290 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:49.039036    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-config\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:31:49.042290 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:49.039068    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:31:49.042290 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:49.039084    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:31:49.042290 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:49.039150    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:31:49.042290 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:49.039211    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:31:49.042290 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:49.039242    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/06b45974-f6ef-4b3a-b354-8103d7895fcf-config-out\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:31:49.042290 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:49.039270    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:31:49.042290 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:49.039294    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-web-config\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:31:49.042290 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:49.039736    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:31:49.043250 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:49.040464    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:31:49.043250 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:49.040944    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/06b45974-f6ef-4b3a-b354-8103d7895fcf-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:31:49.043250 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:49.041213    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:31:49.043250 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:49.041305    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:31:49.043250 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:49.041644    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:31:49.043250 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:49.041982    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/06b45974-f6ef-4b3a-b354-8103d7895fcf-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:31:49.043250 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:49.042156    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:31:49.043250 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:49.042274    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:31:49.045290 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:49.045263    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:31:49.045402 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:49.045328    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/06b45974-f6ef-4b3a-b354-8103d7895fcf-config-out\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:31:49.046764 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:49.046740    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:31:49.047170 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:49.047106    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-config\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:31:49.047331 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:49.047314    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:31:49.049000 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:49.048978    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-web-config\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:31:49.049382 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:49.049354    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:31:49.049458 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:49.049384    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:31:49.051106 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:49.051087    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:31:49.055680 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:49.055657    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w94gr\" (UniqueName: \"kubernetes.io/projected/06b45974-f6ef-4b3a-b354-8103d7895fcf-kube-api-access-w94gr\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:31:49.212547 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:49.212509    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:31:49.526314 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:49.526283    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 19:31:49.535368 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:31:49.535341    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06b45974_f6ef_4b3a_b354_8103d7895fcf.slice/crio-32a5d73fd496680155bec45a71c11338e31bfbe4529f4f057c9ff41e97d1fbe7 WatchSource:0}: Error finding container 32a5d73fd496680155bec45a71c11338e31bfbe4529f4f057c9ff41e97d1fbe7: Status 404 returned error can't find the container with id 32a5d73fd496680155bec45a71c11338e31bfbe4529f4f057c9ff41e97d1fbe7
Oct 30 19:31:49.809961 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:49.809880    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5cb8677769-45w2n" event={"ID":"136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb","Type":"ContainerStarted","Data":"d02ce90fbb72e60b6734c2e6c15e43776d3e2b4e9fdbe9af9398c58156d9f49a"}
Oct 30 19:31:49.811409 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:49.811374    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"06b45974-f6ef-4b3a-b354-8103d7895fcf","Type":"ContainerStarted","Data":"32a5d73fd496680155bec45a71c11338e31bfbe4529f4f057c9ff41e97d1fbe7"}
Oct 30 19:31:49.813177 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:49.813147    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6479499c8f-ptgvc" event={"ID":"1f3e7563-7109-45a5-9a77-7676a0697bd1","Type":"ContainerStarted","Data":"8e54b01508a82f4d9f02eb3b249f74aa8e7595af64edf0a09d57075abfbc7fbe"}
Oct 30 19:31:49.813354 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:49.813331    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6479499c8f-ptgvc" event={"ID":"1f3e7563-7109-45a5-9a77-7676a0697bd1","Type":"ContainerStarted","Data":"e841b85a1079cb72d30538e44a703671f61625f93f932c45d8658bb417102983"}
Oct 30 19:31:49.814987 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:49.814959    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-ph577" event={"ID":"1cc9603f-1c61-43e4-b987-0e185cf26252","Type":"ContainerStarted","Data":"94507b1073d51f24e386ce3a07dc6a9319f44dfd45de61ecc00ab0615dd1c89d"}
Oct 30 19:31:49.815262 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:49.815241    2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-ph577"
Oct 30 19:31:49.820711 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:49.820694    2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-ph577"
Oct 30 19:31:49.834031 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:49.833995    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5cb8677769-45w2n" podStartSLOduration=2.250046512 podStartE2EDuration="3.833981968s" totalImagesPullingTime="1.583935456s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:31:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:31:49.833730659 +0000 UTC m=+56.890514120" watchObservedRunningTime="2026-10-30 19:31:49.833981968 +0000 UTC m=+56.890765427"
Oct 30 19:31:49.859195 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:49.859154    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-6479499c8f-ptgvc" podStartSLOduration=1.859142863 podStartE2EDuration="1.859142863s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:31:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:31:49.858724377 +0000 UTC m=+56.915507836" watchObservedRunningTime="2026-10-30 19:31:49.859142863 +0000 UTC m=+56.915926317"
Oct 30 19:31:49.918957 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:49.918906    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-77f4f76594-ph577" podStartSLOduration=1.29783549 podStartE2EDuration="2.918893301s" totalImagesPullingTime="1.621057811s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:31:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:31:49.9175724 +0000 UTC m=+56.974355860" watchObservedRunningTime="2026-10-30 19:31:49.918893301 +0000 UTC m=+56.975676760"
Oct 30 19:31:50.820409 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:50.820377    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6d56c5d745-9xkj5" event={"ID":"bfd79074-21c8-4caf-8c63-a2c5ff45e26d","Type":"ContainerStarted","Data":"a06aca8ec60ea02eca4bbf89e83c984dabbf7d6834bc0eb1f73d8390ee3ac637"}
Oct 30 19:31:50.820796 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:50.820503    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6d56c5d745-9xkj5" event={"ID":"bfd79074-21c8-4caf-8c63-a2c5ff45e26d","Type":"ContainerStarted","Data":"3b7b4d51bbde45b46077307b1eacae30fa34016402c478c551630c23f0cc2e3d"}
Oct 30 19:31:50.823342 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:50.822135    2578 generic.go:356] "Generic (PLEG): container finished" podID="06b45974-f6ef-4b3a-b354-8103d7895fcf" containerID="67eff76e35d54c9bb9f908398b7e410860adfb0a1adc657d551d62f0fc21744a" exitCode=0
Oct 30 19:31:50.823342 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:50.822200    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"06b45974-f6ef-4b3a-b354-8103d7895fcf","Type":"ContainerDied","Data":"67eff76e35d54c9bb9f908398b7e410860adfb0a1adc657d551d62f0fc21744a"}
Oct 30 19:31:51.828729 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:51.828693    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6d56c5d745-9xkj5" event={"ID":"bfd79074-21c8-4caf-8c63-a2c5ff45e26d","Type":"ContainerStarted","Data":"739774aa1ef6653b580ec3b31bc849f437b060b04e13279a46771d7d9e8ccdfe"}
Oct 30 19:31:51.865634 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:51.865575    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-6d56c5d745-9xkj5" podStartSLOduration=2.526684559 podStartE2EDuration="4.865557585s" totalImagesPullingTime="2.338873026s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:31:47 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:31:51.863742148 +0000 UTC m=+58.920525606" watchObservedRunningTime="2026-10-30 19:31:51.865557585 +0000 UTC m=+58.922341042"
Oct 30 19:31:54.820979 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:54.820954    2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-996bc4dbd-wr65k"
Oct 30 19:31:54.843146 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:54.843113    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"06b45974-f6ef-4b3a-b354-8103d7895fcf","Type":"ContainerStarted","Data":"3d4d0dde720d320dd166a3af41fdbd89194f7a53c8a1ab753b25b58c53ee43bc"}
Oct 30 19:31:54.843274 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:54.843187    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"06b45974-f6ef-4b3a-b354-8103d7895fcf","Type":"ContainerStarted","Data":"c57dd406ebf9f954f23ff488744fa6907ed8865079c549ea006b707c0facc082"}
Oct 30 19:31:54.843274 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:54.843226    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"06b45974-f6ef-4b3a-b354-8103d7895fcf","Type":"ContainerStarted","Data":"3d708174472383233ec9987dac6af32cfc3dc15dbcc8a56e1aa40e07bca81f96"}
Oct 30 19:31:54.843274 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:54.843266    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"06b45974-f6ef-4b3a-b354-8103d7895fcf","Type":"ContainerStarted","Data":"2cf12d0db579be0cb78758ca0b67edf3a7a06735a238750e6a05dbd2122c0ed0"}
Oct 30 19:31:54.843363 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:54.843304    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"06b45974-f6ef-4b3a-b354-8103d7895fcf","Type":"ContainerStarted","Data":"e568bf64d537e305e782d862c5636dea7f5a48a98e3054f7a8b24927051daa68"}
Oct 30 19:31:54.843363 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:54.843332    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"06b45974-f6ef-4b3a-b354-8103d7895fcf","Type":"ContainerStarted","Data":"3760d224c3458dbc7a10ac1532d6d30e7d3156efe9d705342fc1f896360461e9"}
Oct 30 19:31:54.901445 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:54.901388    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.92000062 podStartE2EDuration="6.901374301s" totalImagesPullingTime="3.967330918s" totalInitContainerRuntime="14.042763ms" isStatefulPod=false podCreationTimestamp="2026-10-30 19:31:48 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:31:54.898194426 +0000 UTC m=+61.954977912" watchObservedRunningTime="2026-10-30 19:31:54.901374301 +0000 UTC m=+61.958157759"
Oct 30 19:31:56.071097 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:56.069056    2578 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6479499c8f-ptgvc"]
Oct 30 19:31:56.112523 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:56.112364    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-55994d7c46-sz78s"]
Oct 30 19:31:56.123641 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:56.123618    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55994d7c46-sz78s"
Oct 30 19:31:56.135986 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:56.135961    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-55994d7c46-sz78s"]
Oct 30 19:31:56.210965 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:56.210843    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/63effc7d-50b8-425f-b9a8-891ce2259283-service-ca\") pod \"console-55994d7c46-sz78s\" (UID: \"63effc7d-50b8-425f-b9a8-891ce2259283\") " pod="openshift-console/console-55994d7c46-sz78s"
Oct 30 19:31:56.211235 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:56.211200    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/63effc7d-50b8-425f-b9a8-891ce2259283-oauth-serving-cert\") pod \"console-55994d7c46-sz78s\" (UID: \"63effc7d-50b8-425f-b9a8-891ce2259283\") " pod="openshift-console/console-55994d7c46-sz78s"
Oct 30 19:31:56.211399 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:56.211373    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/63effc7d-50b8-425f-b9a8-891ce2259283-console-serving-cert\") pod \"console-55994d7c46-sz78s\" (UID: \"63effc7d-50b8-425f-b9a8-891ce2259283\") " pod="openshift-console/console-55994d7c46-sz78s"
Oct 30 19:31:56.211568 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:56.211541    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/63effc7d-50b8-425f-b9a8-891ce2259283-console-config\") pod \"console-55994d7c46-sz78s\" (UID: \"63effc7d-50b8-425f-b9a8-891ce2259283\") " pod="openshift-console/console-55994d7c46-sz78s"
Oct 30 19:31:56.211706 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:56.211690    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6kdm\" (UniqueName: \"kubernetes.io/projected/63effc7d-50b8-425f-b9a8-891ce2259283-kube-api-access-q6kdm\") pod \"console-55994d7c46-sz78s\" (UID: \"63effc7d-50b8-425f-b9a8-891ce2259283\") " pod="openshift-console/console-55994d7c46-sz78s"
Oct 30 19:31:56.211860 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:56.211843    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/63effc7d-50b8-425f-b9a8-891ce2259283-tmp\") pod \"console-55994d7c46-sz78s\" (UID: \"63effc7d-50b8-425f-b9a8-891ce2259283\") " pod="openshift-console/console-55994d7c46-sz78s"
Oct 30 19:31:56.212047 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:56.212031    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/63effc7d-50b8-425f-b9a8-891ce2259283-console-oauth-config\") pod \"console-55994d7c46-sz78s\" (UID: \"63effc7d-50b8-425f-b9a8-891ce2259283\") " pod="openshift-console/console-55994d7c46-sz78s"
Oct 30 19:31:56.212178 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:56.212162    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/63effc7d-50b8-425f-b9a8-891ce2259283-trusted-ca-bundle\") pod \"console-55994d7c46-sz78s\" (UID: \"63effc7d-50b8-425f-b9a8-891ce2259283\") " pod="openshift-console/console-55994d7c46-sz78s"
Oct 30 19:31:56.313449 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:56.313416    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/63effc7d-50b8-425f-b9a8-891ce2259283-service-ca\") pod \"console-55994d7c46-sz78s\" (UID: \"63effc7d-50b8-425f-b9a8-891ce2259283\") " pod="openshift-console/console-55994d7c46-sz78s"
Oct 30 19:31:56.313449 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:56.313451    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/63effc7d-50b8-425f-b9a8-891ce2259283-oauth-serving-cert\") pod \"console-55994d7c46-sz78s\" (UID: \"63effc7d-50b8-425f-b9a8-891ce2259283\") " pod="openshift-console/console-55994d7c46-sz78s"
Oct 30 19:31:56.313692 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:56.313489    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/63effc7d-50b8-425f-b9a8-891ce2259283-console-serving-cert\") pod \"console-55994d7c46-sz78s\" (UID: \"63effc7d-50b8-425f-b9a8-891ce2259283\") " pod="openshift-console/console-55994d7c46-sz78s"
Oct 30 19:31:56.313692 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:56.313545    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/63effc7d-50b8-425f-b9a8-891ce2259283-console-config\") pod \"console-55994d7c46-sz78s\" (UID: \"63effc7d-50b8-425f-b9a8-891ce2259283\") " pod="openshift-console/console-55994d7c46-sz78s"
Oct 30 19:31:56.313692 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:56.313570    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6kdm\" (UniqueName: \"kubernetes.io/projected/63effc7d-50b8-425f-b9a8-891ce2259283-kube-api-access-q6kdm\") pod \"console-55994d7c46-sz78s\" (UID: \"63effc7d-50b8-425f-b9a8-891ce2259283\") " pod="openshift-console/console-55994d7c46-sz78s"
Oct 30 19:31:56.313692 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:56.313607    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/63effc7d-50b8-425f-b9a8-891ce2259283-tmp\") pod \"console-55994d7c46-sz78s\" (UID: \"63effc7d-50b8-425f-b9a8-891ce2259283\") " pod="openshift-console/console-55994d7c46-sz78s"
Oct 30 19:31:56.313692 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:56.313665    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/63effc7d-50b8-425f-b9a8-891ce2259283-console-oauth-config\") pod \"console-55994d7c46-sz78s\" (UID: \"63effc7d-50b8-425f-b9a8-891ce2259283\") " pod="openshift-console/console-55994d7c46-sz78s"
Oct 30 19:31:56.313938 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:56.313694    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/63effc7d-50b8-425f-b9a8-891ce2259283-trusted-ca-bundle\") pod \"console-55994d7c46-sz78s\" (UID: \"63effc7d-50b8-425f-b9a8-891ce2259283\") " pod="openshift-console/console-55994d7c46-sz78s"
Oct 30 19:31:56.314207 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:56.314184    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/63effc7d-50b8-425f-b9a8-891ce2259283-service-ca\") pod \"console-55994d7c46-sz78s\" (UID: \"63effc7d-50b8-425f-b9a8-891ce2259283\") " pod="openshift-console/console-55994d7c46-sz78s"
Oct 30 19:31:56.314306 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:56.314236    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/63effc7d-50b8-425f-b9a8-891ce2259283-tmp\") pod \"console-55994d7c46-sz78s\" (UID: \"63effc7d-50b8-425f-b9a8-891ce2259283\") " pod="openshift-console/console-55994d7c46-sz78s"
Oct 30 19:31:56.314306 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:56.314297    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/63effc7d-50b8-425f-b9a8-891ce2259283-oauth-serving-cert\") pod \"console-55994d7c46-sz78s\" (UID: \"63effc7d-50b8-425f-b9a8-891ce2259283\") " pod="openshift-console/console-55994d7c46-sz78s"
Oct 30 19:31:56.314411 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:56.314297    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/63effc7d-50b8-425f-b9a8-891ce2259283-console-config\") pod \"console-55994d7c46-sz78s\" (UID: \"63effc7d-50b8-425f-b9a8-891ce2259283\") " pod="openshift-console/console-55994d7c46-sz78s"
Oct 30 19:31:56.314573 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:56.314551    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/63effc7d-50b8-425f-b9a8-891ce2259283-trusted-ca-bundle\") pod \"console-55994d7c46-sz78s\" (UID: \"63effc7d-50b8-425f-b9a8-891ce2259283\") " pod="openshift-console/console-55994d7c46-sz78s"
Oct 30 19:31:56.316378 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:56.316354    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/63effc7d-50b8-425f-b9a8-891ce2259283-console-oauth-config\") pod \"console-55994d7c46-sz78s\" (UID: \"63effc7d-50b8-425f-b9a8-891ce2259283\") " pod="openshift-console/console-55994d7c46-sz78s"
Oct 30 19:31:56.316548 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:56.316521    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/63effc7d-50b8-425f-b9a8-891ce2259283-console-serving-cert\") pod \"console-55994d7c46-sz78s\" (UID: \"63effc7d-50b8-425f-b9a8-891ce2259283\") " pod="openshift-console/console-55994d7c46-sz78s"
Oct 30 19:31:56.328708 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:56.328648    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6kdm\" (UniqueName: \"kubernetes.io/projected/63effc7d-50b8-425f-b9a8-891ce2259283-kube-api-access-q6kdm\") pod \"console-55994d7c46-sz78s\" (UID: \"63effc7d-50b8-425f-b9a8-891ce2259283\") " pod="openshift-console/console-55994d7c46-sz78s"
Oct 30 19:31:56.438430 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:56.438092    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55994d7c46-sz78s"
Oct 30 19:31:56.744798 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:56.739186    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-55994d7c46-sz78s"]
Oct 30 19:31:56.749829 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:31:56.749801    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63effc7d_50b8_425f_b9a8_891ce2259283.slice/crio-b628691bb3e641012cc218b5b75787f95203955ab8fadc0de9627fe887df7842 WatchSource:0}: Error finding container b628691bb3e641012cc218b5b75787f95203955ab8fadc0de9627fe887df7842: Status 404 returned error can't find the container with id b628691bb3e641012cc218b5b75787f95203955ab8fadc0de9627fe887df7842
Oct 30 19:31:56.853866 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:56.853837    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55994d7c46-sz78s" event={"ID":"63effc7d-50b8-425f-b9a8-891ce2259283","Type":"ContainerStarted","Data":"097b3d016393a37abfc842ff60ecf422d2b2f744fcadf541920f245588f8ec01"}
Oct 30 19:31:56.854222 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:56.853883    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55994d7c46-sz78s" event={"ID":"63effc7d-50b8-425f-b9a8-891ce2259283","Type":"ContainerStarted","Data":"b628691bb3e641012cc218b5b75787f95203955ab8fadc0de9627fe887df7842"}
Oct 30 19:31:56.884612 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:56.884555    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-55994d7c46-sz78s" podStartSLOduration=0.884539932 podStartE2EDuration="884.539932ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:31:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:31:56.883592234 +0000 UTC m=+63.940375692" watchObservedRunningTime="2026-10-30 19:31:56.884539932 +0000 UTC m=+63.941323391"
Oct 30 19:31:58.538380 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:58.538348    2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6479499c8f-ptgvc"
Oct 30 19:31:59.145551 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:59.145510    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/82384cc1-894f-4bb6-9b1d-567c72004575-metrics-certs\") pod \"network-metrics-daemon-vlqp7\" (UID: \"82384cc1-894f-4bb6-9b1d-567c72004575\") " pod="openshift-multus/network-metrics-daemon-vlqp7"
Oct 30 19:31:59.149645 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:59.149622    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 19:31:59.157877 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:59.157857    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/82384cc1-894f-4bb6-9b1d-567c72004575-metrics-certs\") pod \"network-metrics-daemon-vlqp7\" (UID: \"82384cc1-894f-4bb6-9b1d-567c72004575\") " pod="openshift-multus/network-metrics-daemon-vlqp7"
Oct 30 19:31:59.213589 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:59.213562    2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:31:59.245352 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:59.245330    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-wr97x\""
Oct 30 19:31:59.246686 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:59.246657    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/141cf3eb-d99b-4876-b65a-a2847584ff5d-original-pull-secret\") pod \"global-pull-secret-syncer-526rb\" (UID: \"141cf3eb-d99b-4876-b65a-a2847584ff5d\") " pod="kube-system/global-pull-secret-syncer-526rb"
Oct 30 19:31:59.250624 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:59.250609    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 19:31:59.251769 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:59.251758    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vlqp7"
Oct 30 19:31:59.259099 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:59.259082    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/141cf3eb-d99b-4876-b65a-a2847584ff5d-original-pull-secret\") pod \"global-pull-secret-syncer-526rb\" (UID: \"141cf3eb-d99b-4876-b65a-a2847584ff5d\") " pod="kube-system/global-pull-secret-syncer-526rb"
Oct 30 19:31:59.347639 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:59.347602    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nn6bt\" (UniqueName: \"kubernetes.io/projected/7ff4ab79-1d7c-49ac-9d63-31772ad68b42-kube-api-access-nn6bt\") pod \"network-check-target-4f5sl\" (UID: \"7ff4ab79-1d7c-49ac-9d63-31772ad68b42\") " pod="openshift-network-diagnostics/network-check-target-4f5sl"
Oct 30 19:31:59.352134 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:59.352114    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 19:31:59.362110 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:59.362093    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 19:31:59.371439 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:59.371416    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nn6bt\" (UniqueName: \"kubernetes.io/projected/7ff4ab79-1d7c-49ac-9d63-31772ad68b42-kube-api-access-nn6bt\") pod \"network-check-target-4f5sl\" (UID: \"7ff4ab79-1d7c-49ac-9d63-31772ad68b42\") " pod="openshift-network-diagnostics/network-check-target-4f5sl"
Oct 30 19:31:59.529353 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:59.529322    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vlqp7"]
Oct 30 19:31:59.533333 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:59.533296    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-rpkw7\""
Oct 30 19:31:59.534391 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:31:59.534368    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82384cc1_894f_4bb6_9b1d_567c72004575.slice/crio-1970eefbe58c6156df633f635d604fa121b88455a5781a5fda64c3537802322a WatchSource:0}: Error finding container 1970eefbe58c6156df633f635d604fa121b88455a5781a5fda64c3537802322a: Status 404 returned error can't find the container with id 1970eefbe58c6156df633f635d604fa121b88455a5781a5fda64c3537802322a
Oct 30 19:31:59.538957 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:59.538941    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-jznq8\""
Oct 30 19:31:59.540020 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:59.540002    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-526rb"
Oct 30 19:31:59.545753 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:59.545736    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4f5sl"
Oct 30 19:31:59.824949 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:59.824927    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4f5sl"]
Oct 30 19:31:59.826837 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:31:59.826810    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ff4ab79_1d7c_49ac_9d63_31772ad68b42.slice/crio-71dda31d74e3da240f255df32253cd6410431f0fbf23733eea76701b3c04fc5e WatchSource:0}: Error finding container 71dda31d74e3da240f255df32253cd6410431f0fbf23733eea76701b3c04fc5e: Status 404 returned error can't find the container with id 71dda31d74e3da240f255df32253cd6410431f0fbf23733eea76701b3c04fc5e
Oct 30 19:31:59.827507 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:59.827451    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-526rb"]
Oct 30 19:31:59.829683 ip-10-0-141-42 kubenswrapper[2578]: W1030 19:31:59.829656    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod141cf3eb_d99b_4876_b65a_a2847584ff5d.slice/crio-1374ca566c18177a9872baff538f680e5090425d1dc15b2ecc1e65d5929c2237 WatchSource:0}: Error finding container 1374ca566c18177a9872baff538f680e5090425d1dc15b2ecc1e65d5929c2237: Status 404 returned error can't find the container with id 1374ca566c18177a9872baff538f680e5090425d1dc15b2ecc1e65d5929c2237
Oct 30 19:31:59.865877 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:59.865851    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4f5sl" event={"ID":"7ff4ab79-1d7c-49ac-9d63-31772ad68b42","Type":"ContainerStarted","Data":"71dda31d74e3da240f255df32253cd6410431f0fbf23733eea76701b3c04fc5e"}
Oct 30 19:31:59.867043 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:59.867018    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vlqp7" event={"ID":"82384cc1-894f-4bb6-9b1d-567c72004575","Type":"ContainerStarted","Data":"1970eefbe58c6156df633f635d604fa121b88455a5781a5fda64c3537802322a"}
Oct 30 19:31:59.868104 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:31:59.868085    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-526rb" event={"ID":"141cf3eb-d99b-4876-b65a-a2847584ff5d","Type":"ContainerStarted","Data":"1374ca566c18177a9872baff538f680e5090425d1dc15b2ecc1e65d5929c2237"}
Oct 30 19:32:00.875774 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:32:00.875740    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vlqp7" event={"ID":"82384cc1-894f-4bb6-9b1d-567c72004575","Type":"ContainerStarted","Data":"9889021e8f0140aaad9f3cb04c181d0f69a8dc8a8ae36a821e329b78ab183639"}
Oct 30 19:32:01.883466 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:32:01.883422    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vlqp7" event={"ID":"82384cc1-894f-4bb6-9b1d-567c72004575","Type":"ContainerStarted","Data":"94d8437e7f295a9fbbf864657cec84508bee3e9255bbb2048276b97cec5f528d"}
Oct 30 19:32:01.919516 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:32:01.919435    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-vlqp7" podStartSLOduration=67.748480066 podStartE2EDuration="1m8.919415996s" totalImagesPullingTime="1.17093593s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:32:01.91601583 +0000 UTC m=+68.972799311" watchObservedRunningTime="2026-10-30 19:32:01.919415996 +0000 UTC m=+68.976199454"
Oct 30 19:32:03.896922 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:32:03.896877    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4f5sl" event={"ID":"7ff4ab79-1d7c-49ac-9d63-31772ad68b42","Type":"ContainerStarted","Data":"fd5d0d040acdaf8cc9f4285bd8d1b1ff29c90bdbdc6c09c64b4bdc7d286b80aa"}
Oct 30 19:32:03.897442 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:32:03.897038    2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-4f5sl"
Oct 30 19:32:03.918418 ip-10-0-141-42 kubenswrapper[2578]: I1030 19:32:03.918363    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-4f5sl" podStartSLOduration=67.694985841 podStartE2EDuration="1m10.918350632s" totalImagesPullingTime="3.223364791s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:32:03.917892702 +0000 UTC m=+70.974676160" watchObservedRunningTime="2026-10-30 19:32:03.918350632 +0000 UTC m=+70.975134088"
Oct 30 19:32:04.612250 ip-10-0-141-42 systemd[1]: Stopping Kubernetes Kubelet...
Oct 30 19:32:04.625892 ip-10-0-141-42 systemd[1]: kubelet.service: Deactivated successfully.
Oct 30 19:32:04.626060 ip-10-0-141-42 systemd[1]: Stopped Kubernetes Kubelet.
Oct 30 19:32:04.626110 ip-10-0-141-42 systemd[1]: kubelet.service: Consumed 4.606s CPU time.
Oct 30 19:32:04.641001 ip-10-0-141-42 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 19:32:04.702951 ip-10-0-141-42 kubenswrapper[6996]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 19:32:04.702951 ip-10-0-141-42 kubenswrapper[6996]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 19:32:04.702951 ip-10-0-141-42 kubenswrapper[6996]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 19:32:04.702951 ip-10-0-141-42 kubenswrapper[6996]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 19:32:04.704943 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.704865    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 19:32:04.704943 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.704943    6996 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 19:32:04.705013 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.704947    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 19:32:04.705013 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.704950    6996 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 19:32:04.705013 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.704953    6996 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 19:32:04.705013 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.704956    6996 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 19:32:04.705013 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.704959    6996 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 19:32:04.705013 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.704962    6996 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 19:32:04.705013 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.704964    6996 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 19:32:04.705013 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.704967    6996 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 19:32:04.705013 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.704970    6996 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 19:32:04.705013 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.704974    6996 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 19:32:04.705013 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.704976    6996 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 19:32:04.705013 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.704979    6996 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 19:32:04.705013 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.704981    6996 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 19:32:04.705013 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.704984    6996 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 19:32:04.705013 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.704987    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 19:32:04.705013 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.704989    6996 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 19:32:04.705013 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.704991    6996 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 19:32:04.705013 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.704994    6996 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 19:32:04.705013 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.704996    6996 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 19:32:04.705499 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.704999    6996 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 19:32:04.705499 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705001    6996 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 19:32:04.705499 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705004    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 19:32:04.705499 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705007    6996 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 19:32:04.705499 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705009    6996 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 19:32:04.705499 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705012    6996 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 19:32:04.705499 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705015    6996 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 19:32:04.705499 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705019    6996 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 19:32:04.705499 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705023    6996 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 19:32:04.705499 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705027    6996 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 19:32:04.705499 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705031    6996 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 19:32:04.705499 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705034    6996 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 19:32:04.705499 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705036    6996 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 19:32:04.705499 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705039    6996 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 19:32:04.705499 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705042    6996 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 19:32:04.705499 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705044    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 19:32:04.705499 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705047    6996 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 19:32:04.705499 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705049    6996 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 19:32:04.705499 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705052    6996 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 19:32:04.705970 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705055    6996 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 19:32:04.705970 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705057    6996 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 19:32:04.705970 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705060    6996 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 19:32:04.705970 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705063    6996 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 19:32:04.705970 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705066    6996 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 19:32:04.705970 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705068    6996 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 19:32:04.705970 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705071    6996 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 19:32:04.705970 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705073    6996 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 19:32:04.705970 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705076    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 19:32:04.705970 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705078    6996 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 19:32:04.705970 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705080    6996 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 19:32:04.705970 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705083    6996 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 19:32:04.705970 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705086    6996 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 19:32:04.705970 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705089    6996 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 19:32:04.705970 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705091    6996 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 19:32:04.705970 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705094    6996 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 19:32:04.705970 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705097    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 19:32:04.705970 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705099    6996 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 19:32:04.705970 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705103    6996 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 19:32:04.705970 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705105    6996 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 19:32:04.706464 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705108    6996 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 19:32:04.706464 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705110    6996 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 19:32:04.706464 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705113    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 19:32:04.706464 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705116    6996 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 19:32:04.706464 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705118    6996 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 19:32:04.706464 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705121    6996 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 19:32:04.706464 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705123    6996 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 19:32:04.706464 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705125    6996 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 19:32:04.706464 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705128    6996 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 19:32:04.706464 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705130    6996 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 19:32:04.706464 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705132    6996 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 19:32:04.706464 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705135    6996 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 19:32:04.706464 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705138    6996 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 19:32:04.706464 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705141    6996 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 19:32:04.706464 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705143    6996 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 19:32:04.706464 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705146    6996 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 19:32:04.706464 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705148    6996 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 19:32:04.706464 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705150    6996 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 19:32:04.706464 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705153    6996 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 19:32:04.706464 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705155    6996 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 19:32:04.706976 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705158    6996 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 19:32:04.706976 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705160    6996 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 19:32:04.706976 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705163    6996 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 19:32:04.706976 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705166    6996 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 19:32:04.706976 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705168    6996 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 19:32:04.706976 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705171    6996 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 19:32:04.706976 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705174    6996 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 19:32:04.706976 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705177    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 19:32:04.706976 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705179    6996 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 19:32:04.706976 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705182    6996 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 19:32:04.706976 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705184    6996 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 19:32:04.706976 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705187    6996 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 19:32:04.706976 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705189    6996 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 19:32:04.706976 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705192    6996 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 19:32:04.706976 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705194    6996 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 19:32:04.706976 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705197    6996 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 19:32:04.706976 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705201    6996 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 19:32:04.706976 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705205    6996 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 19:32:04.706976 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705208    6996 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 19:32:04.707497 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705211    6996 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 19:32:04.707497 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705214    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 19:32:04.707497 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705216    6996 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 19:32:04.707497 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705218    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 19:32:04.707497 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705221    6996 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 19:32:04.707497 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705224    6996 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 19:32:04.707497 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.705228    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 19:32:04.707497 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706016    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 19:32:04.707497 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706023    6996 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 19:32:04.707497 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706027    6996 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 19:32:04.707497 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706031    6996 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 19:32:04.707497 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706033    6996 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 19:32:04.707497 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706036    6996 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 19:32:04.707497 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706039    6996 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 19:32:04.707497 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706042    6996 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 19:32:04.707497 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706046    6996 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 19:32:04.707497 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706049    6996 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 19:32:04.707497 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706051    6996 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 19:32:04.707497 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706054    6996 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 19:32:04.707497 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706057    6996 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 19:32:04.708020 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706060    6996 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 19:32:04.708020 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706062    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 19:32:04.708020 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706065    6996 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 19:32:04.708020 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706067    6996 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 19:32:04.708020 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706070    6996 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 19:32:04.708020 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706072    6996 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 19:32:04.708020 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706075    6996 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 19:32:04.708020 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706077    6996 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 19:32:04.708020 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706080    6996 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 19:32:04.708020 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706083    6996 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 19:32:04.708020 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706085    6996 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 19:32:04.708020 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706088    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 19:32:04.708020 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706090    6996 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 19:32:04.708020 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706093    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 19:32:04.708020 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706095    6996 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 19:32:04.708020 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706097    6996 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 19:32:04.708020 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706101    6996 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 19:32:04.708020 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706105    6996 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 19:32:04.708020 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706109    6996 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 19:32:04.708517 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706112    6996 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 19:32:04.708517 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706115    6996 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 19:32:04.708517 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706118    6996 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 19:32:04.708517 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706120    6996 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 19:32:04.708517 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706123    6996 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 19:32:04.708517 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706125    6996 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 19:32:04.708517 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706128    6996 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 19:32:04.708517 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706135    6996 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 19:32:04.708517 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706138    6996 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 19:32:04.708517 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706141    6996 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 19:32:04.708517 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706143    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 19:32:04.708517 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706146    6996 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 19:32:04.708517 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706148    6996 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 19:32:04.708517 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706151    6996 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 19:32:04.708517 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706154    6996 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 19:32:04.708517 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706156    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 19:32:04.708517 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706159    6996 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 19:32:04.708517 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706162    6996 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 19:32:04.708517 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706164    6996 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 19:32:04.708517 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706167    6996 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 19:32:04.709010 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706169    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 19:32:04.709010 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706171    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 19:32:04.709010 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706174    6996 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 19:32:04.709010 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706178    6996 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 19:32:04.709010 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706182    6996 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 19:32:04.709010 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706185    6996 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 19:32:04.709010 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706188    6996 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 19:32:04.709010 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706191    6996 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 19:32:04.709010 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706193    6996 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 19:32:04.709010 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706196    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 19:32:04.709010 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706199    6996 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 19:32:04.709010 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706201    6996 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 19:32:04.709010 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706204    6996 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 19:32:04.709010 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706207    6996 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 19:32:04.709010 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706209    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 19:32:04.709010 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706212    6996 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 19:32:04.709010 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706214    6996 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 19:32:04.709010 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706217    6996 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 19:32:04.709010 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706219    6996 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 19:32:04.709468 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706223    6996 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 19:32:04.709468 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706226    6996 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 19:32:04.709468 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706229    6996 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 19:32:04.709468 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706231    6996 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 19:32:04.709468 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706234    6996 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 19:32:04.709468 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706237    6996 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 19:32:04.709468 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706239    6996 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 19:32:04.709468 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706242    6996 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 19:32:04.709468 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706244    6996 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 19:32:04.709468 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706247    6996 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 19:32:04.709468 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706249    6996 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 19:32:04.709468 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706252    6996 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 19:32:04.709468 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706254    6996 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 19:32:04.709468 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706257    6996 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 19:32:04.709468 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706259    6996 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 19:32:04.709468 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706262    6996 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 19:32:04.709468 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706265    6996 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 19:32:04.709468 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706267    6996 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 19:32:04.709468 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706269    6996 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 19:32:04.709468 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706272    6996 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 19:32:04.709979 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706274    6996 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 19:32:04.709979 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706277    6996 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 19:32:04.709979 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706280    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 19:32:04.709979 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706282    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 19:32:04.709979 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706285    6996 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 19:32:04.709979 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706288    6996 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 19:32:04.709979 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706291    6996 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 19:32:04.709979 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706294    6996 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 19:32:04.709979 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706296    6996 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 19:32:04.709979 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706298    6996 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 19:32:04.709979 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706301    6996 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 19:32:04.709979 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706303    6996 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 19:32:04.709979 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706307    6996 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 19:32:04.709979 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706310    6996 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 19:32:04.709979 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706312    6996 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 19:32:04.709979 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706810    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 19:32:04.709979 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706816    6996 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 19:32:04.709979 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706819    6996 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 19:32:04.709979 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706822    6996 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 19:32:04.709979 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706824    6996 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 19:32:04.710464 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706827    6996 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 19:32:04.710464 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706829    6996 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 19:32:04.710464 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706832    6996 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 19:32:04.710464 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706835    6996 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 19:32:04.710464 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706837    6996 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 19:32:04.710464 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706840    6996 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 19:32:04.710464 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706842    6996 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 19:32:04.710464 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706846    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 19:32:04.710464 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706848    6996 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 19:32:04.710464 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706851    6996 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 19:32:04.710464 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706853    6996 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 19:32:04.710464 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706856    6996 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 19:32:04.710464 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706858    6996 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 19:32:04.710464 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706861    6996 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 19:32:04.710464 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706864    6996 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 19:32:04.710464 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706867    6996 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 19:32:04.710464 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706870    6996 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 19:32:04.710464 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706872    6996 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 19:32:04.710464 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706875    6996 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 19:32:04.710464 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706878    6996 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 19:32:04.711316 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706881    6996 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 19:32:04.711316 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706885    6996 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 19:32:04.711316 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706888    6996 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 19:32:04.711316 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706899    6996 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 19:32:04.711316 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706904    6996 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 19:32:04.711316 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706908    6996 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 19:32:04.711316 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706910    6996 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 19:32:04.711316 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706913    6996 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 19:32:04.711316 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706916    6996 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 19:32:04.711316 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706919    6996 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 19:32:04.711316 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706921    6996 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 19:32:04.711316 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706924    6996 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 19:32:04.711316 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706926    6996 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 19:32:04.711316 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706928    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 19:32:04.711316 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706931    6996 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 19:32:04.711316 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706933    6996 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 19:32:04.711316 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706936    6996 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 19:32:04.711316 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706938    6996 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 19:32:04.711316 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706941    6996 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 19:32:04.712157 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706944    6996 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 19:32:04.712157 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706946    6996 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 19:32:04.712157 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706948    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 19:32:04.712157 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706951    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 19:32:04.712157 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706953    6996 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 19:32:04.712157 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706956    6996 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 19:32:04.712157 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706958    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 19:32:04.712157 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706961    6996 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 19:32:04.712157 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706964    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 19:32:04.712157 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706966    6996 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 19:32:04.712157 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706969    6996 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 19:32:04.712157 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706971    6996 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 19:32:04.712157 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706974    6996 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 19:32:04.712157 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706977    6996 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 19:32:04.712157 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706980    6996 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 19:32:04.712157 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706983    6996 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 19:32:04.712157 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706986    6996 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 19:32:04.712157 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706989    6996 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 19:32:04.712157 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706992    6996 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 19:32:04.712157 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706994    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 19:32:04.713013 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706996    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 19:32:04.713013 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.706999    6996 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 19:32:04.713013 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707001    6996 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 19:32:04.713013 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707004    6996 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 19:32:04.713013 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707006    6996 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 19:32:04.713013 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707009    6996 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 19:32:04.713013 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707011    6996 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 19:32:04.713013 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707014    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 19:32:04.713013 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707016    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 19:32:04.713013 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707019    6996 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 19:32:04.713013 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707021    6996 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 19:32:04.713013 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707025    6996 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 19:32:04.713013 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707028    6996 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 19:32:04.713013 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707030    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 19:32:04.713013 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707032    6996 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 19:32:04.713013 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707035    6996 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 19:32:04.713013 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707037    6996 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 19:32:04.713013 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707040    6996 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 19:32:04.713013 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707042    6996 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 19:32:04.713013 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707045    6996 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 19:32:04.713965 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707047    6996 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 19:32:04.713965 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707052    6996 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 19:32:04.713965 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707055    6996 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 19:32:04.713965 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707058    6996 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 19:32:04.713965 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707061    6996 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 19:32:04.713965 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707064    6996 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 19:32:04.713965 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707066    6996 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 19:32:04.713965 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707070    6996 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 19:32:04.713965 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707073    6996 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 19:32:04.713965 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707075    6996 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 19:32:04.713965 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707077    6996 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 19:32:04.713965 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707080    6996 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 19:32:04.713965 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707083    6996 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 19:32:04.713965 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707085    6996 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 19:32:04.713965 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707088    6996 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 19:32:04.713965 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707090    6996 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 19:32:04.713965 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707093    6996 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 19:32:04.713965 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707095    6996 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 19:32:04.713965 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707098    6996 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 19:32:04.714878 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707100    6996 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 19:32:04.714878 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707102    6996 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 19:32:04.714878 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707105    6996 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 19:32:04.714878 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707195    6996 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 19:32:04.714878 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707203    6996 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 19:32:04.714878 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707210    6996 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 19:32:04.714878 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707214    6996 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 19:32:04.714878 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707218    6996 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 19:32:04.714878 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707222    6996 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 19:32:04.714878 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707226    6996 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 19:32:04.714878 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707230    6996 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 19:32:04.714878 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707233    6996 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 19:32:04.714878 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707236    6996 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 19:32:04.714878 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707239    6996 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 19:32:04.714878 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707243    6996 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 19:32:04.714878 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707246    6996 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 19:32:04.714878 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707249    6996 flags.go:64] FLAG: --cgroup-root=""
Oct 30 19:32:04.714878 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707251    6996 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 19:32:04.714878 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707255    6996 flags.go:64] FLAG: --client-ca-file=""
Oct 30 19:32:04.714878 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707258    6996 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 19:32:04.714878 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707261    6996 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 19:32:04.714878 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707265    6996 flags.go:64] FLAG: --cluster-domain=""
Oct 30 19:32:04.714878 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707285    6996 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 19:32:04.715999 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707288    6996 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 19:32:04.715999 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707292    6996 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 19:32:04.715999 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707296    6996 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 19:32:04.715999 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707300    6996 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 19:32:04.715999 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707303    6996 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 19:32:04.715999 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707306    6996 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 19:32:04.715999 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707309    6996 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 19:32:04.715999 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707312    6996 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 19:32:04.715999 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707315    6996 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 19:32:04.715999 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707318    6996 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 19:32:04.715999 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707321    6996 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 19:32:04.715999 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707324    6996 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 19:32:04.715999 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707328    6996 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 19:32:04.715999 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707332    6996 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 19:32:04.715999 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707335    6996 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 19:32:04.715999 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707338    6996 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 19:32:04.715999 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707341    6996 flags.go:64] FLAG: --enable-server="true"
Oct 30 19:32:04.715999 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707344    6996 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 19:32:04.715999 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707349    6996 flags.go:64] FLAG: --event-burst="100"
Oct 30 19:32:04.715999 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707352    6996 flags.go:64] FLAG: --event-qps="50"
Oct 30 19:32:04.715999 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707355    6996 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 19:32:04.715999 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707358    6996 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 19:32:04.715999 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707361    6996 flags.go:64] FLAG: --eviction-hard=""
Oct 30 19:32:04.715999 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707365    6996 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 19:32:04.716629 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707368    6996 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 19:32:04.716629 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707371    6996 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 19:32:04.716629 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707374    6996 flags.go:64] FLAG: --eviction-soft=""
Oct 30 19:32:04.716629 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707377    6996 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 19:32:04.716629 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707380    6996 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 19:32:04.716629 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707383    6996 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 19:32:04.716629 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707387    6996 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 19:32:04.716629 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707389    6996 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 19:32:04.716629 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707392    6996 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 19:32:04.716629 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707395    6996 flags.go:64] FLAG: --feature-gates=""
Oct 30 19:32:04.716629 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707399    6996 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 19:32:04.716629 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707402    6996 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 19:32:04.716629 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707405    6996 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 19:32:04.716629 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707408    6996 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 19:32:04.716629 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707411    6996 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 19:32:04.716629 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707414    6996 flags.go:64] FLAG: --help="false"
Oct 30 19:32:04.716629 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707417    6996 flags.go:64] FLAG: --hostname-override="ip-10-0-141-42.us-east-2.compute.internal"
Oct 30 19:32:04.716629 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707420    6996 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 19:32:04.716629 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707423    6996 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 19:32:04.716629 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707426    6996 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 19:32:04.716629 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707429    6996 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 19:32:04.716629 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707434    6996 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 19:32:04.716629 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707436    6996 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 19:32:04.717179 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707441    6996 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 19:32:04.717179 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707444    6996 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 19:32:04.717179 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707446    6996 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 19:32:04.717179 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707449    6996 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 19:32:04.717179 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707452    6996 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 19:32:04.717179 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707455    6996 flags.go:64] FLAG: --kube-reserved=""
Oct 30 19:32:04.717179 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707458    6996 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 19:32:04.717179 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707461    6996 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 19:32:04.717179 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707464    6996 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 19:32:04.717179 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707467    6996 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 19:32:04.717179 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707470    6996 flags.go:64] FLAG: --lock-file=""
Oct 30 19:32:04.717179 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707473    6996 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 19:32:04.717179 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707491    6996 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 19:32:04.717179 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707495    6996 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 19:32:04.717179 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707500    6996 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 19:32:04.717179 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707503    6996 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 19:32:04.717179 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707507    6996 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 19:32:04.717179 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707511    6996 flags.go:64] FLAG: --logging-format="text"
Oct 30 19:32:04.717179 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707514    6996 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 19:32:04.717179 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707517    6996 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 19:32:04.717179 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707520    6996 flags.go:64] FLAG: --manifest-url=""
Oct 30 19:32:04.717179 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707523    6996 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 19:32:04.717179 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707527    6996 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 19:32:04.717179 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707530    6996 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 19:32:04.717179 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707534    6996 flags.go:64] FLAG: --max-pods="110"
Oct 30 19:32:04.717886 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707537    6996 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 19:32:04.717886 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707540    6996 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 19:32:04.717886 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707543    6996 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 19:32:04.717886 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707546    6996 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 19:32:04.717886 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707549    6996 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 19:32:04.717886 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707553    6996 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 19:32:04.717886 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707556    6996 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 19:32:04.717886 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707564    6996 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 19:32:04.717886 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707567    6996 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 19:32:04.717886 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707570    6996 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 19:32:04.717886 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707574    6996 flags.go:64] FLAG: --pod-cidr=""
Oct 30 19:32:04.717886 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707576    6996 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 19:32:04.717886 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707579    6996 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 19:32:04.717886 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707582    6996 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 19:32:04.717886 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707603    6996 flags.go:64] FLAG: --port="10250"
Oct 30 19:32:04.717886 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707607    6996 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 19:32:04.717886 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707610    6996 flags.go:64] FLAG: --provider-id="aws:///us-east-2a/i-0c2e0ee918aa12755"
Oct 30 19:32:04.717886 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707613    6996 flags.go:64] FLAG: --qos-reserved=""
Oct 30 19:32:04.717886 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707616    6996 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 19:32:04.717886 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707619    6996 flags.go:64] FLAG: --register-node="true"
Oct 30 19:32:04.717886 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707622    6996 flags.go:64] FLAG: --register-with-taints=""
Oct 30 19:32:04.717886 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707626    6996 flags.go:64] FLAG: --registry-burst="10"
Oct 30 19:32:04.717886 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707628    6996 flags.go:64] FLAG: --registry-qps="5"
Oct 30 19:32:04.717886 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707631    6996 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 19:32:04.717886 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707640    6996 flags.go:64] FLAG: --reserved-memory=""
Oct 30 19:32:04.717886 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707644    6996 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 19:32:04.718522 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707647    6996 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 19:32:04.718522 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707650    6996 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 19:32:04.718522 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707653    6996 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 19:32:04.718522 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707656    6996 flags.go:64] FLAG: --runonce="false"
Oct 30 19:32:04.718522 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707659    6996 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 19:32:04.718522 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707662    6996 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 19:32:04.718522 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707665    6996 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 19:32:04.718522 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707668    6996 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 19:32:04.718522 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707671    6996 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 19:32:04.718522 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707674    6996 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 19:32:04.718522 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707677    6996 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 19:32:04.718522 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707680    6996 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 19:32:04.718522 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707683    6996 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 19:32:04.718522 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707688    6996 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 19:32:04.718522 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707691    6996 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 19:32:04.718522 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707694    6996 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 19:32:04.718522 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707697    6996 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 19:32:04.718522 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707701    6996 flags.go:64] FLAG: --system-cgroups=""
Oct 30 19:32:04.718522 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707704    6996 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 19:32:04.718522 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707709    6996 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 19:32:04.718522 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707712    6996 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 19:32:04.718522 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707715    6996 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 19:32:04.718522 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707719    6996 flags.go:64] FLAG: --tls-min-version=""
Oct 30 19:32:04.718522 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707722    6996 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 19:32:04.718522 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707725    6996 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 19:32:04.719181 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707728    6996 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 19:32:04.719181 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707731    6996 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 19:32:04.719181 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707734    6996 flags.go:64] FLAG: --v="2"
Oct 30 19:32:04.719181 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707738    6996 flags.go:64] FLAG: --version="false"
Oct 30 19:32:04.719181 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707742    6996 flags.go:64] FLAG: --vmodule=""
Oct 30 19:32:04.719181 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707746    6996 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 19:32:04.719181 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.707750    6996 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 19:32:04.719181 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707883    6996 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 19:32:04.719181 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707887    6996 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 19:32:04.719181 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707890    6996 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 19:32:04.719181 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707893    6996 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 19:32:04.719181 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707895    6996 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 19:32:04.719181 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707898    6996 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 19:32:04.719181 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707901    6996 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 19:32:04.719181 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707903    6996 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 19:32:04.719181 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707906    6996 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 19:32:04.719181 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707908    6996 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 19:32:04.719181 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707911    6996 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 19:32:04.719181 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707914    6996 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 19:32:04.719181 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707917    6996 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 19:32:04.719181 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707921    6996 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 19:32:04.719896 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707924    6996 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 19:32:04.719896 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707926    6996 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 19:32:04.719896 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707929    6996 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 19:32:04.719896 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707932    6996 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 19:32:04.719896 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707934    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 19:32:04.719896 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707937    6996 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 19:32:04.719896 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707940    6996 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 19:32:04.719896 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707943    6996 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 19:32:04.719896 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707945    6996 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 19:32:04.719896 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707948    6996 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 19:32:04.719896 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707951    6996 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 19:32:04.719896 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707953    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 19:32:04.719896 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707956    6996 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 19:32:04.719896 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707960    6996 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 19:32:04.719896 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707964    6996 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 19:32:04.719896 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707967    6996 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 19:32:04.719896 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707970    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 19:32:04.719896 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707973    6996 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 19:32:04.719896 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707976    6996 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 19:32:04.720473 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707979    6996 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 19:32:04.720473 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707981    6996 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 19:32:04.720473 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707985    6996 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 19:32:04.720473 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707988    6996 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 19:32:04.720473 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707992    6996 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 19:32:04.720473 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707994    6996 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 19:32:04.720473 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.707997    6996 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 19:32:04.720473 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708000    6996 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 19:32:04.720473 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708002    6996 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 19:32:04.720473 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708005    6996 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 19:32:04.720473 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708007    6996 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 19:32:04.720473 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708010    6996 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 19:32:04.720473 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708015    6996 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 19:32:04.720473 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708017    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 19:32:04.720473 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708020    6996 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 19:32:04.720473 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708023    6996 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 19:32:04.720473 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708026    6996 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 19:32:04.720473 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708028    6996 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 19:32:04.720473 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708031    6996 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 19:32:04.720962 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708034    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 19:32:04.720962 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708036    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 19:32:04.720962 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708039    6996 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 19:32:04.720962 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708042    6996 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 19:32:04.720962 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708044    6996 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 19:32:04.720962 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708047    6996 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 19:32:04.720962 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708050    6996 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 19:32:04.720962 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708052    6996 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 19:32:04.720962 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708055    6996 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 19:32:04.720962 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708057    6996 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 19:32:04.720962 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708060    6996 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 19:32:04.720962 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708063    6996 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 19:32:04.720962 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708065    6996 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 19:32:04.720962 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708068    6996 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 19:32:04.720962 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708071    6996 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 19:32:04.720962 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708073    6996 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 19:32:04.720962 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708076    6996 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 19:32:04.720962 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708078    6996 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 19:32:04.720962 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708081    6996 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 19:32:04.720962 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708085    6996 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 19:32:04.721493 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708088    6996 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 19:32:04.721493 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708090    6996 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 19:32:04.721493 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708093    6996 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 19:32:04.721493 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708095    6996 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 19:32:04.721493 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708098    6996 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 19:32:04.721493 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708102    6996 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 19:32:04.721493 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708104    6996 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 19:32:04.721493 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708107    6996 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 19:32:04.721493 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708110    6996 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 19:32:04.721493 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708112    6996 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 19:32:04.721493 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708115    6996 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 19:32:04.721493 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708117    6996 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 19:32:04.721493 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708120    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 19:32:04.721493 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708135    6996 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 19:32:04.721493 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708138    6996 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 19:32:04.721493 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708140    6996 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 19:32:04.721493 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708143    6996 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 19:32:04.721493 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708145    6996 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 19:32:04.721493 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708148    6996 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 19:32:04.721493 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708150    6996 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 19:32:04.722097 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708153    6996 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 19:32:04.722097 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708155    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 19:32:04.722097 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708158    6996 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 19:32:04.722097 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708160    6996 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 19:32:04.722097 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708163    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 19:32:04.722097 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708166    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 19:32:04.722097 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708170    6996 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 19:32:04.722097 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708172    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 19:32:04.722097 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708175    6996 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 19:32:04.722097 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708177    6996 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 19:32:04.722097 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708180    6996 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 19:32:04.722097 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708184    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 19:32:04.722097 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708186    6996 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 19:32:04.722097 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708189    6996 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 19:32:04.722097 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.708214    6996 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 19:32:04.722097 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708248    6996 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 19:32:04.722551 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708252    6996 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 19:32:04.722551 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708255    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 19:32:04.722551 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708258    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 19:32:04.722551 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708261    6996 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 19:32:04.722551 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708264    6996 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 19:32:04.722551 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708267    6996 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 19:32:04.722551 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708270    6996 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 19:32:04.722551 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708272    6996 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 19:32:04.722551 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708275    6996 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 19:32:04.722551 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708278    6996 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 19:32:04.722551 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708280    6996 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 19:32:04.722551 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708283    6996 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 19:32:04.722551 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708285    6996 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 19:32:04.722551 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708288    6996 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 19:32:04.722551 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708291    6996 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 19:32:04.722551 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708293    6996 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 19:32:04.722551 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708296    6996 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 19:32:04.722551 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708298    6996 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 19:32:04.722551 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708301    6996 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 19:32:04.722551 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708304    6996 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 19:32:04.723328 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708306    6996 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 19:32:04.723328 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708309    6996 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 19:32:04.723328 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708311    6996 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 19:32:04.723328 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708314    6996 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 19:32:04.723328 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708316    6996 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 19:32:04.723328 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708319    6996 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 19:32:04.723328 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708321    6996 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 19:32:04.723328 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708325    6996 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 19:32:04.723328 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708328    6996 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 19:32:04.723328 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708331    6996 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 19:32:04.723328 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708334    6996 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 19:32:04.723328 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708336    6996 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 19:32:04.723328 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708339    6996 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 19:32:04.723328 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708341    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 19:32:04.723328 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708346    6996 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 19:32:04.723328 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708349    6996 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 19:32:04.723328 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708352    6996 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 19:32:04.723328 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708354    6996 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 19:32:04.723328 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708357    6996 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 19:32:04.723328 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708360    6996 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 19:32:04.724169 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708362    6996 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 19:32:04.724169 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708365    6996 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 19:32:04.724169 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708367    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 19:32:04.724169 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708370    6996 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 19:32:04.724169 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708372    6996 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 19:32:04.724169 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708375    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 19:32:04.724169 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708378    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 19:32:04.724169 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708380    6996 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 19:32:04.724169 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708383    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 19:32:04.724169 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708385    6996 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 19:32:04.724169 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708388    6996 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 19:32:04.724169 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708391    6996 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 19:32:04.724169 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708393    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 19:32:04.724169 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708396    6996 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 19:32:04.724169 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708398    6996 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 19:32:04.724169 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708401    6996 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 19:32:04.724169 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708404    6996 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 19:32:04.724169 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708406    6996 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 19:32:04.724169 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708409    6996 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 19:32:04.725028 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708412    6996 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 19:32:04.725028 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708415    6996 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 19:32:04.725028 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708417    6996 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 19:32:04.725028 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708420    6996 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 19:32:04.725028 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708422    6996 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 19:32:04.725028 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708424    6996 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 19:32:04.725028 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708427    6996 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 19:32:04.725028 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708431    6996 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 19:32:04.725028 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708434    6996 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 19:32:04.725028 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708436    6996 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 19:32:04.725028 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708439    6996 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 19:32:04.725028 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708441    6996 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 19:32:04.725028 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708444    6996 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 19:32:04.725028 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708447    6996 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 19:32:04.725028 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708449    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 19:32:04.725028 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708452    6996 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 19:32:04.725028 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708455    6996 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 19:32:04.725028 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708458    6996 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 19:32:04.725028 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708460    6996 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 19:32:04.725028 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708463    6996 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 19:32:04.725847 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708466    6996 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 19:32:04.725847 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708468    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 19:32:04.725847 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708471    6996 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 19:32:04.725847 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708487    6996 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 19:32:04.725847 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708494    6996 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 19:32:04.725847 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708499    6996 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 19:32:04.725847 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708503    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 19:32:04.725847 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708507    6996 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 19:32:04.725847 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708510    6996 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 19:32:04.725847 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708513    6996 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 19:32:04.725847 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708516    6996 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 19:32:04.725847 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708520    6996 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 19:32:04.725847 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708526    6996 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 19:32:04.725847 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708529    6996 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 19:32:04.725847 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708532    6996 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 19:32:04.725847 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708534    6996 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 19:32:04.725847 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708537    6996 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 19:32:04.725847 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708539    6996 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 19:32:04.725847 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708542    6996 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 19:32:04.726647 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708546    6996 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 19:32:04.726647 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708548    6996 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 19:32:04.726647 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708551    6996 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 19:32:04.726647 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708555    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 19:32:04.726647 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708558    6996 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 19:32:04.726647 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708560    6996 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 19:32:04.726647 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.708563    6996 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 19:32:04.726647 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.712848    6996 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 19:32:04.726647 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.712870    6996 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 19:32:04.726647 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.712895    6996 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 19:32:04.726647 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.712903    6996 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 19:32:04.726647 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.712985    6996 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 19:32:04.726647 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.712994    6996 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 19:32:04.726647 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.712999    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 19:32:04.726647 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713004    6996 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 19:32:04.726647 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713008    6996 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 19:32:04.726647 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713012    6996 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 19:32:04.726647 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713016    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 19:32:04.726647 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713021    6996 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 19:32:04.727417 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713025    6996 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 19:32:04.727417 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713029    6996 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 19:32:04.727417 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713033    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 19:32:04.727417 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713037    6996 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 19:32:04.727417 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713041    6996 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 19:32:04.727417 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713046    6996 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 19:32:04.727417 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713050    6996 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 19:32:04.727417 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713055    6996 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 19:32:04.727417 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713059    6996 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 19:32:04.727417 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713063    6996 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 19:32:04.727417 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713068    6996 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 19:32:04.727417 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713072    6996 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 19:32:04.727417 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713077    6996 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 19:32:04.727417 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713081    6996 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 19:32:04.727417 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713087    6996 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 19:32:04.727417 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713094    6996 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 19:32:04.727417 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713098    6996 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 19:32:04.727417 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713103    6996 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 19:32:04.727417 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713108    6996 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 19:32:04.728036 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713112    6996 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 19:32:04.728036 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713117    6996 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 19:32:04.728036 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713123    6996 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 19:32:04.728036 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713127    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 19:32:04.728036 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713132    6996 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 19:32:04.728036 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713136    6996 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 19:32:04.728036 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713140    6996 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 19:32:04.728036 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713145    6996 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 19:32:04.728036 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713149    6996 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 19:32:04.728036 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713153    6996 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 19:32:04.728036 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713157    6996 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 19:32:04.728036 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713161    6996 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 19:32:04.728036 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713165    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 19:32:04.728036 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713170    6996 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 19:32:04.728036 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713174    6996 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 19:32:04.728036 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713178    6996 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 19:32:04.728036 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713183    6996 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 19:32:04.728036 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713187    6996 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 19:32:04.728036 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713191    6996 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 19:32:04.728036 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713196    6996 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 19:32:04.728673 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713200    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 19:32:04.728673 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713204    6996 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 19:32:04.728673 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713208    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 19:32:04.728673 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713214    6996 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 19:32:04.728673 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713280    6996 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 19:32:04.728673 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713286    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 19:32:04.728673 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713291    6996 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 19:32:04.728673 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713296    6996 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 19:32:04.728673 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713300    6996 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 19:32:04.728673 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713304    6996 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 19:32:04.728673 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713309    6996 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 19:32:04.728673 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713312    6996 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 19:32:04.728673 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713317    6996 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 19:32:04.728673 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713321    6996 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 19:32:04.728673 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713325    6996 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 19:32:04.728673 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713331    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 19:32:04.728673 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713336    6996 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 19:32:04.728673 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713339    6996 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 19:32:04.728673 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713344    6996 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 19:32:04.729133 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713348    6996 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 19:32:04.729133 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713352    6996 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 19:32:04.729133 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713356    6996 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 19:32:04.729133 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713361    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 19:32:04.729133 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713365    6996 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 19:32:04.729133 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713369    6996 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 19:32:04.729133 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713373    6996 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 19:32:04.729133 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713378    6996 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 19:32:04.729133 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713382    6996 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 19:32:04.729133 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713386    6996 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 19:32:04.729133 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713390    6996 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 19:32:04.729133 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713394    6996 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 19:32:04.729133 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713398    6996 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 19:32:04.729133 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713403    6996 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 19:32:04.729133 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713407    6996 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 19:32:04.729133 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713411    6996 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 19:32:04.729133 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713416    6996 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 19:32:04.729133 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713420    6996 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 19:32:04.729133 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713424    6996 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 19:32:04.729133 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713428    6996 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 19:32:04.729620 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713432    6996 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 19:32:04.729620 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713436    6996 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 19:32:04.729620 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713441    6996 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 19:32:04.729620 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713445    6996 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 19:32:04.729620 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713450    6996 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 19:32:04.729620 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713454    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 19:32:04.729620 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713458    6996 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 19:32:04.729620 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713462    6996 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 19:32:04.729620 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713466    6996 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 19:32:04.729620 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713472    6996 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 19:32:04.729620 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713498    6996 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 19:32:04.729620 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713502    6996 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 19:32:04.729620 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713506    6996 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 19:32:04.729620 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713510    6996 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 19:32:04.729620 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713514    6996 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 19:32:04.729620 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713517    6996 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 19:32:04.729620 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713521    6996 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 19:32:04.729620 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713525    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 19:32:04.729620 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713528    6996 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 19:32:04.729620 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713532    6996 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 19:32:04.730100 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.713577    6996 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 19:32:04.730100 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713794    6996 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 19:32:04.730100 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713803    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 19:32:04.730100 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713809    6996 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 19:32:04.730100 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713813    6996 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 19:32:04.730100 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713818    6996 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 19:32:04.730100 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713823    6996 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 19:32:04.730100 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713827    6996 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 19:32:04.730100 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713831    6996 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 19:32:04.730100 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713835    6996 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 19:32:04.730100 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713841    6996 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 19:32:04.730100 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713849    6996 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 19:32:04.730100 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713854    6996 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 19:32:04.730100 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713858    6996 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 19:32:04.730100 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713862    6996 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 19:32:04.730467 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713867    6996 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 19:32:04.730467 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713871    6996 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 19:32:04.730467 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713875    6996 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 19:32:04.730467 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713879    6996 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 19:32:04.730467 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713883    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 19:32:04.730467 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713888    6996 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 19:32:04.730467 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713892    6996 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 19:32:04.730467 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713897    6996 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 19:32:04.730467 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713901    6996 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 19:32:04.730467 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713905    6996 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 19:32:04.730467 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713909    6996 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 19:32:04.730467 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713913    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 19:32:04.730467 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713918    6996 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 19:32:04.730467 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713921    6996 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 19:32:04.730467 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713926    6996 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 19:32:04.730467 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713930    6996 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 19:32:04.730467 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713934    6996 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 19:32:04.730467 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713938    6996 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 19:32:04.730467 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713942    6996 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 19:32:04.730467 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713946    6996 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 19:32:04.730986 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713950    6996 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 19:32:04.730986 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713954    6996 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 19:32:04.730986 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713957    6996 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 19:32:04.730986 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713961    6996 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 19:32:04.730986 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713965    6996 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 19:32:04.730986 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713969    6996 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 19:32:04.730986 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713973    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 19:32:04.730986 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713976    6996 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 19:32:04.730986 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713980    6996 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 19:32:04.730986 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713984    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 19:32:04.730986 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713988    6996 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 19:32:04.730986 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713992    6996 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 19:32:04.730986 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.713996    6996 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 19:32:04.730986 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714004    6996 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 19:32:04.730986 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714008    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 19:32:04.730986 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714012    6996 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 19:32:04.730986 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714017    6996 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 19:32:04.730986 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714021    6996 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 19:32:04.730986 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714025    6996 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 19:32:04.730986 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714029    6996 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 19:32:04.731568 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714033    6996 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 19:32:04.731568 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714037    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 19:32:04.731568 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714042    6996 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 19:32:04.731568 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714046    6996 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 19:32:04.731568 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714050    6996 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 19:32:04.731568 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714057    6996 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 19:32:04.731568 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714062    6996 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 19:32:04.731568 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714067    6996 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 19:32:04.731568 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714072    6996 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 19:32:04.731568 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714076    6996 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 19:32:04.731568 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714080    6996 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 19:32:04.731568 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714084    6996 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 19:32:04.731568 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714088    6996 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 19:32:04.731568 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714093    6996 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 19:32:04.731568 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714098    6996 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 19:32:04.731568 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714104    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 19:32:04.731568 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714108    6996 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 19:32:04.731568 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714112    6996 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 19:32:04.731568 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714116    6996 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 19:32:04.732030 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714120    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 19:32:04.732030 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714124    6996 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 19:32:04.732030 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714128    6996 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 19:32:04.732030 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714134    6996 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 19:32:04.732030 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714138    6996 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 19:32:04.732030 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714142    6996 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 19:32:04.732030 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714151    6996 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 19:32:04.732030 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714155    6996 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 19:32:04.732030 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714159    6996 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 19:32:04.732030 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714163    6996 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 19:32:04.732030 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714168    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 19:32:04.732030 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714172    6996 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 19:32:04.732030 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714176    6996 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 19:32:04.732030 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714180    6996 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 19:32:04.732030 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714186    6996 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 19:32:04.732030 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714190    6996 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 19:32:04.732030 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714194    6996 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 19:32:04.732030 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714199    6996 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 19:32:04.732030 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714203    6996 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 19:32:04.732030 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714207    6996 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 19:32:04.732553 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714211    6996 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 19:32:04.732553 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714215    6996 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 19:32:04.732553 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714219    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 19:32:04.732553 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714223    6996 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 19:32:04.732553 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714227    6996 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 19:32:04.732553 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714231    6996 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 19:32:04.732553 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714236    6996 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 19:32:04.732553 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714240    6996 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 19:32:04.732553 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714246    6996 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 19:32:04.732553 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714250    6996 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 19:32:04.732553 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714254    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 19:32:04.732553 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714258    6996 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 19:32:04.732553 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714262    6996 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 19:32:04.732553 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.714304    6996 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 19:32:04.732553 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714356    6996 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 19:32:04.732553 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714363    6996 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 19:32:04.732938 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714367    6996 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 19:32:04.732938 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714372    6996 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 19:32:04.732938 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714376    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 19:32:04.732938 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714383    6996 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 19:32:04.732938 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714388    6996 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 19:32:04.732938 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714393    6996 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 19:32:04.732938 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714397    6996 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 19:32:04.732938 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714401    6996 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 19:32:04.732938 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714405    6996 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 19:32:04.732938 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714410    6996 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 19:32:04.732938 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714414    6996 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 19:32:04.732938 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714462    6996 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 19:32:04.732938 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714564    6996 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 19:32:04.732938 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714574    6996 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 19:32:04.732938 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714579    6996 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 19:32:04.732938 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714584    6996 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 19:32:04.732938 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714588    6996 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 19:32:04.732938 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714593    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 19:32:04.732938 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714598    6996 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 19:32:04.732938 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714602    6996 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 19:32:04.733422 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714607    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 19:32:04.733422 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714611    6996 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 19:32:04.733422 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714615    6996 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 19:32:04.733422 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714620    6996 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 19:32:04.733422 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714624    6996 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 19:32:04.733422 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714634    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 19:32:04.733422 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714638    6996 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 19:32:04.733422 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714643    6996 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 19:32:04.733422 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714647    6996 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 19:32:04.733422 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714651    6996 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 19:32:04.733422 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714656    6996 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 19:32:04.733422 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714660    6996 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 19:32:04.733422 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714665    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 19:32:04.733422 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714669    6996 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 19:32:04.733422 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714674    6996 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 19:32:04.733422 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714681    6996 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 19:32:04.733422 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714689    6996 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 19:32:04.733422 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714701    6996 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 19:32:04.733422 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714706    6996 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 19:32:04.734129 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714711    6996 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 19:32:04.734129 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714715    6996 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 19:32:04.734129 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714720    6996 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 19:32:04.734129 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714724    6996 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 19:32:04.734129 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714729    6996 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 19:32:04.734129 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714744    6996 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 19:32:04.734129 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714751    6996 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 19:32:04.734129 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714756    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 19:32:04.734129 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714761    6996 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 19:32:04.734129 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714765    6996 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 19:32:04.734129 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714770    6996 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 19:32:04.734129 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714779    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 19:32:04.734129 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714783    6996 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 19:32:04.734129 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714787    6996 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 19:32:04.734129 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714792    6996 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 19:32:04.734129 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714796    6996 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 19:32:04.734129 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714801    6996 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 19:32:04.734129 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714806    6996 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 19:32:04.734129 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714810    6996 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 19:32:04.734129 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714815    6996 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 19:32:04.734906 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714819    6996 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 19:32:04.734906 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714824    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 19:32:04.734906 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714828    6996 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 19:32:04.734906 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714837    6996 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 19:32:04.734906 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714842    6996 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 19:32:04.734906 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714847    6996 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 19:32:04.734906 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714851    6996 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 19:32:04.734906 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714855    6996 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 19:32:04.734906 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714862    6996 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 19:32:04.734906 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714867    6996 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 19:32:04.734906 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714871    6996 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 19:32:04.734906 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714875    6996 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 19:32:04.734906 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714879    6996 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 19:32:04.734906 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714883    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 19:32:04.734906 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714887    6996 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 19:32:04.734906 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714891    6996 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 19:32:04.734906 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714906    6996 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 19:32:04.734906 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714910    6996 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 19:32:04.734906 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714917    6996 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 19:32:04.735708 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714923    6996 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 19:32:04.735708 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714927    6996 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 19:32:04.735708 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714932    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 19:32:04.735708 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714937    6996 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 19:32:04.735708 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714945    6996 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 19:32:04.735708 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714949    6996 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 19:32:04.735708 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714954    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 19:32:04.735708 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714958    6996 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 19:32:04.735708 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714963    6996 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 19:32:04.735708 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714974    6996 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 19:32:04.735708 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714978    6996 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 19:32:04.735708 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714983    6996 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 19:32:04.735708 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714988    6996 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 19:32:04.735708 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714992    6996 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 19:32:04.735708 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.714999    6996 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 19:32:04.735708 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.715005    6996 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 19:32:04.735708 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.715011    6996 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 19:32:04.735708 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.715015    6996 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 19:32:04.735708 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.715020    6996 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 19:32:04.736515 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.715025    6996 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 19:32:04.736515 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.715029    6996 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 19:32:04.736515 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.715038    6996 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 19:32:04.736515 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.715042    6996 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 19:32:04.736515 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.715046    6996 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 19:32:04.736515 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.715051    6996 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 19:32:04.736515 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:04.715058    6996 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 19:32:04.736515 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.715177    6996 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 19:32:04.736515 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.716606    6996 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Oct 30 19:32:04.736515 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.716672    6996 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Oct 30 19:32:04.736515 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.716862    6996 server.go:1014] "Starting client certificate rotation"
Oct 30 19:32:04.736515 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.717001    6996 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 19:32:04.736515 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.717047    6996 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 19:25:53 +0000 UTC" deadline="2028-03-26 18:04:15.8651703 +0000 UTC"
Oct 30 19:32:04.736515 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.717073    6996 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12310h32m11.148098918s"
Oct 30 19:32:04.736515 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.717345    6996 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 19:32:04.736515 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.718340    6996 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 19:32:04.736992 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.719535    6996 log.go:25] "Validated CRI v1 runtime API"
Oct 30 19:32:04.736992 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.721470    6996 log.go:25] "Validated CRI v1 image API"
Oct 30 19:32:04.736992 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.722209    6996 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 19:32:04.736992 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.725515    6996 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 d6bc9f45-8852-4ac1-b191-801a3d47d7d9:/dev/nvme0n1p4 f03684ec-7bb8-4c81-a0fa-d307748c4205:/dev/nvme0n1p3]
Oct 30 19:32:04.737127 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.725544    6996 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1374ca566c18177a9872baff538f680e5090425d1dc15b2ecc1e65d5929c2237/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1374ca566c18177a9872baff538f680e5090425d1dc15b2ecc1e65d5929c2237/userdata/shm major:0 minor:476 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1970eefbe58c6156df633f635d604fa121b88455a5781a5fda64c3537802322a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1970eefbe58c6156df633f635d604fa121b88455a5781a5fda64c3537802322a/userdata/shm major:0 minor:471 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2313f16d736a5221d08e1f95e5b8ab2af5d3d250f681e6f0d6c58416e0502cc4/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2313f16d736a5221d08e1f95e5b8ab2af5d3d250f681e6f0d6c58416e0502cc4/userdata/shm major:0 minor:205 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/24754be54bc89f364cfe432df10c2c6d7d47a52f7a9313105640ce82a6e5a54e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/24754be54bc89f364cfe432df10c2c6d7d47a52f7a9313105640ce82a6e5a54e/userdata/shm major:0 minor:95 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/32a5d73fd496680155bec45a71c11338e31bfbe4529f4f057c9ff41e97d1fbe7/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/32a5d73fd496680155bec45a71c11338e31bfbe4529f4f057c9ff41e97d1fbe7/userdata/shm major:0 minor:395 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/4134771bb05a59b201e34b75eb0925ec7b9e192de9248a9f0161110e6dc89a32/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/4134771bb05a59b201e34b75eb0925ec7b9e192de9248a9f0161110e6dc89a32/userdata/shm major:0 minor:48 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/4c388040c36db938ab6e8b9aa5380695690ebfc3798a1baba340df88847d47dc/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/4c388040c36db938ab6e8b9aa5380695690ebfc3798a1baba340df88847d47dc/userdata/shm major:0 minor:361 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/4f542d1caf4aeb13175f8810167ccf8611f0265dbcf47e71bdebf1c575d22e7f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/4f542d1caf4aeb13175f8810167ccf8611f0265dbcf47e71bdebf1c575d22e7f/userdata/shm major:0 minor:271 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/54b8f021991e26a526673f6dff681f418709cea4abb400a0abbdfec24cf722be/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/54b8f021991e26a526673f6dff681f418709cea4abb400a0abbdfec24cf722be/userdata/shm major:0 minor:239 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/5959efc147c29abce752d39e65fffae0a2335eec259a1e3de2618a12d358e116/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/5959efc147c29abce752d39e65fffae0a2335eec259a1e3de2618a12d358e116/userdata/shm major:0 minor:311 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/622e7a0cce2a941e66b9921201ac3a833f7d138ac2a11a7a730d0851aa315991/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/622e7a0cce2a941e66b9921201ac3a833f7d138ac2a11a7a730d0851aa315991/userdata/shm major:0 minor:90 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/71dda31d74e3da240f255df32253cd6410431f0fbf23733eea76701b3c04fc5e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/71dda31d74e3da240f255df32253cd6410431f0fbf23733eea76701b3c04fc5e/userdata/shm major:0 minor:478 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8371052014d19ca2a7b06e031c1c5aebd406a27e12a242c1773973b46689359a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8371052014d19ca2a7b06e031c1c5aebd406a27e12a242c1773973b46689359a/userdata/shm major:0 minor:331 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/86c49726545dad2c141d39e101d778e9462326e082fa11f86d0353997aab0e7b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/86c49726545dad2c141d39e101d778e9462326e082fa11f86d0353997aab0e7b/userdata/shm major:0 minor:80 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a78bea986ecd7304024f9866af6d58af75a3209bb96fe5f1644b4e1c6e2d07cb/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a78bea986ecd7304024f9866af6d58af75a3209bb96fe5f1644b4e1c6e2d07cb/userdata/shm major:0 minor:72 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a92e741ee1642fdb90a23191249ff6f142e0f8bdc25b987a15900a41b61c4dcd/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a92e741ee1642fdb90a23191249ff6f142e0f8bdc25b987a15900a41b61c4dcd/userdata/shm major:0 minor:340 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/aa8ff2e85b70fb63a568d8b38871125373cc5857c308306bd7c35426b68d32f2/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/aa8ff2e85b70fb63a568d8b38871125373cc5857c308306bd7c35426b68d32f2/userdata/shm major:0 minor:207 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/aed45bc206861537fda3348ae1fa90ce9ae2788424d46356d5e92ff68acc77a4/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/aed45bc206861537fda3348ae1fa90ce9ae2788424d46356d5e92ff68acc77a4/userdata/shm major:0 minor:209 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b628691bb3e641012cc218b5b75787f95203955ab8fadc0de9627fe887df7842/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b628691bb3e641012cc218b5b75787f95203955ab8fadc0de9627fe887df7842/userdata/shm major:0 minor:459 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/cc9f277b2b9cf4478544dfdbbe924957afc897a82e1b27ae0133c7a94cc6ce3e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/cc9f277b2b9cf4478544dfdbbe924957afc897a82e1b27ae0133c7a94cc6ce3e/userdata/shm major:0 minor:93 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/dc6a849d14e5d5769caf5ac8d09449b75867ed2072639589d9d4bcf42e9eff11/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/dc6a849d14e5d5769caf5ac8d09449b75867ed2072639589d9d4bcf42e9eff11/userdata/shm major:0 minor:84 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/df0e67a4af24841460982f934d29e2f26264261184ee6a69067718ae926c739f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/df0e67a4af24841460982f934d29e2f26264261184ee6a69067718ae926c739f/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e2ef6d2d62c1316cad7cb0a0c15c8d10909426f11d5ce3aa1ace901fda021b50/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e2ef6d2d62c1316cad7cb0a0c15c8d10909426f11d5ce3aa1ace901fda021b50/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e841b85a1079cb72d30538e44a703671f61625f93f932c45d8658bb417102983/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e841b85a1079cb72d30538e44a703671f61625f93f932c45d8658bb417102983/userdata/shm major:0 minor:371 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f1ea5994ad4b3958ca822900d5f83d172b0e2279bbea9cf345f46ef1abf4f082/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f1ea5994ad4b3958ca822900d5f83d172b0e2279bbea9cf345f46ef1abf4f082/userdata/shm major:0 minor:269 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ffc43bb54a4e85852ad1661bee7e08beb0d0b0ce2c01f742abd0c63e16bb9525/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ffc43bb54a4e85852ad1661bee7e08beb0d0b0ce2c01f742abd0c63e16bb9525/userdata/shm major:0 minor:76 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0100ec89-6366-4946-99ce-4a254850c8fd/volumes/kubernetes.io~projected/kube-api-access-gsb6w:{mountpoint:/var/lib/kubelet/pods/0100ec89-6366-4946-99ce-4a254850c8fd/volumes/kubernetes.io~projected/kube-api-access-gsb6w major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/06b45974-f6ef-4b3a-b354-8103d7895fcf/volume-subpaths/web-config/config-reloader/2:{mountpoint:/var/lib/kubelet/pods/06b45974-f6ef-4b3a-b354-8103d7895fcf/volume-subpaths/web-config/config-reloader/2 major:0 minor:384 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/06b45974-f6ef-4b3a-b354-8103d7895fcf/volume-subpaths/web-config/init-config-reloader/2:{mountpoint:/var/lib/kubelet/pods/06b45974-f6ef-4b3a-b354-8103d7895fcf/volume-subpaths/web-config/init-config-reloader/2 major:0 minor:384 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/06b45974-f6ef-4b3a-b354-8103d7895fcf/volume-subpaths/web-config/prometheus/15:{mountpoint:/var/lib/kubelet/pods/06b45974-f6ef-4b3a-b354-8103d7895fcf/volume-subpaths/web-config/prometheus/15 major:0 minor:384 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/06b45974-f6ef-4b3a-b354-8103d7895fcf/volumes/kubernetes.io~empty-dir/config-out:{mountpoint:/var/lib/kubelet/pods/06b45974-f6ef-4b3a-b354-8103d7895fcf/volumes/kubernetes.io~empty-dir/config-out major:0 minor:386 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/06b45974-f6ef-4b3a-b354-8103d7895fcf/volumes/kubernetes.io~projected/kube-api-access-w94gr:{mountpoint:/var/lib/kubelet/pods/06b45974-f6ef-4b3a-b354-8103d7895fcf/volumes/kubernetes.io~projected/kube-api-access-w94gr major:0 minor:392 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/06b45974-f6ef-4b3a-b354-8103d7895fcf/volumes/kubernetes.io~projected/tls-assets:{mountpoint:/var/lib/kubelet/pods/06b45974-f6ef-4b3a-b354-8103d7895fcf/volumes/kubernetes.io~projected/tls-assets major:0 minor:377 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/06b45974-f6ef-4b3a-b354-8103d7895fcf/volumes/kubernetes.io~secret/config:{mountpoint:/var/lib/kubelet/pods/06b45974-f6ef-4b3a-b354-8103d7895fcf/volumes/kubernetes.io~secret/config major:0 minor:389 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/06b45974-f6ef-4b3a-b354-8103d7895fcf/volumes/kubernetes.io~secret/secret-grpc-tls:{mountpoint:/var/lib/kubelet/pods/06b45974-f6ef-4b3a-b354-8103d7895fcf/volumes/kubernetes.io~secret/secret-grpc-tls major:0 minor:382 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/06b45974-f6ef-4b3a-b354-8103d7895fcf/volumes/kubernetes.io~secret/secret-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/06b45974-f6ef-4b3a-b354-8103d7895fcf/volumes/kubernetes.io~secret/secret-kube-rbac-proxy major:0 minor:390 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/06b45974-f6ef-4b3a-b354-8103d7895fcf/volumes/kubernetes.io~secret/secret-metrics-client-certs:{mountpoint:/var/lib/kubelet/pods/06b45974-f6ef-4b3a-b354-8103d7895fcf/volumes/kubernetes.io~secret/secret-metrics-client-certs major:0 minor:387 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/06b45974-f6ef-4b3a-b354-8103d7895fcf/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/06b45974-f6ef-4b3a-b354-8103d7895fcf/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web major:0 minor:388 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/06b45974-f6ef-4b3a-b354-8103d7895fcf/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls:{mountpoint:/var/lib/kubelet/pods/06b45974-f6ef-4b3a-b354-8103d7895fcf/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls major:0 minor:385 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/06b45974-f6ef-4b3a-b354-8103d7895fcf/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls:{mountpoint:/var/lib/kubelet/pods/06b45974-f6ef-4b3a-b354-8103d7895fcf/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls major:0 minor:383 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/06b45974-f6ef-4b3a-b354-8103d7895fcf/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file:{mountpoint:/var/lib/kubelet/pods/06b45974-f6ef-4b3a-b354-8103d7895fcf/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file major:0 minor:391 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/06b45974-f6ef-4b3a-b354-8103d7895fcf/volumes/kubernetes.io~secret/web-config:{mountpoint:/var/lib/kubelet/pods/06b45974-f6ef-4b3a-b354-8103d7895fcf/volumes/kubernetes.io~secret/web-config major:0 minor:384 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd/volumes/kubernetes.io~projected/kube-api-access-lphsg:{mountpoint:/var/lib/kubelet/pods/0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd/volumes/kubernetes.io~projected/kube-api-access-lphsg major:0 minor:268 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config major:0 minor:259 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd/volumes/kubernetes.io~secret/node-exporter-tls:{mountpoint:/var/lib/kubelet/pods/0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd/volumes/kubernetes.io~secret/node-exporter-tls major:0 minor:265 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb/volumes/kubernetes.io~projected/kube-api-access-wdcn4:{mountpoint:/var/lib/kubelet/pods/136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb/volumes/kubernetes.io~projected/kube-api-access-wdcn4 major:0 minor:325 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb/volumes/kubernetes.io~secret/client-ca-bundle:{mountpoint:/var/lib/kubelet/pods/136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb/volumes/kubernetes.io~secret/client-ca-bundle major:0 minor:323 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb/volumes/kubernetes.io~secret/secret-metrics-server-client-certs:{mountpoint:/var/lib/kubelet/pods/136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb/volumes/kubernetes.io~secret/secret-metrics-server-client-certs major:0 minor:321 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb/volumes/kubernetes.io~secret/secret-metrics-server-tls:{mountpoint:/var/lib/kubelet/pods/136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb/volumes/kubernetes.io~secret/secret-metrics-server-tls major:0 minor:322 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/141cf3eb-d99b-4876-b65a-a2847584ff5d/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/141cf3eb-d99b-4876-b65a-a2847584ff5d/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:470 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/17f17805-edc3-4dc5-8e75-819350666927/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/17f17805-edc3-4dc5-8e75-819350666927/volumes/kubernetes.io~secret/agent-certs major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1977f3d7-c7f6-44cc-8a59-9d4f35adc0f7/volumes/kubernetes.io~projected/kube-api-access-4x29s:{mountpoint:/var/lib/kubelet/pods/1977f3d7-c7f6-44cc-8a59-9d4f35adc0f7/volumes/kubernetes.io~projected/kube-api-access-4x29s major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1cc9603f-1c61-43e4-b987-0e185cf26252/volumes/kubernetes.io~projected/kube-api-access-bmz8l:{mountpoint:/var/lib/kubelet/pods/1cc9603f-1c61-43e4-b987-0e185cf26252/volumes/kubernetes.io~projected/kube-api-access-bmz8l major:0 minor:337 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1cc9603f-1c61-43e4-b987-0e185cf26252/volumes/kubernetes.io~secret/monitoring-plugin-cert:{mountpoint:/var/lib/kubelet/pods/1cc9603f-1c61-43e4-b987-0e185cf26252/volumes/kubernetes.io~secret/monitoring-plugin-cert major:0 minor:336 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1f3e7563-7109-45a5-9a77-7676a0697bd1/volumes/kubernetes.io~projected/kube-api-access-2mqpt:{mountpoint:/var/lib/kubelet/pods/1f3e7563-7109-45a5-9a77-7676a0697bd1/volumes/kubernetes.io~projected/kube-api-access-2mqpt major:0 minor:370 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1f3e7563-7109-45a5-9a77-7676a0697bd1/volumes/kubernetes.io~secret/console-oauth-config:{mountpoint:/var/lib/kubelet/pods/1f3e7563-7109-45a5-9a77-7676a0697bd1/volumes/kubernetes.io~secret/console-oauth-config major:0 minor:369 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1f3e7563-7109-45a5-9a77-7676a0697bd1/volumes/kubernetes.io~secret/console-serving-cert:{mountpoint:/var/lib/kubelet/pods/1f3e7563-7109-45a5-9a77-7676a0697bd1/volumes/kubernetes.io~secret/console-serving-cert major:0 minor:368 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/517d6b00-0305-40cc-95c3-bba4adb2f849/volumes/kubernetes.io~projected/kube-api-access-tv2lm:{mountpoint:/var/lib/kubelet/pods/517d6b00-0305-40cc-95c3-bba4adb2f849/volumes/kubernetes.io~projected/kube-api-access-tv2lm major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/517d6b00-0305-40cc-95c3-bba4adb2f849/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/517d6b00-0305-40cc-95c3-bba4adb2f849/volumes/kubernetes.io~secret/cert major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/559fa506-ed9a-4c61-9316-edc85d6c5399/volumes/kubernetes.io~projected/kube-api-access-bh7cj:{mountpoint:/var/lib/kubelet/pods/559fa506-ed9a-4c61-9316-edc85d6c5399/volumes/kubernetes.io~projected/kube-api-access-bh7cj major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/559fa506-ed9a-4c61-9316-edc85d6c5399/volumes/kubernetes.io~secret/insights-runtime-extractor-tls:{mountpoint:/var/lib/kubelet/pods/559fa506-ed9a-4c61-9316-edc85d6c5399/volumes/kubernetes.io~secret/insights-runtime-extractor-tls major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/63678441-271f-46d7-ac1d-57a60bb78b12/volumes/kubernetes.io~projected/kube-api-access-9mwb6:{mountpoint:/var/lib/kubelet/pods/63678441-271f-46d7-ac1d-57a60bb78b12/volumes/kubernetes.io~projected/kube-api-access-9mwb6 major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/63effc7d-50b8-425f-b9a8-891ce2259283/volumes/kubernetes.io~projected/kube-api-access-q6kdm:{mountpoint:/var/lib/kubelet/pods/63effc7d-50b8-425f-b9a8-891ce2259283/volumes/kubernetes.io~projected/kube-api-access-q6kdm major:0 minor:458 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/63effc7d-50b8-425f-b9a8-891ce2259283/volumes/kubernetes.io~secret/console-oauth-config:{mountpoint:/var/lib/kubelet/pods/63effc7d-50b8-425f-b9a8-891ce2259283/volumes/kubernetes.io~secret/console-oauth-config major:0 minor:457 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/63effc7d-50b8-425f-b9a8-891ce2259283/volumes/kubernetes.io~secret/console-serving-cert:{mountpoint:/var/lib/kubelet/pods/63effc7d-50b8-425f-b9a8-891ce2259283/volumes/kubernetes.io~secret/console-serving-cert major:0 minor:453 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e/volumes/kubernetes.io~empty-dir/tmp major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e/volumes/kubernetes.io~projected/kube-api-access-mhsf4:{mountpoint:/var/lib/kubelet/pods/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e/volumes/kubernetes.io~projected/kube-api-access-mhsf4 major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6eca1a1e-101a-43bb-9fde-505c3182c10a/volumes/kubernetes.io~projected/kube-api-access-8bls4:{mountpoint:/var/lib/kubelet/pods/6eca1a1e-101a-43bb-9fde-505c3182c10a/volumes/kubernetes.io~projected/kube-api-access-8bls4 major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/79ea5164-32e2-4505-b92a-1e50495a6492/volumes/kubernetes.io~projected/kube-api-access-n87xw:{mountpoint:/var/lib/kubelet/pods/79ea5164-32e2-4505-b92a-1e50495a6492/volumes/kubernetes.io~projected/kube-api-access-n87xw major:0 minor:65 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7bb33e32-f8ac-44a2-95a4-cb949b08710f/volumes/kubernetes.io~projected/kube-api-access-jx8qx:{mountpoint:/var/lib/kubelet/pods/7bb33e32-f8ac-44a2-95a4-cb949b08710f/volumes/kubernetes.io~projected/kube-api-access-jx8qx major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7bb33e32-f8ac-44a2-95a4-cb949b08710f/volumes/kubernetes.io~secret/metrics-tls:{mountpoint:/var/lib/kubelet/pods/7bb33e32-f8ac-44a2-95a4-cb949b08710f/volumes/kubernetes.io~secret/metrics-tls major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7ff4ab79-1d7c-49ac-9d63-31772ad68b42/volumes/kubernetes.io~projected/kube-api-access-nn6bt:{mountpoint:/var/lib/kubelet/pods/7ff4ab79-1d7c-49ac-9d63-31772ad68b42/volumes/kubernetes.io~projected/kube-api-access-nn6bt major:0 minor:473 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/82384cc1-894f-4bb6-9b1d-567c72004575/volumes/kubernetes.io~projected/kube-api-access-fs44x:{mountpoint:/var/lib/kubelet/pods/82384cc1-894f-4bb6-9b1d-567c72004575/volumes/kubernetes.io~projected/kube-api-access-fs44x major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/82384cc1-894f-4bb6-9b1d-567c72004575/volumes/kubernetes.io~secret/metrics-certs:{mountpoint:/var/lib/kubelet/pods/82384cc1-894f-4bb6-9b1d-567c72004575/volumes/kubernetes.io~secret/metrics-certs major:0 minor:465 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8cde7f30-1d5b-4bea-ad21-2264b567ca1d/volumes/kubernetes.io~projected/kube-api-access-5j8qc:{mountpoint:/var/lib/kubelet/pods/8cde7f30-1d5b-4bea-ad21-2264b567ca1d/volumes/kubernetes.io~projected/kube-api-access-5j8qc major:0 minor:267 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8cde7f30-1d5b-4bea-ad21-2264b567ca1d/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/8cde7f30-1d5b-4bea-ad21-2264b567ca1d/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config major:0 minor:264 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8cde7f30-1d5b-4bea-ad21-2264b567ca1d/volumes/kubernetes.io~secret/openshift-state-metrics-tls:{mountpoint:/var/lib/kubelet/pods/8cde7f30-1d5b-4bea-ad21-2264b567ca1d/volumes/kubernetes.io~secret/openshift-state-metrics-tls major:0 minor:266 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/94ecba26-0809-414d-b084-8de0a693795e/volumes/kubernetes.io~projected/kube-api-access-x98xh:{mountpoint:/var/lib/kubelet/pods/94ecba26-0809-414d-b084-8de0a693795e/volumes/kubernetes.io~projected/kube-api-access-x98xh major:0 minor:299 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/94ecba26-0809-414d-b084-8de0a693795e/volumes/kubernetes.io~secret/secret-grpc-tls:{mountpoint:/var/lib/kubelet/pods/94ecba26-0809-414d-b084-8de0a693795e/volumes/kubernetes.io~secret/secret-grpc-tls major:0 minor:298 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/94ecba26-0809-414d-b084-8de0a693795e/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/94ecba26-0809-414d-b084-8de0a693795e/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy major:0 minor:296 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/94ecba26-0809-414d-b084-8de0a693795e/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics:{mountpoint:/var/lib/kubelet/pods/94ecba26-0809-414d-b084-8de0a693795e/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics major:0 minor:290 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/94ecba26-0809-414d-b084-8de0a693795e/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules:{mountpoint:/var/lib/kubelet/pods/94ecba26-0809-414d-b084-8de0a693795e/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules major:0 minor:295 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/94ecba26-0809-414d-b084-8de0a693795e/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/94ecba26-0809-414d-b084-8de0a693795e/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web major:0 minor:294 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/94ecba26-0809-414d-b084-8de0a693795e/volumes/kubernetes.io~secret/secret-thanos-querier-tls:{mountpoint:/var/lib/kubelet/pods/94ecba26-0809-414d-b084-8de0a693795e/volumes/kubernetes.io~secret/secret-thanos-querier-tls major:0 minor:297 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bfd79074-21c8-4caf-8c63-a2c5ff45e26d/volumes/kubernetes.io~projected/kube-api-access-xlnsz:{mountpoint:/var/lib/kubelet/pods/bfd79074-21c8-4caf-8c63-a2c5ff45e26d/volumes/kubernetes.io~projected/kube-api-access-xlnsz major:0 minor:360 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bfd79074-21c8-4caf-8c63-a2c5ff45e26d/volumes/kubernetes.io~secret/federate-client-tls:{mountpoint:/var/lib/kubelet/pods/bfd79074-21c8-4caf-8c63-a2c5ff45e26d/volumes/kubernetes.io~secret/federate-client-tls major:0 minor:355 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bfd79074-21c8-4caf-8c63-a2c5ff45e26d/volumes/kubernetes.io~secret/secret-telemeter-client:{mountpoint:/var/lib/kubelet/pods/bfd79074-21c8-4caf-8c63-a2c5ff45e26d/volumes/kubernetes.io~secret/secret-telemeter-client major:0 minor:356 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bfd79074-21c8-4caf-8c63-a2c5ff45e26d/volumes/kubernetes.io~secret/secret-telemeter-client-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/bfd79074-21c8-4caf-8c63-a2c5ff45e26d/volumes/kubernetes.io~secret/secret-telemeter-client-kube-rbac-proxy-config major:0 minor:351 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bfd79074-21c8-4caf-8c63-a2c5ff45e26d/volumes/kubernetes.io~secret/telemeter-client-tls:{mountpoint:/var/lib/kubelet/pods/bfd79074-21c8-4caf-8c63-a2c5ff45e26d/volumes/kubernetes.io~secret/telemeter-client-tls major:0 minor:359 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3/volumes/kubernetes.io~projected/kube-api-access-f9wcq:{mountpoint:/var/lib/kubelet/pods/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3/volumes/kubernetes.io~projected/kube-api-access-f9wcq major:0 minor:238 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3/volumes/kubernetes.io~secret/console-oauth-config:{mountpoint:/var/lib/kubelet/pods/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3/volumes/kubernetes.io~secret/console-oauth-config major:0 minor:230 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3/volumes/kubernetes.io~secret/console-serving-cert:{mountpoint:/var/lib/kubelet/pods/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3/volumes/kubernetes.io~secret/console-serving-cert major:0 minor:235 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d0036363-bf0e-44d7-b285-44546ac729e6/volumes/kubernetes.io~projected/kube-api-access-pxjvg:{mountpoint:/var/lib/kubelet/pods/d0036363-bf0e-44d7-b285-44546ac729e6/volumes/kubernetes.io~projected/kube-api-access-pxjvg major:0 minor:64 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d4d142b9-22f1-427f-a374-ba4918487946/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/d4d142b9-22f1-427f-a374-ba4918487946/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d4d142b9-22f1-427f-a374-ba4918487946/volumes/kubernetes.io~projected/kube-api-access-gt5qd:{mountpoint:/var/lib/kubelet/pods/d4d142b9-22f1-427f-a374-ba4918487946/volumes/kubernetes.io~projected/kube-api-access-gt5qd major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d4d142b9-22f1-427f-a374-ba4918487946/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/d4d142b9-22f1-427f-a374-ba4918487946/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:56 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/d73cb1e1a7b516b5c9807220d554dd879d9b7b642efcea0bcd80b1785b75e5b4/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/af3cd633904d6861db4ea131d43dc8194a6be1dffd547a8e2523296bf2213e1f/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/56ae63f315861fae16e9bd2eb0c4053d82cd94813298290abeb04ee907b10afc/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-106:{mountpoint:/var/lib/containers/storage/overlay/3ea8f83472a4d76a6d5732d116c058a1c33178b3c5f7468d2caa1fb543283ab4/merged major:0 minor:106 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/162d8044b8f4a32d60fcc0f4c6c7be5fee2c5ada630011133b4e6c600e98835c/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/7ee1790b00cff707adc308ff043d3d9186ff35fee42b044b8b9d85508d71e5aa/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/8a944798ac50e7428922f3438dd91af727d9125a86f4e03f4146161014a19d94/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/058f09ed0f5ec54433dfea195c33497f21988a366e5c1737ee4c7b051c1c3ef2/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/5ca66569e3a1b928f1df487606f69a1eb472cc9731c4fd21ba601529c4c48357/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-134:{mountpoint:/var/lib/containers/storage/overlay/6af530cf48a06c5ac1841e90148b0b47c14f77930915a3d4b921b6161f5b8000/merged major:0 minor:134 fsType:overlay blockSize:0} overlay_0-142:{mountpoint:/var/lib/containers/storage/overlay/054ee0c4d82decc5ea86bdb3c3e3b62c082024cfc5bce6b0985e37b9f45a11d0/merged major:0 minor:142 fsType:overlay blockSize:0} overlay_0-145:{mountpoint:/var/lib/containers/storage/overlay/72a9a350222201c1a7577476da3813d7f87b6cff237592d06a7d7bcb55f6c2fb/merged major:0 minor:145 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/ed24293e7e3bcd22357b429699c4a467bd703ea1f96f300a24a27e1eea62edf4/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-154:{mountpoint:/var/lib/containers/storage/overlay/cc9401cdbefc5a160240e1f494e91127152188ad0bc6f691320d9956b7a5f190/merged major:0 minor:154 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/01f9b6df54468181be420c6ab58e2a0a437e274d18417c09dcfb98a2e52c7e6f/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/3ea004cbd2a3c89a9f085629b5534ca0a7c790d10ae82ace9f264495e1fdb955/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/d5aceb2fd1a3c6ca0c05fddd00e435bbc3d0976a8f99635d5f6f902aa49c9a1c/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/d424641cee3ba2984584e677ae21670169533d1efbb72a1c476b2cc8489d6b9a/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/a573d5d7fa3fbd658cfbb734cba2967b4d13ada7110070d64bcde65e0a996411/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/b224dd75e452219d7bb380e350ba0904ef87d3e0ce299e0ba0c84f232b29550e/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/ab164e06cf537eeb9bb6f2bbaa4ead598d7d0dd0c08efa00c45eae8f316b6cb8/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-211:{mountpoint:/var/lib/containers/storage/overlay/91a9c8acf5b682256a65dac06b2acd8b96a52c8e4ac180429cd041dd651bc1e4/merged major:0 minor:211 fsType:overlay blockSize:0} overlay_0-213:{mountpoint:/var/lib/containers/storage/overlay/88668531cc88c59bc69af31d90143a6577be80e8761140b5ad40f05786f95936/merged major:0 minor:213 fsType:overlay blockSize:0} overlay_0-215:{mountpoint:/var/lib/containers/storage/overlay/2bf22f9972316e40faec74fb5ed5ce0c75779d4c6c48e483ae6e0c49c7dddd16/merged major:0 minor:215 fsType:overlay blockSize:0} overlay_0-217:{mountpoint:/var/lib/containers/storage/overlay/c8833d1579f498bad1704c34027d0055380f511ef61d018c015da55ecbdd4fe9/merged major:0 minor:217 fsType:overlay blockSize:0} overlay_0-219:{mountpoint:/var/lib/containers/storage/overlay/92cec486a111810839378f355b709e092e889c0f5d1d749a637db8f27852399a/merged major:0 minor:219 fsType:overlay blockSize:0} overlay_0-225:{mountpoint:/var/lib/containers/storage/overlay/625d2a7f048156e6c7567776b339d03503d36e809eac11cda1ab2c80ea3c448b/merged major:0 minor:225 fsType:overlay blockSize:0} overlay_0-236:{mountpoint:/var/lib/containers/storage/overlay/48a0e828d34feb4b3277e9386740fdcc289acb5706acdd5ba917628a4cdb4109/merged major:0 minor:236 fsType:overlay blockSize:0} overlay_0-244:{mountpoint:/var/lib/containers/storage/overlay/99b85d5f1f4522877d0222409e4d3a06d8f74748a064164c35e902cf83230e5d/merged major:0 minor:244 fsType:overlay blockSize:0} overlay_0-246:{mountpoint:/var/lib/containers/storage/overlay/8534fe63a91b3cfc61a3a9ec07a49157a2acd7df19b6f37d2b6d097e1cfa0fc9/merged major:0 minor:246 fsType:overlay blockSize:0} overlay_0-248:{mountpoint:/var/lib/containers/storage/overlay/4dbfacaaa1a6995d813bdcf991eeb4c73a8e4cfeb14b8620f74ea88ed8d20f70/merged major:0 minor:248 fsType:overlay blockSize:0} overlay_0-253:{mountpoint:/var/lib/containers/storage/overlay/4b0f48c4eb5c68b753e0ff67735f0ff8c1c712467d264cd4be279dabddf1deeb/merged major:0 minor:253 fsType:overlay blockSize:0} overlay_0-273:{mountpoint:/var/lib/containers/storage/overlay/80ef24279f6287983c2889ccfdbed7b07d318ecd2baaea9901372c679f537989/merged major:0 minor:273 fsType:overlay blockSize:0} overlay_0-275:{mountpoint:/var/lib/containers/storage/overlay/d707926cc9719a861a04ade9d82b52afb71ec1e16b1ce8bb770e53601a06000d/merged major:0 minor:275 fsType:overlay blockSize:0} overlay_0-277:{mountpoint:/var/lib/containers/storage/overlay/75f8a282e4266a6c4a11763f792fba0eb50d9adba2f6f9edad9b3229ead72461/merged major:0 minor:277 fsType:overlay blockSize:0} overlay_0-279:{mountpoint:/var/lib/containers/storage/overlay/096e6dea742078d332e1703c0da9ff93d5c31087a3a5a08256920d142af25315/merged major:0 minor:279 fsType:overlay blockSize:0} overlay_0-285:{mountpoint:/var/lib/containers/storage/overlay/08e808ce33eea6fb9e4ac3883afef001230539304817dfd7eb73835921eecade/merged major:0 minor:285 fsType:overlay blockSize:0} overlay_0-300:{mountpoint:/var/lib/containers/storage/overlay/d4fca1c4505aa927f5df86b7c4cccd7eff7e5193f102352c836c00dcc67d1961/merged major:0 minor:300 fsType:overlay blockSize:0} overlay_0-302:{mountpoint:/var/lib/containers/storage/overlay/5e9d742594ceca2a99d24d7c23b04e9f92f31ec759f0ac766df1d73b4a6dfd5d/merged major:0 minor:302 fsType:overlay blockSize:0} overlay_0-307:{mountpoint:/var/lib/containers/storage/overlay/9a6dc3f3233c47b132736bf1fd355a3ec06f885f4afd9f989070b85a05213d84/merged major:0 minor:307 fsType:overlay blockSize:0} overlay_0-314:{mountpoint:/var/lib/containers/storage/overlay/fd249df891f2e16eac60629a488f43a8cd49598ffa9f679298bf1d63d9beaf1d/merged major:0 minor:314 fsType:overlay blockSize:0} overlay_0-316:{mountpoint:/var/lib/containers/storage/overlay/ec493678033c561887048245d13f46cafb5be5111b60b31065e90cac07b9296e/merged major:0 minor:316 fsType:overlay blockSize:0} overlay_0-326:{mountpoint:/var/lib/containers/storage/overlay/04390c645df08dbbad2a7fe90285b745e0f1f66a3c6e8279e7c57d4c5a30d618/merged major:0 minor:326 fsType:overlay blockSize:0} overlay_0-338:{mountpoint:/var/lib/containers/storage/overlay/2ed9fceb0428ceb22ff7e9fb968bdbdad1478fc170ae8701a2ad32f84c6f5934/merged major:0 minor:338 fsType:overlay blockSize:0} overlay_0-342:{mountpoint:/var/lib/containers/storage/overlay/b7829dab6b62285da57c6bfa2d320c0ad7ec5a5493370727dc6ccd3fde58f000/merged major:0 minor:342 fsType:overlay blockSize:0} overlay_0-344:{mountpoint:/var/lib/containers/storage/overlay/41d1e835259c4d951d89a007ba18d68353a347d3a87e882cb3827fa16f80df4e/merged major:0 minor:344 fsType:overlay blockSize:0} overlay_0-345:{mountpoint:/var/lib/containers/storage/overlay/3941f5af56337f60d56b8529efe2afd64120a7895ba4f584a546ac6725c15aff/merged major:0 minor:345 fsType:overlay blockSize:0} overlay_0-357:{mountpoint:/var/lib/containers/storage/overlay/f0a31c8b09c04eb78a0a8c10b140b4b3cd6a62c41b1733ca5ffd19f7b02d3920/merged major:0 minor:357 fsType:overlay blockSize:0} overlay_0-366:{mountpoint:/var/lib/containers/storage/overlay/3a22a88af6aa60a46fcff9698973ab13c88e507de6f50cc88bc6e7eeb28af495/merged major:0 minor:366 fsType:overlay blockSize:0} overlay_0-373:{mountpoint:/var/lib/containers/storage/overlay/8fc9e33f287684abd25613fe9d0d21ac3b0d2ed585cee288a6a24e9a2ff31b88/merged major:0 minor:373 fsType:overlay blockSize:0} overlay_0-375:{mountpoint:/var/lib/containers/storage/overlay/5a52c7d947d8f43204d860cf2d8df5b87073a13d111b375d53f90ba85202c416/merged major:0 minor:375 fsType:overlay blockSize:0} overlay_0-393:{mountpoint:/var/lib/containers/storage/overlay/6a6069caed3db27ae02023f18f8679e2251325a901ad6b72c58f018e049be974/merged major:0 minor:393 fsType:overlay blockSize:0} overlay_0-401:{mountpoint:/var/lib/containers/storage/overlay/1130e5bd6812655b4025f5a230d1adcae5978576ce7dae5f36fa0c59e27f77b0/merged major:0 minor:401 fsType:overlay blockSize:0} overlay_0-403:{mountpoint:/var/lib/containers/storage/overlay/253242d3a39964f61d8642dd55fdb13d1ce619b3fdbd1a7b763fc3eb1c08c558/merged major:0 minor:403 fsType:overlay blockSize:0} overlay_0-408:{mountpoint:/var/lib/containers/storage/overlay/a0cd5f15ca17baff5a2c7c9b12c7bdbe5dd921b51674ed82671662aa2d8fe021/merged major:0 minor:408 fsType:overlay blockSize:0} overlay_0-411:{mountpoint:/var/lib/containers/storage/overlay/079aa723e0f719dcab3eb79098fa7621d7f057376045fc6e049df3aae575af48/merged major:0 minor:411 fsType:overlay blockSize:0} overlay_0-417:{mountpoint:/var/lib/containers/storage/overlay/be26d10403dfaadefdadee2d10264af6b5d550dd419e891ecde69aa82ef0295a/merged major:0 minor:417 fsType:overlay blockSize:0} overlay_0-422:{mountpoint:/var/lib/containers/storage/overlay/c03a619ff55ec2989f855319e61247e727e28de7042e1a8b72116a61f254c1a0/merged major:0 minor:422 fsType:overlay blockSize:0} overlay_0-427:{mountpoint:/var/lib/containers/storage/overlay/34e7fe3b5dea46a24049dcce5c99e08c2fe88a3e3e72c576e595166881ac838a/merged major:0 minor:427 fsType:overlay blockSize:0} overlay_0-433:{mountpoint:/var/lib/containers/storage/overlay/942a1e11d8573324294fb145fdc6284b02ad85560e36b65f95dac933f87e7171/merged major:0 minor:433 fsType:overlay blockSize:0} overlay_0-438:{mountpoint:/var/lib/containers/storage/overlay/74c1e1be17000cdfcf498b4a4d17c08821501741ed7fa287ca3813f3c226199f/merged major:0 minor:438 fsType:overlay blockSize:0} overlay_0-443:{mountpoint:/var/lib/containers/storage/overlay/1b1fb2f0fbf4cb3015cbd6791074056d85067cfa66bded2dbbeee571e0f54adf/merged major:0 minor:443 fsType:overlay blockSize:0} overlay_0-448:{mountpoint:/var/lib/containers/storage/overlay/f0a157cb66c01fae1e856213a914d5ee954229915e9cf2a5fcc447549014c617/merged major:0 minor:448 fsType:overlay blockSize:0} overlay_0-461:{mountpoint:/var/lib/containers/storage/overlay/1573a9819a9e3d291639119d321e7b8e408efe77afdc86718ba907cfa6e4b245/merged major:0 minor:461 fsType:overlay blockSize:0} overlay_0-463:{mountpoint:/var/lib/containers/storage/overlay/45cf7d849df3907e3dae4f9a111c45bd26ced0c9d19c20848d4ddeda23f24844/merged major:0 minor:463 fsType:overlay blockSize:0} overlay_0-474:{mountpoint:/var/lib/containers/storage/overlay/369f7f4dec72b020de83499583efe0756a95e17a284e4e4dde7fb014e391b313/merged major:0 minor:474 fsType:overlay blockSize:0} overlay_0-480:{mountpoint:/var/lib/containers/storage/overlay/e653f406478fda0c2e3a70c4b181d4f32c8af647fd82db5a4ba08d4c31b3bea0/merged major:0 minor:480 fsType:overlay blockSize:0} overlay_0-482:{mountpoint:/var/lib/containers/storage/overlay/0515450bfd3801985ce726617d91895ed211c4c7e028249d64188a9f1659cb47/merged major:0 minor:482 fsType:overlay blockSize:0} overlay_0-484:{mountpoint:/var/lib/containers/storage/overlay/22c41a6c1d37c902b58cb47d0cf2bbf93b9943e0c6a1c7487bc3810d7222310e/merged major:0 minor:484 fsType:overlay blockSize:0} overlay_0-486:{mountpoint:/var/lib/containers/storage/overlay/5470329beb7254aa43824bb66e6cd714ba001f13e10b130113e501299d47ec7d/merged major:0 minor:486 fsType:overlay blockSize:0} overlay_0-492:{mountpoint:/var/lib/containers/storage/overlay/df8da33a0c8abf9dc34c58576acae7dc2566298337a561b420c54904bf0f71ee/merged major:0 minor:492 fsType:overlay blockSize:0} overlay_0-497:{mountpoint:/var/lib/containers/storage/overlay/a0a1975b73954676e81400034a0e5cf675393c48f10ffe26b87ede8255812c3c/merged major:0 minor:497 fsType:overlay blockSize:0} overlay_0-51:{mountpoint:/var/lib/containers/storage/overlay/e694a722645fae817eef9eb47d2e4aec6db32f89e073faa3ef38c9661d794282/merged major:0 minor:51 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/ea78a48dd25f0c399d8aaed2c2e940cd08ecccfab358c52510cbbf2fb7d30644/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/e3f66a8782e8621f06b39a3ffa7330c6678c69b7dfd8671d2577988d9666fc58/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/c88972e0132629e98f759998a7457182e539577f19af454b758594ca55a777b8/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-78:{mountpoint:/var/lib/containers/storage/overlay/1b77fc7a993dd2d2800a2df45f7f1915846d358fb4e624250f554489ddf4d205/merged major:0 minor:78 fsType:overlay blockSize:0} overlay_0-82:{mountpoint:/var/lib/containers/storage/overlay/0b59f4706af805ea28736e748007a7356f3ebe4376207ad77f1df5d38a1b80d6/merged major:0 minor:82 fsType:overlay blockSize:0} overlay_0-86:{mountpoint:/var/lib/containers/storage/overlay/6c9f15007a1ee3ef2e22b2bd1cf3999dc45a857513de442dcab56911d4ded11b/merged major:0 minor:86 fsType:overlay blockSize:0} overlay_0-88:{mountpoint:/var/lib/containers/storage/overlay/30767aa5220e1786936bce3b00e5e04cee982a9802a6be8c556b44ed45ea044d/merged major:0 minor:88 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/11934c56f820d1f7353d54020a8b44c2e9e0818ab9303d1df72a21e6d6297e24/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-97:{mountpoint:/var/lib/containers/storage/overlay/a45edf4b32f1f571bb1bdf26d9529253fd56b5d137ba24d901336154abb723af/merged major:0 minor:97 fsType:overlay blockSize:0}]
Oct 30 19:32:04.737424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.735781    6996 manager.go:217] Machine: {Timestamp:2026-10-30 19:32:04.734969425 +0000 UTC m=+0.063087571 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3193198 MemoryCapacity:33164337152 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec264751e003c2da3bc412ca3540ceea SystemUUID:ec264751-e003-c2da-3bc4-12ca3540ceea BootID:60fb3ca4-2a00-4bc5-8550-e209e9d3545b Filesystems:[{Device:/var/lib/kubelet/pods/63effc7d-50b8-425f-b9a8-891ce2259283/volumes/kubernetes.io~projected/kube-api-access-q6kdm DeviceMajor:0 DeviceMinor:458 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1374ca566c18177a9872baff538f680e5090425d1dc15b2ecc1e65d5929c2237/userdata/shm DeviceMajor:0 DeviceMinor:476 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a78bea986ecd7304024f9866af6d58af75a3209bb96fe5f1644b4e1c6e2d07cb/userdata/shm DeviceMajor:0 DeviceMinor:72 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/517d6b00-0305-40cc-95c3-bba4adb2f849/volumes/kubernetes.io~projected/kube-api-access-tv2lm DeviceMajor:0 DeviceMinor:203 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-225 DeviceMajor:0 DeviceMinor:225 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-236 DeviceMajor:0 DeviceMinor:236 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/24754be54bc89f364cfe432df10c2c6d7d47a52f7a9313105640ce82a6e5a54e/userdata/shm DeviceMajor:0 DeviceMinor:95 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-154 DeviceMajor:0 DeviceMinor:154 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-217 DeviceMajor:0 DeviceMinor:217 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb/volumes/kubernetes.io~secret/secret-metrics-server-client-certs DeviceMajor:0 DeviceMinor:321 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f1ea5994ad4b3958ca822900d5f83d172b0e2279bbea9cf345f46ef1abf4f082/userdata/shm DeviceMajor:0 DeviceMinor:269 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb/volumes/kubernetes.io~projected/kube-api-access-wdcn4 DeviceMajor:0 DeviceMinor:325 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e841b85a1079cb72d30538e44a703671f61625f93f932c45d8658bb417102983/userdata/shm DeviceMajor:0 DeviceMinor:371 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ffc43bb54a4e85852ad1661bee7e08beb0d0b0ce2c01f742abd0c63e16bb9525/userdata/shm DeviceMajor:0 DeviceMinor:76 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-253 DeviceMajor:0 DeviceMinor:253 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/06b45974-f6ef-4b3a-b354-8103d7895fcf/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:388 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-438 DeviceMajor:0 DeviceMinor:438 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-463 DeviceMajor:0 DeviceMinor:463 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:overlay_0-97 DeviceMajor:0 DeviceMinor:97 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/94ecba26-0809-414d-b084-8de0a693795e/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:294 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-373 DeviceMajor:0 DeviceMinor:373 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/06b45974-f6ef-4b3a-b354-8103d7895fcf/volumes/kubernetes.io~secret/web-config DeviceMajor:0 DeviceMinor:384 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-422 DeviceMajor:0 DeviceMinor:422 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:60 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-78 DeviceMajor:0 DeviceMinor:78 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3/volumes/kubernetes.io~projected/kube-api-access-f9wcq DeviceMajor:0 DeviceMinor:238 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/8cde7f30-1d5b-4bea-ad21-2264b567ca1d/volumes/kubernetes.io~projected/kube-api-access-5j8qc DeviceMajor:0 DeviceMinor:267 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/4f542d1caf4aeb13175f8810167ccf8611f0265dbcf47e71bdebf1c575d22e7f/userdata/shm DeviceMajor:0 DeviceMinor:271 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/bfd79074-21c8-4caf-8c63-a2c5ff45e26d/volumes/kubernetes.io~secret/federate-client-tls DeviceMajor:0 DeviceMinor:355 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-357 DeviceMajor:0 DeviceMinor:357 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-302 DeviceMajor:0 DeviceMinor:302 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-314 DeviceMajor:0 DeviceMinor:314 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a92e741ee1642fdb90a23191249ff6f142e0f8bdc25b987a15900a41b61c4dcd/userdata/shm DeviceMajor:0 DeviceMinor:340 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/bfd79074-21c8-4caf-8c63-a2c5ff45e26d/volumes/kubernetes.io~projected/kube-api-access-xlnsz DeviceMajor:0 DeviceMinor:360 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/1f3e7563-7109-45a5-9a77-7676a0697bd1/volumes/kubernetes.io~projected/kube-api-access-2mqpt DeviceMajor:0 DeviceMinor:370 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/06b45974-f6ef-4b3a-b354-8103d7895fcf/volumes/kubernetes.io~projected/tls-assets DeviceMajor:0 DeviceMinor:377 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/06b45974-f6ef-4b3a-b354-8103d7895fcf/volumes/kubernetes.io~secret/secret-grpc-tls DeviceMajor:0 DeviceMinor:382 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/94ecba26-0809-414d-b084-8de0a693795e/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules DeviceMajor:0 DeviceMinor:295 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/6eca1a1e-101a-43bb-9fde-505c3182c10a/volumes/kubernetes.io~projected/kube-api-access-8bls4 DeviceMajor:0 DeviceMinor:63 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/559fa506-ed9a-4c61-9316-edc85d6c5399/volumes/kubernetes.io~projected/kube-api-access-bh7cj DeviceMajor:0 DeviceMinor:204 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-213 DeviceMajor:0 DeviceMinor:213 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/bfd79074-21c8-4caf-8c63-a2c5ff45e26d/volumes/kubernetes.io~secret/telemeter-client-tls DeviceMajor:0 DeviceMinor:359 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/82384cc1-894f-4bb6-9b1d-567c72004575/volumes/kubernetes.io~secret/metrics-certs DeviceMajor:0 DeviceMinor:465 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/141cf3eb-d99b-4876-b65a-a2847584ff5d/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:470 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-277 DeviceMajor:0 DeviceMinor:277 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/82384cc1-894f-4bb6-9b1d-567c72004575/volumes/kubernetes.io~projected/kube-api-access-fs44x DeviceMajor:0 DeviceMinor:58 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/aa8ff2e85b70fb63a568d8b38871125373cc5857c308306bd7c35426b68d32f2/userdata/shm DeviceMajor:0 DeviceMinor:207 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/06b45974-f6ef-4b3a-b354-8103d7895fcf/volumes/kubernetes.io~empty-dir/config-out DeviceMajor:0 DeviceMinor:386 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/06b45974-f6ef-4b3a-b354-8103d7895fcf/volumes/kubernetes.io~projected/kube-api-access-w94gr DeviceMajor:0 DeviceMinor:392 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/d4d142b9-22f1-427f-a374-ba4918487946/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:56 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-142 DeviceMajor:0 DeviceMinor:142 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-219 DeviceMajor:0 DeviceMinor:219 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/94ecba26-0809-414d-b084-8de0a693795e/volumes/kubernetes.io~secret/secret-grpc-tls DeviceMajor:0 DeviceMinor:298 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/06b45974-f6ef-4b3a-b354-8103d7895fcf/volume-subpaths/web-config/init-config-reloader/2 DeviceMajor:0 DeviceMinor:384 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-279 DeviceMajor:0 DeviceMinor:279 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/06b45974-f6ef-4b3a-b354-8103d7895fcf/volumes/kubernetes.io~secret/secret-kube-rbac-proxy DeviceMajor:0 DeviceMinor:390 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/dc6a849d14e5d5769caf5ac8d09449b75867ed2072639589d9d4bcf42e9eff11/userdata/shm DeviceMajor:0 DeviceMinor:84 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-326 DeviceMajor:0 DeviceMinor:326 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-344 DeviceMajor:0 DeviceMinor:344 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1f3e7563-7109-45a5-9a77-7676a0697bd1/volumes/kubernetes.io~secret/console-serving-cert DeviceMajor:0 DeviceMinor:368 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/71dda31d74e3da240f255df32253cd6410431f0fbf23733eea76701b3c04fc5e/userdata/shm DeviceMajor:0 DeviceMinor:478 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-497 DeviceMajor:0 DeviceMinor:497 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d4d142b9-22f1-427f-a374-ba4918487946/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/54b8f021991e26a526673f6dff681f418709cea4abb400a0abbdfec24cf722be/userdata/shm DeviceMajor:0 DeviceMinor:239 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-244 DeviceMajor:0 DeviceMinor:244 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/94ecba26-0809-414d-b084-8de0a693795e/volumes/kubernetes.io~projected/kube-api-access-x98xh DeviceMajor:0 DeviceMinor:299 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/06b45974-f6ef-4b3a-b354-8103d7895fcf/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls DeviceMajor:0 DeviceMinor:383 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/06b45974-f6ef-4b3a-b354-8103d7895fcf/volumes/kubernetes.io~secret/secret-metrics-client-certs DeviceMajor:0 DeviceMinor:387 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/06b45974-f6ef-4b3a-b354-8103d7895fcf/volumes/kubernetes.io~secret/config DeviceMajor:0 DeviceMinor:389 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-51 DeviceMajor:0 DeviceMinor:51 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-82 DeviceMajor:0 DeviceMinor:82 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-285 DeviceMajor:0 DeviceMinor:285 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-393 DeviceMajor:0 DeviceMinor:393 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-443 DeviceMajor:0 DeviceMinor:443 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb/volumes/kubernetes.io~secret/secret-metrics-server-tls DeviceMajor:0 DeviceMinor:322 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/06b45974-f6ef-4b3a-b354-8103d7895fcf/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls DeviceMajor:0 DeviceMinor:385 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-408 DeviceMajor:0 DeviceMinor:408 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0100ec89-6366-4946-99ce-4a254850c8fd/volumes/kubernetes.io~projected/kube-api-access-gsb6w DeviceMajor:0 DeviceMinor:61 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-275 DeviceMajor:0 DeviceMinor:275 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-345 DeviceMajor:0 DeviceMinor:345 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/06b45974-f6ef-4b3a-b354-8103d7895fcf/volume-subpaths/web-config/config-reloader/2 DeviceMajor:0 DeviceMinor:384 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/63678441-271f-46d7-ac1d-57a60bb78b12/volumes/kubernetes.io~projected/kube-api-access-9mwb6 DeviceMajor:0 DeviceMinor:66 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3/volumes/kubernetes.io~secret/console-oauth-config DeviceMajor:0 DeviceMinor:230 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/94ecba26-0809-414d-b084-8de0a693795e/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics DeviceMajor:0 DeviceMinor:290 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/1cc9603f-1c61-43e4-b987-0e185cf26252/volumes/kubernetes.io~projected/kube-api-access-bmz8l DeviceMajor:0 DeviceMinor:337 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/63effc7d-50b8-425f-b9a8-891ce2259283/volumes/kubernetes.io~secret/console-oauth-config DeviceMajor:0 DeviceMinor:457 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-145 DeviceMajor:0 DeviceMinor:145 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-246 DeviceMajor:0 DeviceMinor:246 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-316 DeviceMajor:0 DeviceMinor:316 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-461 DeviceMajor:0 DeviceMinor:461 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-486 DeviceMajor:0 DeviceMinor:486 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/559fa506-ed9a-4c61-9316-edc85d6c5399/volumes/kubernetes.io~secret/insights-runtime-extractor-tls DeviceMajor:0 DeviceMinor:197 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-248 DeviceMajor:0 DeviceMinor:248 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-273 DeviceMajor:0 DeviceMinor:273 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/bfd79074-21c8-4caf-8c63-a2c5ff45e26d/volumes/kubernetes.io~secret/secret-telemeter-client DeviceMajor:0 DeviceMinor:356 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-401 DeviceMajor:0 DeviceMinor:401 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-433 DeviceMajor:0 DeviceMinor:433 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e2ef6d2d62c1316cad7cb0a0c15c8d10909426f11d5ce3aa1ace901fda021b50/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/1cc9603f-1c61-43e4-b987-0e185cf26252/volumes/kubernetes.io~secret/monitoring-plugin-cert DeviceMajor:0 DeviceMinor:336 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-427 DeviceMajor:0 DeviceMinor:427 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-484 DeviceMajor:0 DeviceMinor:484 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-492 DeviceMajor:0 DeviceMinor:492 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/17f17805-edc3-4dc5-8e75-819350666927/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:55 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-211 DeviceMajor:0 DeviceMinor:211 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/94ecba26-0809-414d-b084-8de0a693795e/volumes/kubernetes.io~secret/secret-thanos-querier-tls DeviceMajor:0 DeviceMinor:297 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-375 DeviceMajor:0 DeviceMinor:375 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-417 DeviceMajor:0 DeviceMinor:417 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1977f3d7-c7f6-44cc-8a59-9d4f35adc0f7/volumes/kubernetes.io~projected/kube-api-access-4x29s DeviceMajor:0 DeviceMinor:67 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/86c49726545dad2c141d39e101d778e9462326e082fa11f86d0353997aab0e7b/userdata/shm DeviceMajor:0 DeviceMinor:80 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-86 DeviceMajor:0 DeviceMinor:86 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-215 DeviceMajor:0 DeviceMinor:215 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8cde7f30-1d5b-4bea-ad21-2264b567ca1d/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:264 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/32a5d73fd496680155bec45a71c11338e31bfbe4529f4f057c9ff41e97d1fbe7/userdata/shm DeviceMajor:0 DeviceMinor:395 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/7ff4ab79-1d7c-49ac-9d63-31772ad68b42/volumes/kubernetes.io~projected/kube-api-access-nn6bt DeviceMajor:0 DeviceMinor:473 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e/volumes/kubernetes.io~projected/kube-api-access-mhsf4 DeviceMajor:0 DeviceMinor:62 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/d0036363-bf0e-44d7-b285-44546ac729e6/volumes/kubernetes.io~projected/kube-api-access-pxjvg DeviceMajor:0 DeviceMinor:64 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-338 DeviceMajor:0 DeviceMinor:338 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-106 DeviceMajor:0 DeviceMinor:106 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-134 DeviceMajor:0 DeviceMinor:134 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7bb33e32-f8ac-44a2-95a4-cb949b08710f/volumes/kubernetes.io~secret/metrics-tls DeviceMajor:0 DeviceMinor:45 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-474 DeviceMajor:0 DeviceMinor:474 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d4d142b9-22f1-427f-a374-ba4918487946/volumes/kubernetes.io~projected/kube-api-access-gt5qd DeviceMajor:0 DeviceMinor:57 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb/volumes/kubernetes.io~secret/client-ca-bundle DeviceMajor:0 DeviceMinor:323 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/df0e67a4af24841460982f934d29e2f26264261184ee6a69067718ae926c739f/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/cc9f277b2b9cf4478544dfdbbe924957afc897a82e1b27ae0133c7a94cc6ce3e/userdata/shm DeviceMajor:0 DeviceMinor:93 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/4c388040c36db938ab6e8b9aa5380695690ebfc3798a1baba340df88847d47dc/userdata/shm DeviceMajor:0 DeviceMinor:361 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/06b45974-f6ef-4b3a-b354-8103d7895fcf/volume-subpaths/web-config/prometheus/15 DeviceMajor:0 DeviceMinor:384 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/var/lib/kubelet/pods/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3/volumes/kubernetes.io~secret/console-serving-cert DeviceMajor:0 DeviceMinor:235 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8371052014d19ca2a7b06e031c1c5aebd406a27e12a242c1773973b46689359a/userdata/shm DeviceMajor:0 DeviceMinor:331 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-411 DeviceMajor:0 DeviceMinor:411 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/63effc7d-50b8-425f-b9a8-891ce2259283/volumes/kubernetes.io~secret/console-serving-cert DeviceMajor:0 DeviceMinor:453 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/517d6b00-0305-40cc-95c3-bba4adb2f849/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:201 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2313f16d736a5221d08e1f95e5b8ab2af5d3d250f681e6f0d6c58416e0502cc4/userdata/shm DeviceMajor:0 DeviceMinor:205 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/8cde7f30-1d5b-4bea-ad21-2264b567ca1d/volumes/kubernetes.io~secret/openshift-state-metrics-tls DeviceMajor:0 DeviceMinor:266 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-300 DeviceMajor:0 DeviceMinor:300 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-366 DeviceMajor:0 DeviceMinor:366 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-480 DeviceMajor:0 DeviceMinor:480 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:59 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd/volumes/kubernetes.io~secret/node-exporter-tls DeviceMajor:0 DeviceMinor:265 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-342 DeviceMajor:0 DeviceMinor:342 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/bfd79074-21c8-4caf-8c63-a2c5ff45e26d/volumes/kubernetes.io~secret/secret-telemeter-client-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:351 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/4134771bb05a59b201e34b75eb0925ec7b9e192de9248a9f0161110e6dc89a32/userdata/shm DeviceMajor:0 DeviceMinor:48 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-88 DeviceMajor:0 DeviceMinor:88 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/622e7a0cce2a941e66b9921201ac3a833f7d138ac2a11a7a730d0851aa315991/userdata/shm DeviceMajor:0 DeviceMinor:90 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd/volumes/kubernetes.io~projected/kube-api-access-lphsg DeviceMajor:0 DeviceMinor:268 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/5959efc147c29abce752d39e65fffae0a2335eec259a1e3de2618a12d358e116/userdata/shm DeviceMajor:0 DeviceMinor:311 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/1f3e7563-7109-45a5-9a77-7676a0697bd1/volumes/kubernetes.io~secret/console-oauth-config DeviceMajor:0 DeviceMinor:369 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-448 DeviceMajor:0 DeviceMinor:448 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1970eefbe58c6156df633f635d604fa121b88455a5781a5fda64c3537802322a/userdata/shm DeviceMajor:0 DeviceMinor:471 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7bb33e32-f8ac-44a2-95a4-cb949b08710f/volumes/kubernetes.io~projected/kube-api-access-jx8qx DeviceMajor:0 DeviceMinor:202 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b628691bb3e641012cc218b5b75787f95203955ab8fadc0de9627fe887df7842/userdata/shm DeviceMajor:0 DeviceMinor:459 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-482 DeviceMajor:0 DeviceMinor:482 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/79ea5164-32e2-4505-b92a-1e50495a6492/volumes/kubernetes.io~projected/kube-api-access-n87xw DeviceMajor:0 DeviceMinor:65 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/94ecba26-0809-414d-b084-8de0a693795e/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy DeviceMajor:0 DeviceMinor:296 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-307 DeviceMajor:0 DeviceMinor:307 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/06b45974-f6ef-4b3a-b354-8103d7895fcf/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file DeviceMajor:0 DeviceMinor:391 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/aed45bc206861537fda3348ae1fa90ce9ae2788424d46356d5e92ff68acc77a4/userdata/shm DeviceMajor:0 DeviceMinor:209 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:259 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-403 DeviceMajor:0 DeviceMinor:403 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:1374ca566c18177 MacAddress:6e:30:a0:73:bb:bf Speed:10000 Mtu:8901} {Name:1970eefbe58c615 MacAddress:ae:52:eb:9e:94:28 Speed:10000 Mtu:8901} {Name:2313f16d736a522 MacAddress:ae:45:4d:cd:52:50 Speed:10000 Mtu:8901} {Name:32a5d73fd496680 MacAddress:0e:d4:90:44:f8:7d Speed:10000 Mtu:8901} {Name:4c388040c36db93 MacAddress:ba:bd:52:3f:eb:87 Speed:10000 Mtu:8901} {Name:54b8f021991e26a MacAddress:36:fa:ce:4a:5b:93 Speed:10000 Mtu:8901} {Name:5959efc147c29ab MacAddress:1e:80:c7:df:a7:c5 Speed:10000 Mtu:8901} {Name:71dda31d74e3da2 MacAddress:82:c8:5e:cb:7b:7d Speed:10000 Mtu:8901} {Name:8371052014d19ca MacAddress:f6:1a:46:66:92:24 Speed:10000 Mtu:8901} {Name:a92e741ee1642fd MacAddress:ca:41:db:b7:14:35 Speed:10000 Mtu:8901} {Name:aa8ff2e85b70fb6 MacAddress:da:4e:27:42:66:3a Speed:10000 Mtu:8901} {Name:aed45bc20686153 MacAddress:be:c5:83:1c:6c:34 Speed:10000 Mtu:8901} {Name:b628691bb3e6410 MacAddress:fe:97:66:e7:7a:9e Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:ff:ed:e0:f7:8b Speed:0 Mtu:9001} {Name:br-int MacAddress:96:4e:e1:33:4e:d2 Speed:0 Mtu:8901} {Name:e841b85a1079cb7 MacAddress:fe:fa:36:5d:72:1c Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:ff:ed:e0:f7:8b Speed:0 Mtu:9001} {Name:f1ea5994ad4b395 MacAddress:7a:e4:23:a2:35:1f Speed:10000 Mtu:8901} {Name:genev_sys_6081 MacAddress:f2:e9:51:47:21:a3 Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:86:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:1e:fa:2b:9f:ac:ed Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164337152 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 19:32:04.737424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.736570    6996 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 19:32:04.737424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.736630    6996 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 19:32:04.737424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.736899    6996 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 19:32:04.737424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.736926    6996 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-42.us-east-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 19:32:04.737424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.737049    6996 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 19:32:04.737424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.737056    6996 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 19:32:04.737424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.737066    6996 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 19:32:04.737424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.737085    6996 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 19:32:04.737424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.737173    6996 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 19:32:04.737424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.737322    6996 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 19:32:04.737424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.737423    6996 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 19:32:04.738080 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.737456    6996 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 19:32:04.738080 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.737469    6996 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 19:32:04.738080 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.737495    6996 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 19:32:04.738080 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.737504    6996 kubelet.go:405] "Adding apiserver pod source"
Oct 30 19:32:04.738080 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.737512    6996 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 19:32:04.738369 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.738353    6996 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 19:32:04.738399 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.738372    6996 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 19:32:04.738927 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.738898    6996 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 19:32:04.742324 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.742309    6996 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 19:32:04.742324 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.742323    6996 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 19:32:04.742429 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.742420    6996 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Oct 30 19:32:04.742630 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.742619    6996 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 19:32:04.742662 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.742648    6996 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 19:32:04.742756 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.742747    6996 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 19:32:04.742785 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.742761    6996 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 19:32:04.742785 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.742767    6996 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 19:32:04.742785 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.742772    6996 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 19:32:04.742785 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.742779    6996 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 19:32:04.742877 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.742788    6996 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 19:32:04.742877 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.742797    6996 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 19:32:04.742877 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.742803    6996 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 19:32:04.742877 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.742810    6996 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 19:32:04.742877 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.742816    6996 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 19:32:04.742877 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.742829    6996 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 19:32:04.742877 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.742838    6996 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 19:32:04.742877 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.742863    6996 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 19:32:04.742877 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.742870    6996 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 19:32:04.746351 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.746335    6996 server.go:1263] "Started kubelet"
Oct 30 19:32:04.748636 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.746544    6996 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 19:32:04.748636 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.746993    6996 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 19:32:04.748636 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.747253    6996 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 19:32:04.748636 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.747565    6996 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 19:32:04.748636 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.748042    6996 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 19:32:04.746925 ip-10-0-141-42 systemd[1]: Started Kubernetes Kubelet.
Oct 30 19:32:04.754193 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.754010    6996 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 19:32:04.754193 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.754036    6996 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 19:32:04.754338 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.754258    6996 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 19:32:04.754338 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.754271    6996 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 19:32:04.754448 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.754366    6996 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 19:32:04.754556 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.754096    6996 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 19:25:55 +0000 UTC" deadline="2028-06-16 09:11:12.450047476 +0000 UTC"
Oct 30 19:32:04.754605 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.754561    6996 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14269h39m7.695492103s"
Oct 30 19:32:04.754781 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:32:04.754761    6996 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-141-42.us-east-2.compute.internal\" not found"
Oct 30 19:32:04.756534 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.756516    6996 factory.go:55] Registering systemd factory
Oct 30 19:32:04.756534 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.756536    6996 factory.go:223] Registration of the systemd container factory successfully
Oct 30 19:32:04.756929 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.756908    6996 factory.go:153] Registering CRI-O factory
Oct 30 19:32:04.757006 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.756934    6996 factory.go:223] Registration of the crio container factory successfully
Oct 30 19:32:04.757006 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.756983    6996 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 19:32:04.757006 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.757003    6996 factory.go:103] Registering Raw factory
Oct 30 19:32:04.757142 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.757017    6996 manager.go:1199] Started watching for new ooms in manager
Oct 30 19:32:04.757435 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.757423    6996 manager.go:319] Starting recovery of all containers
Oct 30 19:32:04.757806 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.757757    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="559fa506-ed9a-4c61-9316-edc85d6c5399" volumeName="kubernetes.io/configmap/559fa506-ed9a-4c61-9316-edc85d6c5399-kube-rbac-proxy-cm" seLinuxMountContext=""
Oct 30 19:32:04.757879 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.757813    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="06b45974-f6ef-4b3a-b354-8103d7895fcf" volumeName="kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-configmap-kubelet-serving-ca-bundle" seLinuxMountContext=""
Oct 30 19:32:04.757879 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.757828    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="17f17805-edc3-4dc5-8e75-819350666927" volumeName="kubernetes.io/secret/17f17805-edc3-4dc5-8e75-819350666927-agent-certs" seLinuxMountContext=""
Oct 30 19:32:04.757879 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.757841    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1977f3d7-c7f6-44cc-8a59-9d4f35adc0f7" volumeName="kubernetes.io/empty-dir/1977f3d7-c7f6-44cc-8a59-9d4f35adc0f7-tmp-dir" seLinuxMountContext=""
Oct 30 19:32:04.757879 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.757854    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3" volumeName="kubernetes.io/configmap/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-oauth-serving-cert" seLinuxMountContext=""
Oct 30 19:32:04.757879 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.757866    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="06b45974-f6ef-4b3a-b354-8103d7895fcf" volumeName="kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-web-config" seLinuxMountContext=""
Oct 30 19:32:04.757879 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.757879    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd" volumeName="kubernetes.io/projected/0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd-kube-api-access-lphsg" seLinuxMountContext=""
Oct 30 19:32:04.758186 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.757907    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1f3e7563-7109-45a5-9a77-7676a0697bd1" volumeName="kubernetes.io/configmap/1f3e7563-7109-45a5-9a77-7676a0697bd1-oauth-serving-cert" seLinuxMountContext=""
Oct 30 19:32:04.758186 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.757926    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e" volumeName="kubernetes.io/empty-dir/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-tmp" seLinuxMountContext=""
Oct 30 19:32:04.758186 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.757937    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="94ecba26-0809-414d-b084-8de0a693795e" volumeName="kubernetes.io/secret/94ecba26-0809-414d-b084-8de0a693795e-secret-thanos-querier-kube-rbac-proxy" seLinuxMountContext=""
Oct 30 19:32:04.758186 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.757947    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="06b45974-f6ef-4b3a-b354-8103d7895fcf" volumeName="kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-secret-prometheus-k8s-kube-rbac-proxy-web" seLinuxMountContext=""
Oct 30 19:32:04.758186 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.757959    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e" volumeName="kubernetes.io/projected/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-kube-api-access-mhsf4" seLinuxMountContext=""
Oct 30 19:32:04.758186 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.757974    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bfd79074-21c8-4caf-8c63-a2c5ff45e26d" volumeName="kubernetes.io/secret/bfd79074-21c8-4caf-8c63-a2c5ff45e26d-federate-client-tls" seLinuxMountContext=""
Oct 30 19:32:04.758186 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.757986    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d4d142b9-22f1-427f-a374-ba4918487946" volumeName="kubernetes.io/configmap/d4d142b9-22f1-427f-a374-ba4918487946-ovnkube-script-lib" seLinuxMountContext=""
Oct 30 19:32:04.758186 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.757999    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="06b45974-f6ef-4b3a-b354-8103d7895fcf" volumeName="kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-configmap-metrics-client-ca" seLinuxMountContext=""
Oct 30 19:32:04.758186 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758011    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb" volumeName="kubernetes.io/projected/136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb-kube-api-access-wdcn4" seLinuxMountContext=""
Oct 30 19:32:04.758186 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758022    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="63effc7d-50b8-425f-b9a8-891ce2259283" volumeName="kubernetes.io/projected/63effc7d-50b8-425f-b9a8-891ce2259283-kube-api-access-q6kdm" seLinuxMountContext=""
Oct 30 19:32:04.758186 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758037    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="79ea5164-32e2-4505-b92a-1e50495a6492" volumeName="kubernetes.io/projected/79ea5164-32e2-4505-b92a-1e50495a6492-kube-api-access-n87xw" seLinuxMountContext=""
Oct 30 19:32:04.758186 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758057    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d0036363-bf0e-44d7-b285-44546ac729e6" volumeName="kubernetes.io/projected/d0036363-bf0e-44d7-b285-44546ac729e6-kube-api-access-pxjvg" seLinuxMountContext=""
Oct 30 19:32:04.758186 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758068    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="06b45974-f6ef-4b3a-b354-8103d7895fcf" volumeName="kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-thanos-prometheus-http-client-file" seLinuxMountContext=""
Oct 30 19:32:04.758186 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758081    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="94ecba26-0809-414d-b084-8de0a693795e" volumeName="kubernetes.io/empty-dir/94ecba26-0809-414d-b084-8de0a693795e-tmp" seLinuxMountContext=""
Oct 30 19:32:04.758186 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758094    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="06b45974-f6ef-4b3a-b354-8103d7895fcf" volumeName="kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-secret-metrics-client-certs" seLinuxMountContext=""
Oct 30 19:32:04.758186 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758102    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="63effc7d-50b8-425f-b9a8-891ce2259283" volumeName="kubernetes.io/configmap/63effc7d-50b8-425f-b9a8-891ce2259283-console-config" seLinuxMountContext=""
Oct 30 19:32:04.758186 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758111    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="63effc7d-50b8-425f-b9a8-891ce2259283" volumeName="kubernetes.io/empty-dir/63effc7d-50b8-425f-b9a8-891ce2259283-tmp" seLinuxMountContext=""
Oct 30 19:32:04.758186 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758119    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3" volumeName="kubernetes.io/configmap/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-console-config" seLinuxMountContext=""
Oct 30 19:32:04.758186 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758128    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="06b45974-f6ef-4b3a-b354-8103d7895fcf" volumeName="kubernetes.io/empty-dir/06b45974-f6ef-4b3a-b354-8103d7895fcf-config-out" seLinuxMountContext=""
Oct 30 19:32:04.758186 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758137    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1f3e7563-7109-45a5-9a77-7676a0697bd1" volumeName="kubernetes.io/projected/1f3e7563-7109-45a5-9a77-7676a0697bd1-kube-api-access-2mqpt" seLinuxMountContext=""
Oct 30 19:32:04.758186 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758144    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="517d6b00-0305-40cc-95c3-bba4adb2f849" volumeName="kubernetes.io/secret/517d6b00-0305-40cc-95c3-bba4adb2f849-cert" seLinuxMountContext=""
Oct 30 19:32:04.758186 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758155    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="79ea5164-32e2-4505-b92a-1e50495a6492" volumeName="kubernetes.io/configmap/79ea5164-32e2-4505-b92a-1e50495a6492-whereabouts-flatfile-configmap" seLinuxMountContext=""
Oct 30 19:32:04.758186 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758176    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="94ecba26-0809-414d-b084-8de0a693795e" volumeName="kubernetes.io/projected/94ecba26-0809-414d-b084-8de0a693795e-kube-api-access-x98xh" seLinuxMountContext=""
Oct 30 19:32:04.758186 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758185    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d4d142b9-22f1-427f-a374-ba4918487946" volumeName="kubernetes.io/configmap/d4d142b9-22f1-427f-a374-ba4918487946-env-overrides" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758206    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="06b45974-f6ef-4b3a-b354-8103d7895fcf" volumeName="kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-secret-prometheus-k8s-tls" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758215    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd" volumeName="kubernetes.io/empty-dir/0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd-node-exporter-textfile" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758225    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bfd79074-21c8-4caf-8c63-a2c5ff45e26d" volumeName="kubernetes.io/secret/bfd79074-21c8-4caf-8c63-a2c5ff45e26d-secret-telemeter-client-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758233    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1f3e7563-7109-45a5-9a77-7676a0697bd1" volumeName="kubernetes.io/configmap/1f3e7563-7109-45a5-9a77-7676a0697bd1-trusted-ca-bundle" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758241    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="559fa506-ed9a-4c61-9316-edc85d6c5399" volumeName="kubernetes.io/empty-dir/559fa506-ed9a-4c61-9316-edc85d6c5399-data-volume" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758249    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="63effc7d-50b8-425f-b9a8-891ce2259283" volumeName="kubernetes.io/configmap/63effc7d-50b8-425f-b9a8-891ce2259283-trusted-ca-bundle" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758257    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0100ec89-6366-4946-99ce-4a254850c8fd" volumeName="kubernetes.io/projected/0100ec89-6366-4946-99ce-4a254850c8fd-kube-api-access-gsb6w" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758264    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb" volumeName="kubernetes.io/empty-dir/136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb-audit-log" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758272    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="63678441-271f-46d7-ac1d-57a60bb78b12" volumeName="kubernetes.io/projected/63678441-271f-46d7-ac1d-57a60bb78b12-kube-api-access-9mwb6" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758279    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="63effc7d-50b8-425f-b9a8-891ce2259283" volumeName="kubernetes.io/configmap/63effc7d-50b8-425f-b9a8-891ce2259283-service-ca" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758286    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bfd79074-21c8-4caf-8c63-a2c5ff45e26d" volumeName="kubernetes.io/configmap/bfd79074-21c8-4caf-8c63-a2c5ff45e26d-metrics-client-ca" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758297    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bfd79074-21c8-4caf-8c63-a2c5ff45e26d" volumeName="kubernetes.io/secret/bfd79074-21c8-4caf-8c63-a2c5ff45e26d-secret-telemeter-client" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758304    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb" volumeName="kubernetes.io/configmap/136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb-configmap-kubelet-serving-ca-bundle" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758311    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1f3e7563-7109-45a5-9a77-7676a0697bd1" volumeName="kubernetes.io/configmap/1f3e7563-7109-45a5-9a77-7676a0697bd1-console-config" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758318    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="517d6b00-0305-40cc-95c3-bba4adb2f849" volumeName="kubernetes.io/projected/517d6b00-0305-40cc-95c3-bba4adb2f849-kube-api-access-tv2lm" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758325    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bfd79074-21c8-4caf-8c63-a2c5ff45e26d" volumeName="kubernetes.io/configmap/bfd79074-21c8-4caf-8c63-a2c5ff45e26d-serving-certs-ca-bundle" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758333    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="06b45974-f6ef-4b3a-b354-8103d7895fcf" volumeName="kubernetes.io/projected/06b45974-f6ef-4b3a-b354-8103d7895fcf-kube-api-access-w94gr" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758342    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd" volumeName="kubernetes.io/configmap/0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd-metrics-client-ca" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758350    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb" volumeName="kubernetes.io/secret/136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb-secret-metrics-server-client-certs" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758358    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="63effc7d-50b8-425f-b9a8-891ce2259283" volumeName="kubernetes.io/configmap/63effc7d-50b8-425f-b9a8-891ce2259283-oauth-serving-cert" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758367    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bfd79074-21c8-4caf-8c63-a2c5ff45e26d" volumeName="kubernetes.io/projected/bfd79074-21c8-4caf-8c63-a2c5ff45e26d-kube-api-access-xlnsz" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758374    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb" volumeName="kubernetes.io/secret/136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb-secret-metrics-server-tls" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758382    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="94ecba26-0809-414d-b084-8de0a693795e" volumeName="kubernetes.io/secret/94ecba26-0809-414d-b084-8de0a693795e-secret-grpc-tls" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758392    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3" volumeName="kubernetes.io/empty-dir/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-tmp" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758401    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3" volumeName="kubernetes.io/secret/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-console-oauth-config" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758432    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="06b45974-f6ef-4b3a-b354-8103d7895fcf" volumeName="kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-configmap-serving-certs-ca-bundle" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758439    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="06b45974-f6ef-4b3a-b354-8103d7895fcf" volumeName="kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-prometheus-trusted-ca-bundle" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758447    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e" volumeName="kubernetes.io/empty-dir/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-etc-tuned" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758454    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3" volumeName="kubernetes.io/projected/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-kube-api-access-f9wcq" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758462    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="06b45974-f6ef-4b3a-b354-8103d7895fcf" volumeName="kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-secret-kube-rbac-proxy" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758471    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="141cf3eb-d99b-4876-b65a-a2847584ff5d" volumeName="kubernetes.io/secret/141cf3eb-d99b-4876-b65a-a2847584ff5d-original-pull-secret" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758498    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7bb33e32-f8ac-44a2-95a4-cb949b08710f" volumeName="kubernetes.io/projected/7bb33e32-f8ac-44a2-95a4-cb949b08710f-kube-api-access-jx8qx" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758510    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8cde7f30-1d5b-4bea-ad21-2264b567ca1d" volumeName="kubernetes.io/projected/8cde7f30-1d5b-4bea-ad21-2264b567ca1d-kube-api-access-5j8qc" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758519    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bfd79074-21c8-4caf-8c63-a2c5ff45e26d" volumeName="kubernetes.io/secret/bfd79074-21c8-4caf-8c63-a2c5ff45e26d-telemeter-client-tls" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758528    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1f3e7563-7109-45a5-9a77-7676a0697bd1" volumeName="kubernetes.io/secret/1f3e7563-7109-45a5-9a77-7676a0697bd1-console-oauth-config" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758545    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="94ecba26-0809-414d-b084-8de0a693795e" volumeName="kubernetes.io/configmap/94ecba26-0809-414d-b084-8de0a693795e-metrics-client-ca" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758556    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb" volumeName="kubernetes.io/configmap/136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb-metrics-server-audit-profiles" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758566    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb" volumeName="kubernetes.io/secret/136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb-client-ca-bundle" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758621    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="559fa506-ed9a-4c61-9316-edc85d6c5399" volumeName="kubernetes.io/secret/559fa506-ed9a-4c61-9316-edc85d6c5399-insights-runtime-extractor-tls" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758633    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3" volumeName="kubernetes.io/configmap/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-trusted-ca-bundle" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758640    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1f3e7563-7109-45a5-9a77-7676a0697bd1" volumeName="kubernetes.io/configmap/1f3e7563-7109-45a5-9a77-7676a0697bd1-service-ca" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758648    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1f3e7563-7109-45a5-9a77-7676a0697bd1" volumeName="kubernetes.io/secret/1f3e7563-7109-45a5-9a77-7676a0697bd1-console-serving-cert" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758657    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8cde7f30-1d5b-4bea-ad21-2264b567ca1d" volumeName="kubernetes.io/secret/8cde7f30-1d5b-4bea-ad21-2264b567ca1d-openshift-state-metrics-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758666    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7bb33e32-f8ac-44a2-95a4-cb949b08710f" volumeName="kubernetes.io/secret/7bb33e32-f8ac-44a2-95a4-cb949b08710f-metrics-tls" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758673    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7ff4ab79-1d7c-49ac-9d63-31772ad68b42" volumeName="kubernetes.io/projected/7ff4ab79-1d7c-49ac-9d63-31772ad68b42-kube-api-access-nn6bt" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758681    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="94ecba26-0809-414d-b084-8de0a693795e" volumeName="kubernetes.io/secret/94ecba26-0809-414d-b084-8de0a693795e-secret-thanos-querier-kube-rbac-proxy-metrics" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758687    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3" volumeName="kubernetes.io/secret/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-console-serving-cert" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758695    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="17f17805-edc3-4dc5-8e75-819350666927" volumeName="kubernetes.io/configmap/17f17805-edc3-4dc5-8e75-819350666927-konnectivity-ca" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758704    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="559fa506-ed9a-4c61-9316-edc85d6c5399" volumeName="kubernetes.io/projected/559fa506-ed9a-4c61-9316-edc85d6c5399-kube-api-access-bh7cj" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758711    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6eca1a1e-101a-43bb-9fde-505c3182c10a" volumeName="kubernetes.io/configmap/6eca1a1e-101a-43bb-9fde-505c3182c10a-serviceca" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758720    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="79ea5164-32e2-4505-b92a-1e50495a6492" volumeName="kubernetes.io/configmap/79ea5164-32e2-4505-b92a-1e50495a6492-cni-binary-copy" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758728    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="94ecba26-0809-414d-b084-8de0a693795e" volumeName="kubernetes.io/secret/94ecba26-0809-414d-b084-8de0a693795e-secret-thanos-querier-kube-rbac-proxy-web" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758736    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d4d142b9-22f1-427f-a374-ba4918487946" volumeName="kubernetes.io/secret/d4d142b9-22f1-427f-a374-ba4918487946-ovn-node-metrics-cert" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758744    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="06b45974-f6ef-4b3a-b354-8103d7895fcf" volumeName="kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-prometheus-k8s-rulefiles-0" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758751    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="06b45974-f6ef-4b3a-b354-8103d7895fcf" volumeName="kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-secret-grpc-tls" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758761    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="06b45974-f6ef-4b3a-b354-8103d7895fcf" volumeName="kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-secret-prometheus-k8s-thanos-sidecar-tls" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758772    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd" volumeName="kubernetes.io/secret/0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd-node-exporter-tls" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758779    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1cc9603f-1c61-43e4-b987-0e185cf26252" volumeName="kubernetes.io/projected/1cc9603f-1c61-43e4-b987-0e185cf26252-kube-api-access-bmz8l" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758786    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3" volumeName="kubernetes.io/configmap/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-service-ca" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758796    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="06b45974-f6ef-4b3a-b354-8103d7895fcf" volumeName="kubernetes.io/projected/06b45974-f6ef-4b3a-b354-8103d7895fcf-tls-assets" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758803    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1977f3d7-c7f6-44cc-8a59-9d4f35adc0f7" volumeName="kubernetes.io/projected/1977f3d7-c7f6-44cc-8a59-9d4f35adc0f7-kube-api-access-4x29s" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758811    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1cc9603f-1c61-43e4-b987-0e185cf26252" volumeName="kubernetes.io/secret/1cc9603f-1c61-43e4-b987-0e185cf26252-monitoring-plugin-cert" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758820    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7bb33e32-f8ac-44a2-95a4-cb949b08710f" volumeName="kubernetes.io/configmap/7bb33e32-f8ac-44a2-95a4-cb949b08710f-config-volume" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758828    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7bb33e32-f8ac-44a2-95a4-cb949b08710f" volumeName="kubernetes.io/empty-dir/7bb33e32-f8ac-44a2-95a4-cb949b08710f-tmp-dir" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758837    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="82384cc1-894f-4bb6-9b1d-567c72004575" volumeName="kubernetes.io/secret/82384cc1-894f-4bb6-9b1d-567c72004575-metrics-certs" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758845    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8cde7f30-1d5b-4bea-ad21-2264b567ca1d" volumeName="kubernetes.io/configmap/8cde7f30-1d5b-4bea-ad21-2264b567ca1d-metrics-client-ca" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758863    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="06b45974-f6ef-4b3a-b354-8103d7895fcf" volumeName="kubernetes.io/empty-dir/06b45974-f6ef-4b3a-b354-8103d7895fcf-prometheus-k8s-db" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758873    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1f3e7563-7109-45a5-9a77-7676a0697bd1" volumeName="kubernetes.io/empty-dir/1f3e7563-7109-45a5-9a77-7676a0697bd1-tmp" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758883    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="63678441-271f-46d7-ac1d-57a60bb78b12" volumeName="kubernetes.io/configmap/63678441-271f-46d7-ac1d-57a60bb78b12-multus-daemon-config" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758890    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="79ea5164-32e2-4505-b92a-1e50495a6492" volumeName="kubernetes.io/configmap/79ea5164-32e2-4505-b92a-1e50495a6492-cni-sysctl-allowlist" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758904    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bfd79074-21c8-4caf-8c63-a2c5ff45e26d" volumeName="kubernetes.io/configmap/bfd79074-21c8-4caf-8c63-a2c5ff45e26d-telemeter-trusted-ca-bundle" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758912    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="94ecba26-0809-414d-b084-8de0a693795e" volumeName="kubernetes.io/secret/94ecba26-0809-414d-b084-8de0a693795e-secret-thanos-querier-kube-rbac-proxy-rules" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758919    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d4d142b9-22f1-427f-a374-ba4918487946" volumeName="kubernetes.io/configmap/d4d142b9-22f1-427f-a374-ba4918487946-ovnkube-config" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758926    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d4d142b9-22f1-427f-a374-ba4918487946" volumeName="kubernetes.io/projected/d4d142b9-22f1-427f-a374-ba4918487946-kube-api-access-gt5qd" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758933    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="63678441-271f-46d7-ac1d-57a60bb78b12" volumeName="kubernetes.io/configmap/63678441-271f-46d7-ac1d-57a60bb78b12-cni-binary-copy" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758940    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="63effc7d-50b8-425f-b9a8-891ce2259283" volumeName="kubernetes.io/secret/63effc7d-50b8-425f-b9a8-891ce2259283-console-serving-cert" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758948    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0100ec89-6366-4946-99ce-4a254850c8fd" volumeName="kubernetes.io/configmap/0100ec89-6366-4946-99ce-4a254850c8fd-iptables-alerter-script" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758956    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="06b45974-f6ef-4b3a-b354-8103d7895fcf" volumeName="kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-prometheus-k8s-rulefiles-1" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758963    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd" volumeName="kubernetes.io/configmap/0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd-node-exporter-accelerators-collector-config" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758972    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="06b45974-f6ef-4b3a-b354-8103d7895fcf" volumeName="kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-config" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.758979    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd" volumeName="kubernetes.io/secret/0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd-node-exporter-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.759007    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="63effc7d-50b8-425f-b9a8-891ce2259283" volumeName="kubernetes.io/secret/63effc7d-50b8-425f-b9a8-891ce2259283-console-oauth-config" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.759031    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6eca1a1e-101a-43bb-9fde-505c3182c10a" volumeName="kubernetes.io/projected/6eca1a1e-101a-43bb-9fde-505c3182c10a-kube-api-access-8bls4" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.759038    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="82384cc1-894f-4bb6-9b1d-567c72004575" volumeName="kubernetes.io/projected/82384cc1-894f-4bb6-9b1d-567c72004575-kube-api-access-fs44x" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.759046    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8cde7f30-1d5b-4bea-ad21-2264b567ca1d" volumeName="kubernetes.io/secret/8cde7f30-1d5b-4bea-ad21-2264b567ca1d-openshift-state-metrics-tls" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.759055    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="06b45974-f6ef-4b3a-b354-8103d7895fcf" volumeName="kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-prometheus-k8s-rulefiles-2" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.759063    6996 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="94ecba26-0809-414d-b084-8de0a693795e" volumeName="kubernetes.io/secret/94ecba26-0809-414d-b084-8de0a693795e-secret-thanos-querier-tls" seLinuxMountContext=""
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.759070    6996 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 19:32:04.759424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.759076    6996 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 19:32:04.767642 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.767625    6996 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Oct 30 19:32:04.776127 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.775890    6996 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 19:32:04.778471 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.778441    6996 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 19:32:04.778978 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.778953    6996 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 19:32:04.783592 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.783559    6996 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 19:32:04.783592 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.783591    6996 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 19:32:04.783733 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.783615    6996 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 19:32:04.783733 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:32:04.783670    6996 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Oct 30 19:32:04.783844 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.783825    6996 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 19:32:04.793617 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.793587    6996 generic.go:356] "Generic (PLEG): container finished" podID="c73221e6d3bc811e32d70f92ba61e280" containerID="fe5d517e6465dec695facc2299abdf8f8b576c4000b73562db9ae5441a2c5b1c" exitCode=0
Oct 30 19:32:04.795943 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.795919    6996 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 19:32:04.808628 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.808605    6996 generic.go:356] "Generic (PLEG): container finished" podID="0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd" containerID="7bffe48f200a9e527f3f41a2e90681541f98710b76ce283d380d916857687dc4" exitCode=0
Oct 30 19:32:04.824666 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.824644    6996 generic.go:356] "Generic (PLEG): container finished" podID="06b45974-f6ef-4b3a-b354-8103d7895fcf" containerID="67eff76e35d54c9bb9f908398b7e410860adfb0a1adc657d551d62f0fc21744a" exitCode=0
Oct 30 19:32:04.833601 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.833581    6996 generic.go:356] "Generic (PLEG): container finished" podID="79ea5164-32e2-4505-b92a-1e50495a6492" containerID="17c529387bfe794eedc3eb734d277de132f57cc48c4ecc0cffbf3dc3c1f6c057" exitCode=0
Oct 30 19:32:04.833601 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.833599    6996 generic.go:356] "Generic (PLEG): container finished" podID="79ea5164-32e2-4505-b92a-1e50495a6492" containerID="59e177e0ff64c1a680dc22a75215aeb1640a97b49d204789f9ec8276c744ce07" exitCode=0
Oct 30 19:32:04.833601 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.833605    6996 generic.go:356] "Generic (PLEG): container finished" podID="79ea5164-32e2-4505-b92a-1e50495a6492" containerID="92ab9b79e4d2dca1f83a300cf372440b5f0ba7742b259371d4ad309627f9738b" exitCode=0
Oct 30 19:32:04.833756 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.833611    6996 generic.go:356] "Generic (PLEG): container finished" podID="79ea5164-32e2-4505-b92a-1e50495a6492" containerID="c88f94df906706a3e9dd404f1a8c10215763b241ced15c9a44778d060e0b3067" exitCode=0
Oct 30 19:32:04.833756 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.833616    6996 generic.go:356] "Generic (PLEG): container finished" podID="79ea5164-32e2-4505-b92a-1e50495a6492" containerID="1e81737bca963e643ff03b6a198cde56fcd2c6976197077e33e4b6d2266b1b83" exitCode=0
Oct 30 19:32:04.833756 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.833620    6996 generic.go:356] "Generic (PLEG): container finished" podID="79ea5164-32e2-4505-b92a-1e50495a6492" containerID="5f9af21001b3bd286e77635d95130794e36b2aedad876c3f3206c4b306002b04" exitCode=0
Oct 30 19:32:04.870742 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.870722    6996 manager.go:324] Recovery completed
Oct 30 19:32:04.884245 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:32:04.884192    6996 kubelet.go:2545] "Skipping pod synchronization" err="container runtime status check may not have completed yet"
Oct 30 19:32:04.893667 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.893641    6996 cpu_manager.go:226] "Starting" policy="none"
Oct 30 19:32:04.893667 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.893664    6996 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 19:32:04.893829 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.893685    6996 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 19:32:04.893878 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.893863    6996 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Oct 30 19:32:04.893920 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.893878    6996 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Oct 30 19:32:04.893920 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.893902    6996 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Oct 30 19:32:04.893920 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.893910    6996 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Oct 30 19:32:04.893920 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.893917    6996 policy_none.go:50] "Start"
Oct 30 19:32:04.894074 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.893940    6996 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 19:32:04.894074 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.893952    6996 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 19:32:04.895139 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.895117    6996 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Oct 30 19:32:04.895139 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.895141    6996 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Oct 30 19:32:04.895270 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.895150    6996 policy_none.go:44] "Start"
Oct 30 19:32:04.900252 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.900225    6996 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 19:32:04.902736 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.902724    6996 manager.go:341] "Starting Device Plugin manager"
Oct 30 19:32:04.903264 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:32:04.902754    6996 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 19:32:04.903264 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.903265    6996 server.go:87] "Starting device plugin registration server"
Oct 30 19:32:04.903549 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.903533    6996 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 19:32:04.903645 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.903547    6996 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 19:32:04.903697 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.903638    6996 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 19:32:04.903774 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.903743    6996 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 19:32:04.903838 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.903789    6996 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 19:32:04.903838 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.903797    6996 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 19:32:04.903911 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.903845    6996 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T19:32:04.903772665Z","UUID":"769bd483-8b8c-4890-a91b-940c9a2c0b8b","Handler":null,"Name":"","Endpoint":""}
Oct 30 19:32:04.905190 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.905172    6996 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 19:32:04.905279 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:04.905196    6996 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 19:32:04.905356 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:32:04.905341    6996 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 19:32:05.004236 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.004202    6996 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 19:32:05.006133 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.006112    6996 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-42.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 19:32:05.006243 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.006143    6996 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-42.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 19:32:05.006243 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.006154    6996 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-42.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 19:32:05.006243 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.006222    6996 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-141-42.us-east-2.compute.internal"
Oct 30 19:32:05.024710 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.024687    6996 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-141-42.us-east-2.compute.internal"
Oct 30 19:32:05.024811 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.024744    6996 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-141-42.us-east-2.compute.internal"
Oct 30 19:32:05.084855 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.084815    6996 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-42.us-east-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-42.us-east-2.compute.internal"]
Oct 30 19:32:05.085371 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.085331    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-42.us-east-2.compute.internal" event={"ID":"c73221e6d3bc811e32d70f92ba61e280","Type":"ContainerStarted","Data":"47d27f3fb2ed66891a28fba8ffe3f81bc65634aad05104cba1f0678caa5b2171"}
Oct 30 19:32:05.085475 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.085381    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-42.us-east-2.compute.internal" event={"ID":"c73221e6d3bc811e32d70f92ba61e280","Type":"ContainerDied","Data":"fe5d517e6465dec695facc2299abdf8f8b576c4000b73562db9ae5441a2c5b1c"}
Oct 30 19:32:05.085475 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.085399    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-42.us-east-2.compute.internal" event={"ID":"c73221e6d3bc811e32d70f92ba61e280","Type":"ContainerStarted","Data":"4134771bb05a59b201e34b75eb0925ec7b9e192de9248a9f0161110e6dc89a32"}
Oct 30 19:32:05.085475 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.085466    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-42.us-east-2.compute.internal" event={"ID":"cb07204d0709daabd91966727e3665a5","Type":"ContainerStarted","Data":"5a7d3c6d6b1a4185e00b14684f86ae2d2b8ffb7a2b04deb308868529763314df"}
Oct 30 19:32:05.085593 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.085498    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-42.us-east-2.compute.internal" event={"ID":"cb07204d0709daabd91966727e3665a5","Type":"ContainerStarted","Data":"df0e67a4af24841460982f934d29e2f26264261184ee6a69067718ae926c739f"}
Oct 30 19:32:05.085593 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.085506    6996 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-42.us-east-2.compute.internal"
Oct 30 19:32:05.085593 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.085551    6996 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-42.us-east-2.compute.internal"
Oct 30 19:32:05.096248 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.096230    6996 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 19:32:05.096289 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:32:05.096270    6996 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-141-42.us-east-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-42.us-east-2.compute.internal"
Oct 30 19:32:05.097231 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.097214    6996 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 19:32:05.097265 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:32:05.097250    6996 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-141-42.us-east-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-42.us-east-2.compute.internal"
Oct 30 19:32:05.169596 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.169540    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c73221e6d3bc811e32d70f92ba61e280-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-42.us-east-2.compute.internal\" (UID: \"c73221e6d3bc811e32d70f92ba61e280\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-42.us-east-2.compute.internal"
Oct 30 19:32:05.169596 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.169570    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c73221e6d3bc811e32d70f92ba61e280-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-42.us-east-2.compute.internal\" (UID: \"c73221e6d3bc811e32d70f92ba61e280\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-42.us-east-2.compute.internal"
Oct 30 19:32:05.169705 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.169615    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/cb07204d0709daabd91966727e3665a5-config\") pod \"kube-apiserver-proxy-ip-10-0-141-42.us-east-2.compute.internal\" (UID: \"cb07204d0709daabd91966727e3665a5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-42.us-east-2.compute.internal"
Oct 30 19:32:05.270164 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.270138    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c73221e6d3bc811e32d70f92ba61e280-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-42.us-east-2.compute.internal\" (UID: \"c73221e6d3bc811e32d70f92ba61e280\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-42.us-east-2.compute.internal"
Oct 30 19:32:05.270278 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.270168    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c73221e6d3bc811e32d70f92ba61e280-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-42.us-east-2.compute.internal\" (UID: \"c73221e6d3bc811e32d70f92ba61e280\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-42.us-east-2.compute.internal"
Oct 30 19:32:05.270278 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.270190    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/cb07204d0709daabd91966727e3665a5-config\") pod \"kube-apiserver-proxy-ip-10-0-141-42.us-east-2.compute.internal\" (UID: \"cb07204d0709daabd91966727e3665a5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-42.us-east-2.compute.internal"
Oct 30 19:32:05.270278 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.270242    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/cb07204d0709daabd91966727e3665a5-config\") pod \"kube-apiserver-proxy-ip-10-0-141-42.us-east-2.compute.internal\" (UID: \"cb07204d0709daabd91966727e3665a5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-42.us-east-2.compute.internal"
Oct 30 19:32:05.270278 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.270244    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c73221e6d3bc811e32d70f92ba61e280-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-42.us-east-2.compute.internal\" (UID: \"c73221e6d3bc811e32d70f92ba61e280\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-42.us-east-2.compute.internal"
Oct 30 19:32:05.270278 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.270242    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c73221e6d3bc811e32d70f92ba61e280-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-42.us-east-2.compute.internal\" (UID: \"c73221e6d3bc811e32d70f92ba61e280\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-42.us-east-2.compute.internal"
Oct 30 19:32:05.737949 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.737916    6996 apiserver.go:52] "Watching apiserver"
Oct 30 19:32:05.746212 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.746189    6996 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 19:32:05.746905 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.746877    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-859jf","openshift-ovn-kubernetes/ovnkube-node-74lps","openshift-insights/insights-runtime-extractor-qdkpz","openshift-multus/multus-97df4","openshift-multus/multus-additional-cni-plugins-9n92x","kube-system/global-pull-secret-syncer-526rb","kube-system/kube-apiserver-proxy-ip-10-0-141-42.us-east-2.compute.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hts7f","openshift-monitoring/monitoring-plugin-77f4f76594-ph577","openshift-image-registry/node-ca-j4cdl","openshift-cluster-node-tuning-operator/tuned-g4c6r","openshift-dns/dns-default-8dn55","openshift-monitoring/node-exporter-sx8t4","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-42.us-east-2.compute.internal","openshift-monitoring/prometheus-k8s-0","openshift-console/console-55994d7c46-sz78s","openshift-monitoring/metrics-server-5cb8677769-45w2n","openshift-multus/network-metrics-daemon-vlqp7","openshift-monitoring/openshift-state-metrics-7bd7489957-nq89t","openshift-monitoring/telemeter-client-6d56c5d745-9xkj5","openshift-monitoring/thanos-querier-996bc4dbd-wr65k","openshift-network-diagnostics/network-check-target-4f5sl","openshift-network-operator/iptables-alerter-5jq48","openshift-console/console-7c4f845644-d8mc5","openshift-console/console-6479499c8f-ptgvc","openshift-ingress-canary/ingress-canary-8hjtk","kube-system/konnectivity-agent-bx5vk"]
Oct 30 19:32:05.755245 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.755210    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 19:32:05.755472 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.755454    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 19:32:05.756420 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.756394    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 19:32:05.756530 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.756426    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 19:32:05.756530 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.756427    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-wr97x\""
Oct 30 19:32:05.756530 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.756471    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 19:32:05.756530 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.756518    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 19:32:05.756723 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.756564    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 19:32:05.756723 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.756569    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 19:32:05.756723 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.756588    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-jznq8\""
Oct 30 19:32:05.756723 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.756602    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 19:32:05.756723 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.756628    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 19:32:05.756723 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.756393    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 19:32:05.756723 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.756666    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 19:32:05.756723 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.756694    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-ztltl\""
Oct 30 19:32:05.757071 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.756746    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-5r9wt\""
Oct 30 19:32:05.757071 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.756750    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-vcqml\""
Oct 30 19:32:05.757071 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.756810    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 19:32:05.757071 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.756826    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 19:32:05.757071 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.756856    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 19:32:05.757071 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.756885    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 19:32:05.757071 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.756927    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 19:32:05.757071 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.756953    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 19:32:05.757071 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.756958    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-rpkw7\""
Oct 30 19:32:05.757071 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.756994    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-x9qfs\""
Oct 30 19:32:05.757071 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.757037    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 19:32:05.757071 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.757052    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 19:32:05.757567 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.757097    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-4mthb\""
Oct 30 19:32:05.757567 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.757140    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 19:32:05.757567 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.757186    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 19:32:05.757567 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.757297    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 19:32:05.757567 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.757302    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 19:32:05.757567 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.757387    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 19:32:05.757567 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.757389    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-g4c9f\""
Oct 30 19:32:05.757567 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.757467    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 19:32:05.757900 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.757572    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-7mr7r\""
Oct 30 19:32:05.757900 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.757652    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 19:32:05.757900 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.757738    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 19:32:05.757900 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.757850    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-7c6bv\""
Oct 30 19:32:05.758064 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.757981    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 19:32:05.758064 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.758048    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 19:32:05.758167 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.758150    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 19:32:05.758239 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.758219    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 19:32:05.758343 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.758331    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-79gww\""
Oct 30 19:32:05.758424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.758412    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 19:32:05.758507 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.758494    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-h46gl\""
Oct 30 19:32:05.759912 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.759894    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 19:32:05.760011 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.759938    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 19:32:05.760139 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.760125    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 19:32:05.760248 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.760234    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-scxl6\""
Oct 30 19:32:05.760848 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.760829    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 19:32:05.761466 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.761446    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-f2mgm\""
Oct 30 19:32:05.761615 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.761597    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 19:32:05.761679 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.761604    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 19:32:05.761679 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.761633    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 19:32:05.761875 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.761861    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-vzztc\""
Oct 30 19:32:05.761956 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.761931    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 30 19:32:05.762858 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.762837    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 30 19:32:05.772041 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.772018    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d0036363-bf0e-44d7-b285-44546ac729e6-sys-fs\") pod \"aws-ebs-csi-driver-node-hts7f\" (UID: \"d0036363-bf0e-44d7-b285-44546ac729e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hts7f"
Oct 30 19:32:05.772146 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.772060    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7bb33e32-f8ac-44a2-95a4-cb949b08710f-metrics-tls\") pod \"dns-default-8dn55\" (UID: \"7bb33e32-f8ac-44a2-95a4-cb949b08710f\") " pod="openshift-dns/dns-default-8dn55"
Oct 30 19:32:05.772146 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.772087    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/559fa506-ed9a-4c61-9316-edc85d6c5399-data-volume\") pod \"insights-runtime-extractor-qdkpz\" (UID: \"559fa506-ed9a-4c61-9316-edc85d6c5399\") " pod="openshift-insights/insights-runtime-extractor-qdkpz"
Oct 30 19:32:05.772146 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.772125    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/559fa506-ed9a-4c61-9316-edc85d6c5399-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qdkpz\" (UID: \"559fa506-ed9a-4c61-9316-edc85d6c5399\") " pod="openshift-insights/insights-runtime-extractor-qdkpz"
Oct 30 19:32:05.772247 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.772196    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-host-var-lib-cni-multus\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:32:05.772247 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.772221    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-multus-conf-dir\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:32:05.772333 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.772264    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fs44x\" (UniqueName: \"kubernetes.io/projected/82384cc1-894f-4bb6-9b1d-567c72004575-kube-api-access-fs44x\") pod \"network-metrics-daemon-vlqp7\" (UID: \"82384cc1-894f-4bb6-9b1d-567c72004575\") " pod="openshift-multus/network-metrics-daemon-vlqp7"
Oct 30 19:32:05.772333 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.772274    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/559fa506-ed9a-4c61-9316-edc85d6c5399-data-volume\") pod \"insights-runtime-extractor-qdkpz\" (UID: \"559fa506-ed9a-4c61-9316-edc85d6c5399\") " pod="openshift-insights/insights-runtime-extractor-qdkpz"
Oct 30 19:32:05.772333 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.772287    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-etc-modprobe-d\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:32:05.772465 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.772336    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/559fa506-ed9a-4c61-9316-edc85d6c5399-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qdkpz\" (UID: \"559fa506-ed9a-4c61-9316-edc85d6c5399\") " pod="openshift-insights/insights-runtime-extractor-qdkpz"
Oct 30 19:32:05.772465 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.772345    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-etc-tuned\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:32:05.772465 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.772362    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7bb33e32-f8ac-44a2-95a4-cb949b08710f-metrics-tls\") pod \"dns-default-8dn55\" (UID: \"7bb33e32-f8ac-44a2-95a4-cb949b08710f\") " pod="openshift-dns/dns-default-8dn55"
Oct 30 19:32:05.772465 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.772385    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7bb33e32-f8ac-44a2-95a4-cb949b08710f-config-volume\") pod \"dns-default-8dn55\" (UID: \"7bb33e32-f8ac-44a2-95a4-cb949b08710f\") " pod="openshift-dns/dns-default-8dn55"
Oct 30 19:32:05.772465 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.772418    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/559fa506-ed9a-4c61-9316-edc85d6c5399-crio-socket\") pod \"insights-runtime-extractor-qdkpz\" (UID: \"559fa506-ed9a-4c61-9316-edc85d6c5399\") " pod="openshift-insights/insights-runtime-extractor-qdkpz"
Oct 30 19:32:05.772465 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.772447    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-host-slash\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:32:05.772465 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.772453    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-etc-tuned\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:32:05.772777 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.772469    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-host-cni-netd\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:32:05.772777 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.772527    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-multus-cni-dir\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:32:05.772777 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.772563    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-hostroot\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:32:05.772777 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.772575    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7bb33e32-f8ac-44a2-95a4-cb949b08710f-config-volume\") pod \"dns-default-8dn55\" (UID: \"7bb33e32-f8ac-44a2-95a4-cb949b08710f\") " pod="openshift-dns/dns-default-8dn55"
Oct 30 19:32:05.772777 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.772602    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mwb6\" (UniqueName: \"kubernetes.io/projected/63678441-271f-46d7-ac1d-57a60bb78b12-kube-api-access-9mwb6\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:32:05.772777 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.772623    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-host-kubelet\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:32:05.772777 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.772638    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d4d142b9-22f1-427f-a374-ba4918487946-ovnkube-script-lib\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:32:05.772777 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.772657    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/79ea5164-32e2-4505-b92a-1e50495a6492-system-cni-dir\") pod \"multus-additional-cni-plugins-9n92x\" (UID: \"79ea5164-32e2-4505-b92a-1e50495a6492\") " pod="openshift-multus/multus-additional-cni-plugins-9n92x"
Oct 30 19:32:05.772777 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.772690    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n87xw\" (UniqueName: \"kubernetes.io/projected/79ea5164-32e2-4505-b92a-1e50495a6492-kube-api-access-n87xw\") pod \"multus-additional-cni-plugins-9n92x\" (UID: \"79ea5164-32e2-4505-b92a-1e50495a6492\") " pod="openshift-multus/multus-additional-cni-plugins-9n92x"
Oct 30 19:32:05.772777 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.772706    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1977f3d7-c7f6-44cc-8a59-9d4f35adc0f7-hosts-file\") pod \"node-resolver-859jf\" (UID: \"1977f3d7-c7f6-44cc-8a59-9d4f35adc0f7\") " pod="openshift-dns/node-resolver-859jf"
Oct 30 19:32:05.772777 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.772725    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/79ea5164-32e2-4505-b92a-1e50495a6492-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9n92x\" (UID: \"79ea5164-32e2-4505-b92a-1e50495a6492\") " pod="openshift-multus/multus-additional-cni-plugins-9n92x"
Oct 30 19:32:05.772777 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.772743    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-host-var-lib-cni-bin\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:32:05.772777 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.772775    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1977f3d7-c7f6-44cc-8a59-9d4f35adc0f7-tmp-dir\") pod \"node-resolver-859jf\" (UID: \"1977f3d7-c7f6-44cc-8a59-9d4f35adc0f7\") " pod="openshift-dns/node-resolver-859jf"
Oct 30 19:32:05.773327 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.772793    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-tmp\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:32:05.773327 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.772815    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bh7cj\" (UniqueName: \"kubernetes.io/projected/559fa506-ed9a-4c61-9316-edc85d6c5399-kube-api-access-bh7cj\") pod \"insights-runtime-extractor-qdkpz\" (UID: \"559fa506-ed9a-4c61-9316-edc85d6c5399\") " pod="openshift-insights/insights-runtime-extractor-qdkpz"
Oct 30 19:32:05.773327 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.772892    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-cnibin\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:32:05.773327 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.772909    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-var-lib-openvswitch\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:32:05.773327 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.772928    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-etc-openvswitch\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:32:05.773327 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.772938    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1977f3d7-c7f6-44cc-8a59-9d4f35adc0f7-tmp-dir\") pod \"node-resolver-859jf\" (UID: \"1977f3d7-c7f6-44cc-8a59-9d4f35adc0f7\") " pod="openshift-dns/node-resolver-859jf"
Oct 30 19:32:05.773327 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.772993    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d4d142b9-22f1-427f-a374-ba4918487946-ovnkube-script-lib\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:32:05.773327 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.773067    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d4d142b9-22f1-427f-a374-ba4918487946-env-overrides\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:32:05.773327 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.773100    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/517d6b00-0305-40cc-95c3-bba4adb2f849-cert\") pod \"ingress-canary-8hjtk\" (UID: \"517d6b00-0305-40cc-95c3-bba4adb2f849\") " pod="openshift-ingress-canary/ingress-canary-8hjtk"
Oct 30 19:32:05.773327 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.773139    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/82384cc1-894f-4bb6-9b1d-567c72004575-metrics-certs\") pod \"network-metrics-daemon-vlqp7\" (UID: \"82384cc1-894f-4bb6-9b1d-567c72004575\") " pod="openshift-multus/network-metrics-daemon-vlqp7"
Oct 30 19:32:05.773327 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.773165    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-etc-systemd\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:32:05.773327 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.773168    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-tmp\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:32:05.773327 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.773205    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-host-run-netns\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:32:05.773327 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.773293    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d4d142b9-22f1-427f-a374-ba4918487946-env-overrides\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:32:05.773838 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.773361    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-node-log\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:32:05.773838 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.773450    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gt5qd\" (UniqueName: \"kubernetes.io/projected/d4d142b9-22f1-427f-a374-ba4918487946-kube-api-access-gt5qd\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:32:05.773838 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.773521    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-multus-socket-dir-parent\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:32:05.773838 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.773563    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6eca1a1e-101a-43bb-9fde-505c3182c10a-serviceca\") pod \"node-ca-j4cdl\" (UID: \"6eca1a1e-101a-43bb-9fde-505c3182c10a\") " pod="openshift-image-registry/node-ca-j4cdl"
Oct 30 19:32:05.773838 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.773613    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/17f17805-edc3-4dc5-8e75-819350666927-konnectivity-ca\") pod \"konnectivity-agent-bx5vk\" (UID: \"17f17805-edc3-4dc5-8e75-819350666927\") " pod="kube-system/konnectivity-agent-bx5vk"
Oct 30 19:32:05.773838 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.773620    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/82384cc1-894f-4bb6-9b1d-567c72004575-metrics-certs\") pod \"network-metrics-daemon-vlqp7\" (UID: \"82384cc1-894f-4bb6-9b1d-567c72004575\") " pod="openshift-multus/network-metrics-daemon-vlqp7"
Oct 30 19:32:05.773838 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.773650    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/517d6b00-0305-40cc-95c3-bba4adb2f849-cert\") pod \"ingress-canary-8hjtk\" (UID: \"517d6b00-0305-40cc-95c3-bba4adb2f849\") " pod="openshift-ingress-canary/ingress-canary-8hjtk"
Oct 30 19:32:05.773838 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.773672    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-system-cni-dir\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:32:05.773838 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.773724    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/63678441-271f-46d7-ac1d-57a60bb78b12-cni-binary-copy\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:32:05.774125 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.773927    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-host-run-netns\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:32:05.774125 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.773970    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-run-systemd\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:32:05.774125 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.773997    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nn6bt\" (UniqueName: \"kubernetes.io/projected/7ff4ab79-1d7c-49ac-9d63-31772ad68b42-kube-api-access-nn6bt\") pod \"network-check-target-4f5sl\" (UID: \"7ff4ab79-1d7c-49ac-9d63-31772ad68b42\") " pod="openshift-network-diagnostics/network-check-target-4f5sl"
Oct 30 19:32:05.774125 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.774031    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-etc-sysconfig\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:32:05.774125 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.774054    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-run\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:32:05.774125 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.774101    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-var-lib-kubelet\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:32:05.774125 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.774124    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-host-run-k8s-cni-cncf-io\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:32:05.774447 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.774167    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bls4\" (UniqueName: \"kubernetes.io/projected/6eca1a1e-101a-43bb-9fde-505c3182c10a-kube-api-access-8bls4\") pod \"node-ca-j4cdl\" (UID: \"6eca1a1e-101a-43bb-9fde-505c3182c10a\") " pod="openshift-image-registry/node-ca-j4cdl"
Oct 30 19:32:05.774447 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.774193    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxjvg\" (UniqueName: \"kubernetes.io/projected/d0036363-bf0e-44d7-b285-44546ac729e6-kube-api-access-pxjvg\") pod \"aws-ebs-csi-driver-node-hts7f\" (UID: \"d0036363-bf0e-44d7-b285-44546ac729e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hts7f"
Oct 30 19:32:05.774447 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.774215    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/63678441-271f-46d7-ac1d-57a60bb78b12-cni-binary-copy\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:32:05.774447 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.774217    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-etc-sysctl-conf\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:32:05.774447 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.774245    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-sys\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:32:05.774447 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.774261    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-host\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:32:05.774447 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.774265    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/17f17805-edc3-4dc5-8e75-819350666927-konnectivity-ca\") pod \"konnectivity-agent-bx5vk\" (UID: \"17f17805-edc3-4dc5-8e75-819350666927\") " pod="kube-system/konnectivity-agent-bx5vk"
Oct 30 19:32:05.774447 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.774308    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhsf4\" (UniqueName: \"kubernetes.io/projected/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-kube-api-access-mhsf4\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:32:05.774447 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.774325    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-host-run-multus-certs\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:32:05.774447 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.774341    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-log-socket\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:32:05.774447 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.774375    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/79ea5164-32e2-4505-b92a-1e50495a6492-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9n92x\" (UID: \"79ea5164-32e2-4505-b92a-1e50495a6492\") " pod="openshift-multus/multus-additional-cni-plugins-9n92x"
Oct 30 19:32:05.774447 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.774399    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/141cf3eb-d99b-4876-b65a-a2847584ff5d-original-pull-secret\") pod \"global-pull-secret-syncer-526rb\" (UID: \"141cf3eb-d99b-4876-b65a-a2847584ff5d\") " pod="kube-system/global-pull-secret-syncer-526rb"
Oct 30 19:32:05.774447 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.774415    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d0036363-bf0e-44d7-b285-44546ac729e6-registration-dir\") pod \"aws-ebs-csi-driver-node-hts7f\" (UID: \"d0036363-bf0e-44d7-b285-44546ac729e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hts7f"
Oct 30 19:32:05.774447 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.774433    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-etc-kubernetes\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:32:05.775060 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.774473    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6eca1a1e-101a-43bb-9fde-505c3182c10a-host\") pod \"node-ca-j4cdl\" (UID: \"6eca1a1e-101a-43bb-9fde-505c3182c10a\") " pod="openshift-image-registry/node-ca-j4cdl"
Oct 30 19:32:05.775060 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.774516    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d4d142b9-22f1-427f-a374-ba4918487946-ovn-node-metrics-cert\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:32:05.775060 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.774555    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7bb33e32-f8ac-44a2-95a4-cb949b08710f-tmp-dir\") pod \"dns-default-8dn55\" (UID: \"7bb33e32-f8ac-44a2-95a4-cb949b08710f\") " pod="openshift-dns/dns-default-8dn55"
Oct 30 19:32:05.775060 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.774598    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/79ea5164-32e2-4505-b92a-1e50495a6492-cni-binary-copy\") pod \"multus-additional-cni-plugins-9n92x\" (UID: \"79ea5164-32e2-4505-b92a-1e50495a6492\") " pod="openshift-multus/multus-additional-cni-plugins-9n92x"
Oct 30 19:32:05.775060 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.774657    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6eca1a1e-101a-43bb-9fde-505c3182c10a-serviceca\") pod \"node-ca-j4cdl\" (UID: \"6eca1a1e-101a-43bb-9fde-505c3182c10a\") " pod="openshift-image-registry/node-ca-j4cdl"
Oct 30 19:32:05.775060 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.774668    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7bb33e32-f8ac-44a2-95a4-cb949b08710f-tmp-dir\") pod \"dns-default-8dn55\" (UID: \"7bb33e32-f8ac-44a2-95a4-cb949b08710f\") " pod="openshift-dns/dns-default-8dn55"
Oct 30 19:32:05.775060 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.774708    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-systemd-units\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:32:05.775060 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.774734    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-run-openvswitch\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:32:05.775060 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.774746    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/79ea5164-32e2-4505-b92a-1e50495a6492-cni-binary-copy\") pod \"multus-additional-cni-plugins-9n92x\" (UID: \"79ea5164-32e2-4505-b92a-1e50495a6492\") " pod="openshift-multus/multus-additional-cni-plugins-9n92x"
Oct 30 19:32:05.775060 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.774776    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-run-ovn\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:32:05.775060 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.774827    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-host-cni-bin\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:32:05.775060 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.774849    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/141cf3eb-d99b-4876-b65a-a2847584ff5d-original-pull-secret\") pod \"global-pull-secret-syncer-526rb\" (UID: \"141cf3eb-d99b-4876-b65a-a2847584ff5d\") " pod="kube-system/global-pull-secret-syncer-526rb"
Oct 30 19:32:05.775060 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.774853    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/79ea5164-32e2-4505-b92a-1e50495a6492-cnibin\") pod \"multus-additional-cni-plugins-9n92x\" (UID: \"79ea5164-32e2-4505-b92a-1e50495a6492\") " pod="openshift-multus/multus-additional-cni-plugins-9n92x"
Oct 30 19:32:05.775060 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.774913    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4x29s\" (UniqueName: \"kubernetes.io/projected/1977f3d7-c7f6-44cc-8a59-9d4f35adc0f7-kube-api-access-4x29s\") pod \"node-resolver-859jf\" (UID: \"1977f3d7-c7f6-44cc-8a59-9d4f35adc0f7\") " pod="openshift-dns/node-resolver-859jf"
Oct 30 19:32:05.775060 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.774995    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/63678441-271f-46d7-ac1d-57a60bb78b12-multus-daemon-config\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:32:05.775060 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.775035    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d4d142b9-22f1-427f-a374-ba4918487946-ovn-node-metrics-cert\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:32:05.775060 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.775041    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsb6w\" (UniqueName: \"kubernetes.io/projected/0100ec89-6366-4946-99ce-4a254850c8fd-kube-api-access-gsb6w\") pod \"iptables-alerter-5jq48\" (UID: \"0100ec89-6366-4946-99ce-4a254850c8fd\") " pod="openshift-network-operator/iptables-alerter-5jq48"
Oct 30 19:32:05.775060 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.775061    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/79ea5164-32e2-4505-b92a-1e50495a6492-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9n92x\" (UID: \"79ea5164-32e2-4505-b92a-1e50495a6492\") " pod="openshift-multus/multus-additional-cni-plugins-9n92x"
Oct 30 19:32:05.775878 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.775080    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d0036363-bf0e-44d7-b285-44546ac729e6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hts7f\" (UID: \"d0036363-bf0e-44d7-b285-44546ac729e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hts7f"
Oct 30 19:32:05.775878 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.775106    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d0036363-bf0e-44d7-b285-44546ac729e6-socket-dir\") pod \"aws-ebs-csi-driver-node-hts7f\" (UID: \"d0036363-bf0e-44d7-b285-44546ac729e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hts7f"
Oct 30 19:32:05.775878 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.775139    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-etc-kubernetes\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:32:05.775878 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.775163    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-lib-modules\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:32:05.775878 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.775168    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/63678441-271f-46d7-ac1d-57a60bb78b12-multus-daemon-config\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:32:05.775878 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.775206    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-host-var-lib-kubelet\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:32:05.775878 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.775234    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0100ec89-6366-4946-99ce-4a254850c8fd-iptables-alerter-script\") pod \"iptables-alerter-5jq48\" (UID: \"0100ec89-6366-4946-99ce-4a254850c8fd\") " pod="openshift-network-operator/iptables-alerter-5jq48"
Oct 30 19:32:05.775878 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.775261    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0100ec89-6366-4946-99ce-4a254850c8fd-host-slash\") pod \"iptables-alerter-5jq48\" (UID: \"0100ec89-6366-4946-99ce-4a254850c8fd\") " pod="openshift-network-operator/iptables-alerter-5jq48"
Oct 30 19:32:05.775878 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.775380    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d4d142b9-22f1-427f-a374-ba4918487946-ovnkube-config\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:32:05.775878 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.775407    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-etc-sysctl-d\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:32:05.775878 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.775448    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/559fa506-ed9a-4c61-9316-edc85d6c5399-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qdkpz\" (UID: \"559fa506-ed9a-4c61-9316-edc85d6c5399\") " pod="openshift-insights/insights-runtime-extractor-qdkpz"
Oct 30 19:32:05.775878 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.775605    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0100ec89-6366-4946-99ce-4a254850c8fd-iptables-alerter-script\") pod \"iptables-alerter-5jq48\" (UID: \"0100ec89-6366-4946-99ce-4a254850c8fd\") " pod="openshift-network-operator/iptables-alerter-5jq48"
Oct 30 19:32:05.775878 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.775660    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d4d142b9-22f1-427f-a374-ba4918487946-ovnkube-config\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:32:05.775878 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.775699    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/17f17805-edc3-4dc5-8e75-819350666927-agent-certs\") pod \"konnectivity-agent-bx5vk\" (UID: \"17f17805-edc3-4dc5-8e75-819350666927\") " pod="kube-system/konnectivity-agent-bx5vk"
Oct 30 19:32:05.775878 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.775753    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/79ea5164-32e2-4505-b92a-1e50495a6492-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9n92x\" (UID: \"79ea5164-32e2-4505-b92a-1e50495a6492\") " pod="openshift-multus/multus-additional-cni-plugins-9n92x"
Oct 30 19:32:05.775878 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.775780    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/141cf3eb-d99b-4876-b65a-a2847584ff5d-kubelet-config\") pod \"global-pull-secret-syncer-526rb\" (UID: \"141cf3eb-d99b-4876-b65a-a2847584ff5d\") " pod="kube-system/global-pull-secret-syncer-526rb"
Oct 30 19:32:05.775878 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.775806    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/559fa506-ed9a-4c61-9316-edc85d6c5399-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qdkpz\" (UID: \"559fa506-ed9a-4c61-9316-edc85d6c5399\") " pod="openshift-insights/insights-runtime-extractor-qdkpz"
Oct 30 19:32:05.775878 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.775839    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/141cf3eb-d99b-4876-b65a-a2847584ff5d-dbus\") pod \"global-pull-secret-syncer-526rb\" (UID: \"141cf3eb-d99b-4876-b65a-a2847584ff5d\") " pod="kube-system/global-pull-secret-syncer-526rb"
Oct 30 19:32:05.775878 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.775868    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d0036363-bf0e-44d7-b285-44546ac729e6-device-dir\") pod \"aws-ebs-csi-driver-node-hts7f\" (UID: \"d0036363-bf0e-44d7-b285-44546ac729e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hts7f"
Oct 30 19:32:05.776729 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.775917    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d0036363-bf0e-44d7-b285-44546ac729e6-etc-selinux\") pod \"aws-ebs-csi-driver-node-hts7f\" (UID: \"d0036363-bf0e-44d7-b285-44546ac729e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hts7f"
Oct 30 19:32:05.776729 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.775946    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jx8qx\" (UniqueName: \"kubernetes.io/projected/7bb33e32-f8ac-44a2-95a4-cb949b08710f-kube-api-access-jx8qx\") pod \"dns-default-8dn55\" (UID: \"7bb33e32-f8ac-44a2-95a4-cb949b08710f\") " pod="openshift-dns/dns-default-8dn55"
Oct 30 19:32:05.776729 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.775975    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-host-run-ovn-kubernetes\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:32:05.776729 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.776018    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/17f17805-edc3-4dc5-8e75-819350666927-agent-certs\") pod \"konnectivity-agent-bx5vk\" (UID: \"17f17805-edc3-4dc5-8e75-819350666927\") " pod="kube-system/konnectivity-agent-bx5vk"
Oct 30 19:32:05.776729 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.776034    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:32:05.776729 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.776064    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tv2lm\" (UniqueName: \"kubernetes.io/projected/517d6b00-0305-40cc-95c3-bba4adb2f849-kube-api-access-tv2lm\") pod \"ingress-canary-8hjtk\" (UID: \"517d6b00-0305-40cc-95c3-bba4adb2f849\") " pod="openshift-ingress-canary/ingress-canary-8hjtk"
Oct 30 19:32:05.776729 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.776162    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/79ea5164-32e2-4505-b92a-1e50495a6492-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9n92x\" (UID: \"79ea5164-32e2-4505-b92a-1e50495a6492\") " pod="openshift-multus/multus-additional-cni-plugins-9n92x"
Oct 30 19:32:05.798552 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.798526    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bh7cj\" (UniqueName: \"kubernetes.io/projected/559fa506-ed9a-4c61-9316-edc85d6c5399-kube-api-access-bh7cj\") pod \"insights-runtime-extractor-qdkpz\" (UID: \"559fa506-ed9a-4c61-9316-edc85d6c5399\") " pod="openshift-insights/insights-runtime-extractor-qdkpz"
Oct 30 19:32:05.798657 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.798526    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fs44x\" (UniqueName: \"kubernetes.io/projected/82384cc1-894f-4bb6-9b1d-567c72004575-kube-api-access-fs44x\") pod \"network-metrics-daemon-vlqp7\" (UID: \"82384cc1-894f-4bb6-9b1d-567c72004575\") " pod="openshift-multus/network-metrics-daemon-vlqp7"
Oct 30 19:32:05.799027 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.799010    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n87xw\" (UniqueName: \"kubernetes.io/projected/79ea5164-32e2-4505-b92a-1e50495a6492-kube-api-access-n87xw\") pod \"multus-additional-cni-plugins-9n92x\" (UID: \"79ea5164-32e2-4505-b92a-1e50495a6492\") " pod="openshift-multus/multus-additional-cni-plugins-9n92x"
Oct 30 19:32:05.800317 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.800300    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mwb6\" (UniqueName: \"kubernetes.io/projected/63678441-271f-46d7-ac1d-57a60bb78b12-kube-api-access-9mwb6\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:32:05.802983 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.802965    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gt5qd\" (UniqueName: \"kubernetes.io/projected/d4d142b9-22f1-427f-a374-ba4918487946-kube-api-access-gt5qd\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:32:05.804700 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.804674    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxjvg\" (UniqueName: \"kubernetes.io/projected/d0036363-bf0e-44d7-b285-44546ac729e6-kube-api-access-pxjvg\") pod \"aws-ebs-csi-driver-node-hts7f\" (UID: \"d0036363-bf0e-44d7-b285-44546ac729e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hts7f"
Oct 30 19:32:05.804874 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.804852    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gsb6w\" (UniqueName: \"kubernetes.io/projected/0100ec89-6366-4946-99ce-4a254850c8fd-kube-api-access-gsb6w\") pod \"iptables-alerter-5jq48\" (UID: \"0100ec89-6366-4946-99ce-4a254850c8fd\") " pod="openshift-network-operator/iptables-alerter-5jq48"
Oct 30 19:32:05.805052 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.805029    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bls4\" (UniqueName: \"kubernetes.io/projected/6eca1a1e-101a-43bb-9fde-505c3182c10a-kube-api-access-8bls4\") pod \"node-ca-j4cdl\" (UID: \"6eca1a1e-101a-43bb-9fde-505c3182c10a\") " pod="openshift-image-registry/node-ca-j4cdl"
Oct 30 19:32:05.805847 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.805822    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhsf4\" (UniqueName: \"kubernetes.io/projected/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-kube-api-access-mhsf4\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:32:05.805933 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.805899    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4x29s\" (UniqueName: \"kubernetes.io/projected/1977f3d7-c7f6-44cc-8a59-9d4f35adc0f7-kube-api-access-4x29s\") pod \"node-resolver-859jf\" (UID: \"1977f3d7-c7f6-44cc-8a59-9d4f35adc0f7\") " pod="openshift-dns/node-resolver-859jf"
Oct 30 19:32:05.806253 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.806230    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nn6bt\" (UniqueName: \"kubernetes.io/projected/7ff4ab79-1d7c-49ac-9d63-31772ad68b42-kube-api-access-nn6bt\") pod \"network-check-target-4f5sl\" (UID: \"7ff4ab79-1d7c-49ac-9d63-31772ad68b42\") " pod="openshift-network-diagnostics/network-check-target-4f5sl"
Oct 30 19:32:05.806421 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.806398    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jx8qx\" (UniqueName: \"kubernetes.io/projected/7bb33e32-f8ac-44a2-95a4-cb949b08710f-kube-api-access-jx8qx\") pod \"dns-default-8dn55\" (UID: \"7bb33e32-f8ac-44a2-95a4-cb949b08710f\") " pod="openshift-dns/dns-default-8dn55"
Oct 30 19:32:05.806594 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.806574    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tv2lm\" (UniqueName: \"kubernetes.io/projected/517d6b00-0305-40cc-95c3-bba4adb2f849-kube-api-access-tv2lm\") pod \"ingress-canary-8hjtk\" (UID: \"517d6b00-0305-40cc-95c3-bba4adb2f849\") " pod="openshift-ingress-canary/ingress-canary-8hjtk"
Oct 30 19:32:05.852132 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.852101    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 30 19:32:05.852302 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.852208    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Oct 30 19:32:05.852379 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.852320    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 19:32:05.852379 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.852321    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Oct 30 19:32:05.852515 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.852378    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Oct 30 19:32:05.852515 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.852395    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Oct 30 19:32:05.852515 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.852438    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Oct 30 19:32:05.852695 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.852600    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Oct 30 19:32:05.852695 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.852621    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Oct 30 19:32:05.852695 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.852657    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Oct 30 19:32:05.852695 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.852696    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Oct 30 19:32:05.852903 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.852708    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Oct 30 19:32:05.852903 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.852736    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Oct 30 19:32:05.852903 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.852897    6996 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-42.us-east-2.compute.internal"
Oct 30 19:32:05.853033 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.852923    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 30 19:32:05.853033 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.853005    6996 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-42.us-east-2.compute.internal"
Oct 30 19:32:05.854194 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.854175    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Oct 30 19:32:05.854320 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.854305    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Oct 30 19:32:05.854386 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.854308    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Oct 30 19:32:05.854764 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.854745    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Oct 30 19:32:05.854848 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.854790    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Oct 30 19:32:05.854848 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.854810    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Oct 30 19:32:05.854964 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.854874    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Oct 30 19:32:05.854964 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.854911    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Oct 30 19:32:05.855049 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.855035    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Oct 30 19:32:05.855211 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.855200    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Oct 30 19:32:05.855378 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.855364    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Oct 30 19:32:05.855439 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.855376    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Oct 30 19:32:05.855661 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.855613    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Oct 30 19:32:05.855661 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.855632    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Oct 30 19:32:05.855773 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.855742    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-z26kr\""
Oct 30 19:32:05.855773 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.855764    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-md7hj\""
Oct 30 19:32:05.855887 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.855790    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Oct 30 19:32:05.855887 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.855792    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-55h96gg718jqk\""
Oct 30 19:32:05.855986 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.855913    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Oct 30 19:32:05.855986 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.855915    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Oct 30 19:32:05.855986 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.855930    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Oct 30 19:32:05.855986 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.855960    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-j67kr\""
Oct 30 19:32:05.856135 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.856020    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Oct 30 19:32:05.856187 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.856154    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-f6ia1krs5s708\""
Oct 30 19:32:05.856233 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.856185    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 30 19:32:05.856233 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.856192    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-f6m98\""
Oct 30 19:32:05.856343 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.856236    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Oct 30 19:32:05.856343 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.856245    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Oct 30 19:32:05.856343 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.856254    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Oct 30 19:32:05.856343 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.856272    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Oct 30 19:32:05.856343 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.856335    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-gxpqn\""
Oct 30 19:32:05.856610 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.856371    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-4l7fj\""
Oct 30 19:32:05.856610 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.856371    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Oct 30 19:32:05.856610 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.856383    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-pkmls\""
Oct 30 19:32:05.856610 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.856393    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-cgn2r6prpq3cp\""
Oct 30 19:32:05.856610 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.856464    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Oct 30 19:32:05.856610 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.856508    6996 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 19:32:05.857502 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.857469    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-28r105lbnph2c\""
Oct 30 19:32:05.857602 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.857520    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Oct 30 19:32:05.857760 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.857743    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Oct 30 19:32:05.857967 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.857953    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Oct 30 19:32:05.868671 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.868653    6996 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 19:32:05.868775 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:32:05.868699    6996 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-141-42.us-east-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-42.us-east-2.compute.internal"
Oct 30 19:32:05.869343 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.869325    6996 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 19:32:05.869443 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:32:05.869365    6996 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-141-42.us-east-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-42.us-east-2.compute.internal"
Oct 30 19:32:05.876823 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.876805    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6eca1a1e-101a-43bb-9fde-505c3182c10a-host\") pod \"node-ca-j4cdl\" (UID: \"6eca1a1e-101a-43bb-9fde-505c3182c10a\") " pod="openshift-image-registry/node-ca-j4cdl"
Oct 30 19:32:05.876945 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.876829    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1f3e7563-7109-45a5-9a77-7676a0697bd1-oauth-serving-cert\") pod \"console-6479499c8f-ptgvc\" (UID: \"1f3e7563-7109-45a5-9a77-7676a0697bd1\") " pod="openshift-console/console-6479499c8f-ptgvc"
Oct 30 19:32:05.876945 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.876847    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:05.876945 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.876862    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:05.876945 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.876882    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-run-ovn\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:32:05.876945 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.876907    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/79ea5164-32e2-4505-b92a-1e50495a6492-cnibin\") pod \"multus-additional-cni-plugins-9n92x\" (UID: \"79ea5164-32e2-4505-b92a-1e50495a6492\") " pod="openshift-multus/multus-additional-cni-plugins-9n92x"
Oct 30 19:32:05.876945 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.876925    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6eca1a1e-101a-43bb-9fde-505c3182c10a-host\") pod \"node-ca-j4cdl\" (UID: \"6eca1a1e-101a-43bb-9fde-505c3182c10a\") " pod="openshift-image-registry/node-ca-j4cdl"
Oct 30 19:32:05.877228 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.876930    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d0036363-bf0e-44d7-b285-44546ac729e6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hts7f\" (UID: \"d0036363-bf0e-44d7-b285-44546ac729e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hts7f"
Oct 30 19:32:05.877228 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.876952    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-run-ovn\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:32:05.877228 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.876955    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/79ea5164-32e2-4505-b92a-1e50495a6492-cnibin\") pod \"multus-additional-cni-plugins-9n92x\" (UID: \"79ea5164-32e2-4505-b92a-1e50495a6492\") " pod="openshift-multus/multus-additional-cni-plugins-9n92x"
Oct 30 19:32:05.877228 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.876979    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d0036363-bf0e-44d7-b285-44546ac729e6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hts7f\" (UID: \"d0036363-bf0e-44d7-b285-44546ac729e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hts7f"
Oct 30 19:32:05.877228 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.876980    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-lib-modules\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:32:05.877228 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.877021    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-service-ca\") pod \"console-7c4f845644-d8mc5\" (UID: \"c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3\") " pod="openshift-console/console-7c4f845644-d8mc5"
Oct 30 19:32:05.877228 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.877044    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdcn4\" (UniqueName: \"kubernetes.io/projected/136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb-kube-api-access-wdcn4\") pod \"metrics-server-5cb8677769-45w2n\" (UID: \"136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb\") " pod="openshift-monitoring/metrics-server-5cb8677769-45w2n"
Oct 30 19:32:05.877228 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.877063    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1f3e7563-7109-45a5-9a77-7676a0697bd1-oauth-serving-cert\") pod \"console-6479499c8f-ptgvc\" (UID: \"1f3e7563-7109-45a5-9a77-7676a0697bd1\") " pod="openshift-console/console-6479499c8f-ptgvc"
Oct 30 19:32:05.877228 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.877070    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:05.877228 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.877089    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-lib-modules\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:32:05.877228 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.877096    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-host-var-lib-kubelet\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:32:05.877228 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.877091    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:05.877228 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.877124    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9wcq\" (UniqueName: \"kubernetes.io/projected/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-kube-api-access-f9wcq\") pod \"console-7c4f845644-d8mc5\" (UID: \"c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3\") " pod="openshift-console/console-7c4f845644-d8mc5"
Oct 30 19:32:05.877228 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.877147    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd-metrics-client-ca\") pod \"node-exporter-sx8t4\" (UID: \"0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd\") " pod="openshift-monitoring/node-exporter-sx8t4"
Oct 30 19:32:05.877228 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.877169    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mqpt\" (UniqueName: \"kubernetes.io/projected/1f3e7563-7109-45a5-9a77-7676a0697bd1-kube-api-access-2mqpt\") pod \"console-6479499c8f-ptgvc\" (UID: \"1f3e7563-7109-45a5-9a77-7676a0697bd1\") " pod="openshift-console/console-6479499c8f-ptgvc"
Oct 30 19:32:05.877228 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.877189    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/06b45974-f6ef-4b3a-b354-8103d7895fcf-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:05.877228 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.877213    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/141cf3eb-d99b-4876-b65a-a2847584ff5d-kubelet-config\") pod \"global-pull-secret-syncer-526rb\" (UID: \"141cf3eb-d99b-4876-b65a-a2847584ff5d\") " pod="kube-system/global-pull-secret-syncer-526rb"
Oct 30 19:32:05.877941 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.877247    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-service-ca\") pod \"console-7c4f845644-d8mc5\" (UID: \"c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3\") " pod="openshift-console/console-7c4f845644-d8mc5"
Oct 30 19:32:05.877941 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.877254    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:05.877941 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.877274    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d0036363-bf0e-44d7-b285-44546ac729e6-device-dir\") pod \"aws-ebs-csi-driver-node-hts7f\" (UID: \"d0036363-bf0e-44d7-b285-44546ac729e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hts7f"
Oct 30 19:32:05.877941 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.877299    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d0036363-bf0e-44d7-b285-44546ac729e6-etc-selinux\") pod \"aws-ebs-csi-driver-node-hts7f\" (UID: \"d0036363-bf0e-44d7-b285-44546ac729e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hts7f"
Oct 30 19:32:05.877941 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.877331    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd-node-exporter-tls\") pod \"node-exporter-sx8t4\" (UID: \"0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd\") " pod="openshift-monitoring/node-exporter-sx8t4"
Oct 30 19:32:05.877941 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.877338    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-host-var-lib-kubelet\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:32:05.877941 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.877367    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d0036363-bf0e-44d7-b285-44546ac729e6-sys-fs\") pod \"aws-ebs-csi-driver-node-hts7f\" (UID: \"d0036363-bf0e-44d7-b285-44546ac729e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hts7f"
Oct 30 19:32:05.877941 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.877393    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/94ecba26-0809-414d-b084-8de0a693795e-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-996bc4dbd-wr65k\" (UID: \"94ecba26-0809-414d-b084-8de0a693795e\") " pod="openshift-monitoring/thanos-querier-996bc4dbd-wr65k"
Oct 30 19:32:05.877941 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.877421    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/94ecba26-0809-414d-b084-8de0a693795e-secret-grpc-tls\") pod \"thanos-querier-996bc4dbd-wr65k\" (UID: \"94ecba26-0809-414d-b084-8de0a693795e\") " pod="openshift-monitoring/thanos-querier-996bc4dbd-wr65k"
Oct 30 19:32:05.877941 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.877426    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd-metrics-client-ca\") pod \"node-exporter-sx8t4\" (UID: \"0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd\") " pod="openshift-monitoring/node-exporter-sx8t4"
Oct 30 19:32:05.877941 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.877444    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmz8l\" (UniqueName: \"kubernetes.io/projected/1cc9603f-1c61-43e4-b987-0e185cf26252-kube-api-access-bmz8l\") pod \"monitoring-plugin-77f4f76594-ph577\" (UID: \"1cc9603f-1c61-43e4-b987-0e185cf26252\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-ph577"
Oct 30 19:32:05.878448 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.878053    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8cde7f30-1d5b-4bea-ad21-2264b567ca1d-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-nq89t\" (UID: \"8cde7f30-1d5b-4bea-ad21-2264b567ca1d\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-nq89t"
Oct 30 19:32:05.878448 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.878175    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d0036363-bf0e-44d7-b285-44546ac729e6-device-dir\") pod \"aws-ebs-csi-driver-node-hts7f\" (UID: \"d0036363-bf0e-44d7-b285-44546ac729e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hts7f"
Oct 30 19:32:05.878448 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.878217    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d0036363-bf0e-44d7-b285-44546ac729e6-etc-selinux\") pod \"aws-ebs-csi-driver-node-hts7f\" (UID: \"d0036363-bf0e-44d7-b285-44546ac729e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hts7f"
Oct 30 19:32:05.878448 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.877394    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/06b45974-f6ef-4b3a-b354-8103d7895fcf-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:05.878448 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.878240    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd-node-exporter-tls\") pod \"node-exporter-sx8t4\" (UID: \"0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd\") " pod="openshift-monitoring/node-exporter-sx8t4"
Oct 30 19:32:05.878448 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.877529    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8cde7f30-1d5b-4bea-ad21-2264b567ca1d-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-nq89t\" (UID: \"8cde7f30-1d5b-4bea-ad21-2264b567ca1d\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-nq89t"
Oct 30 19:32:05.878448 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.878326    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d0036363-bf0e-44d7-b285-44546ac729e6-sys-fs\") pod \"aws-ebs-csi-driver-node-hts7f\" (UID: \"d0036363-bf0e-44d7-b285-44546ac729e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hts7f"
Oct 30 19:32:05.878448 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.878442    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/141cf3eb-d99b-4876-b65a-a2847584ff5d-kubelet-config\") pod \"global-pull-secret-syncer-526rb\" (UID: \"141cf3eb-d99b-4876-b65a-a2847584ff5d\") " pod="kube-system/global-pull-secret-syncer-526rb"
Oct 30 19:32:05.878915 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.878594    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-host-var-lib-cni-multus\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:32:05.878915 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.878609    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/94ecba26-0809-414d-b084-8de0a693795e-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-996bc4dbd-wr65k\" (UID: \"94ecba26-0809-414d-b084-8de0a693795e\") " pod="openshift-monitoring/thanos-querier-996bc4dbd-wr65k"
Oct 30 19:32:05.878915 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.878658    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-etc-modprobe-d\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:32:05.878915 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.878698    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-host-var-lib-cni-multus\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:32:05.878915 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.878701    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd-node-exporter-wtmp\") pod \"node-exporter-sx8t4\" (UID: \"0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd\") " pod="openshift-monitoring/node-exporter-sx8t4"
Oct 30 19:32:05.878915 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.878752    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8cde7f30-1d5b-4bea-ad21-2264b567ca1d-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-nq89t\" (UID: \"8cde7f30-1d5b-4bea-ad21-2264b567ca1d\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-nq89t"
Oct 30 19:32:05.878915 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.878778    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/bfd79074-21c8-4caf-8c63-a2c5ff45e26d-federate-client-tls\") pod \"telemeter-client-6d56c5d745-9xkj5\" (UID: \"bfd79074-21c8-4caf-8c63-a2c5ff45e26d\") " pod="openshift-monitoring/telemeter-client-6d56c5d745-9xkj5"
Oct 30 19:32:05.878915 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.878812    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bfd79074-21c8-4caf-8c63-a2c5ff45e26d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6d56c5d745-9xkj5\" (UID: \"bfd79074-21c8-4caf-8c63-a2c5ff45e26d\") " pod="openshift-monitoring/telemeter-client-6d56c5d745-9xkj5"
Oct 30 19:32:05.878915 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.878843    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-host-slash\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:32:05.878915 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.878872    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-host-cni-netd\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:32:05.878915 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.878903    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/06b45974-f6ef-4b3a-b354-8103d7895fcf-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:05.879392 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.878968    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-multus-cni-dir\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:32:05.879392 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.879200    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:05.879392 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.879207    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-multus-cni-dir\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:32:05.879392 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.879285    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/06b45974-f6ef-4b3a-b354-8103d7895fcf-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:05.879392 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.879289    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-host-slash\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:32:05.879392 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.879376    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/bfd79074-21c8-4caf-8c63-a2c5ff45e26d-federate-client-tls\") pod \"telemeter-client-6d56c5d745-9xkj5\" (UID: \"bfd79074-21c8-4caf-8c63-a2c5ff45e26d\") " pod="openshift-monitoring/telemeter-client-6d56c5d745-9xkj5"
Oct 30 19:32:05.879672 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.879538    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-hostroot\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:32:05.879672 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.879587    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1977f3d7-c7f6-44cc-8a59-9d4f35adc0f7-hosts-file\") pod \"node-resolver-859jf\" (UID: \"1977f3d7-c7f6-44cc-8a59-9d4f35adc0f7\") " pod="openshift-dns/node-resolver-859jf"
Oct 30 19:32:05.879759 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.879689    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/94ecba26-0809-414d-b084-8de0a693795e-secret-thanos-querier-tls\") pod \"thanos-querier-996bc4dbd-wr65k\" (UID: \"94ecba26-0809-414d-b084-8de0a693795e\") " pod="openshift-monitoring/thanos-querier-996bc4dbd-wr65k"
Oct 30 19:32:05.879759 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.879738    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/79ea5164-32e2-4505-b92a-1e50495a6492-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9n92x\" (UID: \"79ea5164-32e2-4505-b92a-1e50495a6492\") " pod="openshift-multus/multus-additional-cni-plugins-9n92x"
Oct 30 19:32:05.879854 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.879766    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-host-var-lib-cni-bin\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:32:05.879854 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.879811    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb-secret-metrics-server-client-certs\") pod \"metrics-server-5cb8677769-45w2n\" (UID: \"136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb\") " pod="openshift-monitoring/metrics-server-5cb8677769-45w2n"
Oct 30 19:32:05.879979 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.879916    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb-metrics-server-audit-profiles\") pod \"metrics-server-5cb8677769-45w2n\" (UID: \"136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb\") " pod="openshift-monitoring/metrics-server-5cb8677769-45w2n"
Oct 30 19:32:05.879979 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.879962    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-var-lib-openvswitch\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:32:05.880074 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.879994    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-etc-systemd\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:32:05.880074 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.880043    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/94ecba26-0809-414d-b084-8de0a693795e-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-996bc4dbd-wr65k\" (UID: \"94ecba26-0809-414d-b084-8de0a693795e\") " pod="openshift-monitoring/thanos-querier-996bc4dbd-wr65k"
Oct 30 19:32:05.880165 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.880080    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-host-run-netns\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:32:05.880165 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.880127    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6kdm\" (UniqueName: \"kubernetes.io/projected/63effc7d-50b8-425f-b9a8-891ce2259283-kube-api-access-q6kdm\") pod \"console-55994d7c46-sz78s\" (UID: \"63effc7d-50b8-425f-b9a8-891ce2259283\") " pod="openshift-console/console-55994d7c46-sz78s"
Oct 30 19:32:05.881417 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.880411    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb-secret-metrics-server-client-certs\") pod \"metrics-server-5cb8677769-45w2n\" (UID: \"136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb\") " pod="openshift-monitoring/metrics-server-5cb8677769-45w2n"
Oct 30 19:32:05.881417 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.880433    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/94ecba26-0809-414d-b084-8de0a693795e-secret-grpc-tls\") pod \"thanos-querier-996bc4dbd-wr65k\" (UID: \"94ecba26-0809-414d-b084-8de0a693795e\") " pod="openshift-monitoring/thanos-querier-996bc4dbd-wr65k"
Oct 30 19:32:05.881417 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.880506    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1977f3d7-c7f6-44cc-8a59-9d4f35adc0f7-hosts-file\") pod \"node-resolver-859jf\" (UID: \"1977f3d7-c7f6-44cc-8a59-9d4f35adc0f7\") " pod="openshift-dns/node-resolver-859jf"
Oct 30 19:32:05.881417 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.880519    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-host-var-lib-cni-bin\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:32:05.881417 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.880563    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-hostroot\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:32:05.881417 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.880640    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bfd79074-21c8-4caf-8c63-a2c5ff45e26d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6d56c5d745-9xkj5\" (UID: \"bfd79074-21c8-4caf-8c63-a2c5ff45e26d\") " pod="openshift-monitoring/telemeter-client-6d56c5d745-9xkj5"
Oct 30 19:32:05.881417 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.880721    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/79ea5164-32e2-4505-b92a-1e50495a6492-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9n92x\" (UID: \"79ea5164-32e2-4505-b92a-1e50495a6492\") " pod="openshift-multus/multus-additional-cni-plugins-9n92x"
Oct 30 19:32:05.881417 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.880792    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/94ecba26-0809-414d-b084-8de0a693795e-secret-thanos-querier-tls\") pod \"thanos-querier-996bc4dbd-wr65k\" (UID: \"94ecba26-0809-414d-b084-8de0a693795e\") " pod="openshift-monitoring/thanos-querier-996bc4dbd-wr65k"
Oct 30 19:32:05.881417 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.880826    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb-metrics-server-audit-profiles\") pod \"metrics-server-5cb8677769-45w2n\" (UID: \"136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb\") " pod="openshift-monitoring/metrics-server-5cb8677769-45w2n"
Oct 30 19:32:05.881417 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.880901    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-var-lib-openvswitch\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:32:05.881417 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.880941    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/94ecba26-0809-414d-b084-8de0a693795e-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-996bc4dbd-wr65k\" (UID: \"94ecba26-0809-414d-b084-8de0a693795e\") " pod="openshift-monitoring/thanos-querier-996bc4dbd-wr65k"
Oct 30 19:32:05.881417 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.880997    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-host-run-netns\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:32:05.881417 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.881040    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-trusted-ca-bundle\") pod \"console-7c4f845644-d8mc5\" (UID: \"c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3\") " pod="openshift-console/console-7c4f845644-d8mc5"
Oct 30 19:32:05.881417 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.881051    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8cde7f30-1d5b-4bea-ad21-2264b567ca1d-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-nq89t\" (UID: \"8cde7f30-1d5b-4bea-ad21-2264b567ca1d\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-nq89t"
Oct 30 19:32:05.881417 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.881098    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd-sys\") pod \"node-exporter-sx8t4\" (UID: \"0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd\") " pod="openshift-monitoring/node-exporter-sx8t4"
Oct 30 19:32:05.881417 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.881132    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd-node-exporter-textfile\") pod \"node-exporter-sx8t4\" (UID: \"0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd\") " pod="openshift-monitoring/node-exporter-sx8t4"
Oct 30 19:32:05.881417 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.881163    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/94ecba26-0809-414d-b084-8de0a693795e-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-996bc4dbd-wr65k\" (UID: \"94ecba26-0809-414d-b084-8de0a693795e\") " pod="openshift-monitoring/thanos-querier-996bc4dbd-wr65k"
Oct 30 19:32:05.881417 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.881180    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-etc-modprobe-d\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:32:05.881417 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.881234    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-host-cni-netd\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:32:05.881417 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.881308    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5j8qc\" (UniqueName: \"kubernetes.io/projected/8cde7f30-1d5b-4bea-ad21-2264b567ca1d-kube-api-access-5j8qc\") pod \"openshift-state-metrics-7bd7489957-nq89t\" (UID: \"8cde7f30-1d5b-4bea-ad21-2264b567ca1d\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-nq89t"
Oct 30 19:32:05.881417 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.881315    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd-node-exporter-textfile\") pod \"node-exporter-sx8t4\" (UID: \"0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd\") " pod="openshift-monitoring/node-exporter-sx8t4"
Oct 30 19:32:05.881417 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.881341    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:05.881417 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.881374    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-etc-systemd\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:32:05.881417 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.881414    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-host-run-netns\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:32:05.882543 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.881691    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/94ecba26-0809-414d-b084-8de0a693795e-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-996bc4dbd-wr65k\" (UID: \"94ecba26-0809-414d-b084-8de0a693795e\") " pod="openshift-monitoring/thanos-querier-996bc4dbd-wr65k"
Oct 30 19:32:05.882543 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.881751    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-run-systemd\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:32:05.882543 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.881801    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-run\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:32:05.882543 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.881843    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-var-lib-kubelet\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:32:05.882543 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.881844    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:05.882543 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.881899    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-trusted-ca-bundle\") pod \"console-7c4f845644-d8mc5\" (UID: \"c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3\") " pod="openshift-console/console-7c4f845644-d8mc5"
Oct 30 19:32:05.882543 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.881910    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sx8t4\" (UID: \"0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd\") " pod="openshift-monitoring/node-exporter-sx8t4"
Oct 30 19:32:05.882543 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.881950    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lphsg\" (UniqueName: \"kubernetes.io/projected/0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd-kube-api-access-lphsg\") pod \"node-exporter-sx8t4\" (UID: \"0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd\") " pod="openshift-monitoring/node-exporter-sx8t4"
Oct 30 19:32:05.882543 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.881948    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-var-lib-kubelet\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:32:05.882543 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.881985    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-host-run-k8s-cni-cncf-io\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:32:05.882543 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.881987    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-run\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:32:05.882543 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.882034    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-run-systemd\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:32:05.882543 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.882049    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-etc-sysctl-conf\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:32:05.882543 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.882082    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-sys\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:32:05.882543 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.882137    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w94gr\" (UniqueName: \"kubernetes.io/projected/06b45974-f6ef-4b3a-b354-8103d7895fcf-kube-api-access-w94gr\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:05.882543 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.882168    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/63effc7d-50b8-425f-b9a8-891ce2259283-console-oauth-config\") pod \"console-55994d7c46-sz78s\" (UID: \"63effc7d-50b8-425f-b9a8-891ce2259283\") " pod="openshift-console/console-55994d7c46-sz78s"
Oct 30 19:32:05.882543 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.882195    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/63effc7d-50b8-425f-b9a8-891ce2259283-service-ca\") pod \"console-55994d7c46-sz78s\" (UID: \"63effc7d-50b8-425f-b9a8-891ce2259283\") " pod="openshift-console/console-55994d7c46-sz78s"
Oct 30 19:32:05.882543 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.882214    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sx8t4\" (UID: \"0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd\") " pod="openshift-monitoring/node-exporter-sx8t4"
Oct 30 19:32:05.882543 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.882232    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-host-run-multus-certs\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:32:05.882543 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.882330    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-host-run-multus-certs\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:32:05.882543 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.882332    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d0036363-bf0e-44d7-b285-44546ac729e6-registration-dir\") pod \"aws-ebs-csi-driver-node-hts7f\" (UID: \"d0036363-bf0e-44d7-b285-44546ac729e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hts7f"
Oct 30 19:32:05.882543 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.882368    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8cde7f30-1d5b-4bea-ad21-2264b567ca1d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-nq89t\" (UID: \"8cde7f30-1d5b-4bea-ad21-2264b567ca1d\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-nq89t"
Oct 30 19:32:05.882543 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.882377    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-etc-sysctl-conf\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:32:05.882543 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.882393    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:05.882543 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.882424    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/63effc7d-50b8-425f-b9a8-891ce2259283-console-serving-cert\") pod \"console-55994d7c46-sz78s\" (UID: \"63effc7d-50b8-425f-b9a8-891ce2259283\") " pod="openshift-console/console-55994d7c46-sz78s"
Oct 30 19:32:05.882543 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.882445    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d0036363-bf0e-44d7-b285-44546ac729e6-registration-dir\") pod \"aws-ebs-csi-driver-node-hts7f\" (UID: \"d0036363-bf0e-44d7-b285-44546ac729e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hts7f"
Oct 30 19:32:05.882543 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.882458    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-etc-kubernetes\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:32:05.882543 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.882544    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-host-run-netns\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:32:05.883891 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.882567    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-host-run-k8s-cni-cncf-io\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:32:05.883891 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.882614    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-oauth-serving-cert\") pod \"console-7c4f845644-d8mc5\" (UID: \"c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3\") " pod="openshift-console/console-7c4f845644-d8mc5"
Oct 30 19:32:05.883891 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.882657    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-sys\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:32:05.883891 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.882674    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1f3e7563-7109-45a5-9a77-7676a0697bd1-tmp\") pod \"console-6479499c8f-ptgvc\" (UID: \"1f3e7563-7109-45a5-9a77-7676a0697bd1\") " pod="openshift-console/console-6479499c8f-ptgvc"
Oct 30 19:32:05.883891 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.882710    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-systemd-units\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:32:05.883891 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.882770    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-run-openvswitch\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:32:05.883891 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.882801    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-host-cni-bin\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:32:05.883891 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.882847    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/63effc7d-50b8-425f-b9a8-891ce2259283-console-oauth-config\") pod \"console-55994d7c46-sz78s\" (UID: \"63effc7d-50b8-425f-b9a8-891ce2259283\") " pod="openshift-console/console-55994d7c46-sz78s"
Oct 30 19:32:05.883891 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.882888    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1f3e7563-7109-45a5-9a77-7676a0697bd1-tmp\") pod \"console-6479499c8f-ptgvc\" (UID: \"1f3e7563-7109-45a5-9a77-7676a0697bd1\") " pod="openshift-console/console-6479499c8f-ptgvc"
Oct 30 19:32:05.883891 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.882919    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-run-openvswitch\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:32:05.883891 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.882951    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-host-cni-bin\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:32:05.883891 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.882962    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-oauth-serving-cert\") pod \"console-7c4f845644-d8mc5\" (UID: \"c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3\") " pod="openshift-console/console-7c4f845644-d8mc5"
Oct 30 19:32:05.883891 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.882991    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-etc-kubernetes\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:32:05.883891 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.883016    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-systemd-units\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:32:05.883891 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.883062    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/94ecba26-0809-414d-b084-8de0a693795e-metrics-client-ca\") pod \"thanos-querier-996bc4dbd-wr65k\" (UID: \"94ecba26-0809-414d-b084-8de0a693795e\") " pod="openshift-monitoring/thanos-querier-996bc4dbd-wr65k"
Oct 30 19:32:05.883891 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.883095    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb-client-ca-bundle\") pod \"metrics-server-5cb8677769-45w2n\" (UID: \"136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb\") " pod="openshift-monitoring/metrics-server-5cb8677769-45w2n"
Oct 30 19:32:05.883891 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.883122    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/63effc7d-50b8-425f-b9a8-891ce2259283-console-serving-cert\") pod \"console-55994d7c46-sz78s\" (UID: \"63effc7d-50b8-425f-b9a8-891ce2259283\") " pod="openshift-console/console-55994d7c46-sz78s"
Oct 30 19:32:05.883891 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.883178    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-etc-kubernetes\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:32:05.883891 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.883183    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:05.883891 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.883224    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d0036363-bf0e-44d7-b285-44546ac729e6-socket-dir\") pod \"aws-ebs-csi-driver-node-hts7f\" (UID: \"d0036363-bf0e-44d7-b285-44546ac729e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hts7f"
Oct 30 19:32:05.883891 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.883256    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/94ecba26-0809-414d-b084-8de0a693795e-tmp\") pod \"thanos-querier-996bc4dbd-wr65k\" (UID: \"94ecba26-0809-414d-b084-8de0a693795e\") " pod="openshift-monitoring/thanos-querier-996bc4dbd-wr65k"
Oct 30 19:32:05.883891 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.883275    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:05.883891 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.883328    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/06b45974-f6ef-4b3a-b354-8103d7895fcf-config-out\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:05.883891 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.883356    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/63effc7d-50b8-425f-b9a8-891ce2259283-oauth-serving-cert\") pod \"console-55994d7c46-sz78s\" (UID: \"63effc7d-50b8-425f-b9a8-891ce2259283\") " pod="openshift-console/console-55994d7c46-sz78s"
Oct 30 19:32:05.883891 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.883375    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/63effc7d-50b8-425f-b9a8-891ce2259283-service-ca\") pod \"console-55994d7c46-sz78s\" (UID: \"63effc7d-50b8-425f-b9a8-891ce2259283\") " pod="openshift-console/console-55994d7c46-sz78s"
Oct 30 19:32:05.883891 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.883388    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0100ec89-6366-4946-99ce-4a254850c8fd-host-slash\") pod \"iptables-alerter-5jq48\" (UID: \"0100ec89-6366-4946-99ce-4a254850c8fd\") " pod="openshift-network-operator/iptables-alerter-5jq48"
Oct 30 19:32:05.883891 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.883422    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-etc-sysctl-d\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:32:05.883891 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.883525    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/94ecba26-0809-414d-b084-8de0a693795e-metrics-client-ca\") pod \"thanos-querier-996bc4dbd-wr65k\" (UID: \"94ecba26-0809-414d-b084-8de0a693795e\") " pod="openshift-monitoring/thanos-querier-996bc4dbd-wr65k"
Oct 30 19:32:05.883891 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.883580    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:05.883891 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.883586    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-etc-kubernetes\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:32:05.883891 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.883623    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0100ec89-6366-4946-99ce-4a254850c8fd-host-slash\") pod \"iptables-alerter-5jq48\" (UID: \"0100ec89-6366-4946-99ce-4a254850c8fd\") " pod="openshift-network-operator/iptables-alerter-5jq48"
Oct 30 19:32:05.883891 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.883650    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:05.883891 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.883684    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/141cf3eb-d99b-4876-b65a-a2847584ff5d-dbus\") pod \"global-pull-secret-syncer-526rb\" (UID: \"141cf3eb-d99b-4876-b65a-a2847584ff5d\") " pod="kube-system/global-pull-secret-syncer-526rb"
Oct 30 19:32:05.883891 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.883697    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/94ecba26-0809-414d-b084-8de0a693795e-tmp\") pod \"thanos-querier-996bc4dbd-wr65k\" (UID: \"94ecba26-0809-414d-b084-8de0a693795e\") " pod="openshift-monitoring/thanos-querier-996bc4dbd-wr65k"
Oct 30 19:32:05.883891 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.883751    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/06b45974-f6ef-4b3a-b354-8103d7895fcf-config-out\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:05.883891 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.883815    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb-client-ca-bundle\") pod \"metrics-server-5cb8677769-45w2n\" (UID: \"136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb\") " pod="openshift-monitoring/metrics-server-5cb8677769-45w2n"
Oct 30 19:32:05.883891 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.883874    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/63effc7d-50b8-425f-b9a8-891ce2259283-oauth-serving-cert\") pod \"console-55994d7c46-sz78s\" (UID: \"63effc7d-50b8-425f-b9a8-891ce2259283\") " pod="openshift-console/console-55994d7c46-sz78s"
Oct 30 19:32:05.885612 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.883927    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d0036363-bf0e-44d7-b285-44546ac729e6-socket-dir\") pod \"aws-ebs-csi-driver-node-hts7f\" (UID: \"d0036363-bf0e-44d7-b285-44546ac729e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hts7f"
Oct 30 19:32:05.885612 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.883964    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-etc-sysctl-d\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:32:05.885612 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.884019    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:05.885612 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.883597    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8cde7f30-1d5b-4bea-ad21-2264b567ca1d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-nq89t\" (UID: \"8cde7f30-1d5b-4bea-ad21-2264b567ca1d\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-nq89t"
Oct 30 19:32:05.885612 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.884166    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:05.885612 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.884241    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/141cf3eb-d99b-4876-b65a-a2847584ff5d-dbus\") pod \"global-pull-secret-syncer-526rb\" (UID: \"141cf3eb-d99b-4876-b65a-a2847584ff5d\") " pod="kube-system/global-pull-secret-syncer-526rb"
Oct 30 19:32:05.885612 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.884280    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-console-config\") pod \"console-7c4f845644-d8mc5\" (UID: \"c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3\") " pod="openshift-console/console-7c4f845644-d8mc5"
Oct 30 19:32:05.885612 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.884518    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:05.885612 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.884579    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:05.885612 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.884616    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-host-run-ovn-kubernetes\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:32:05.885612 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.884680    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:32:05.885612 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.884745    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd-root\") pod \"node-exporter-sx8t4\" (UID: \"0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd\") " pod="openshift-monitoring/node-exporter-sx8t4"
Oct 30 19:32:05.885612 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.884830    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-config\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:05.885612 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.884871    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-multus-conf-dir\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:32:05.885612 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.884961    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:05.885612 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.884977    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:32:05.885612 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.884927    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-multus-conf-dir\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:32:05.885612 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.885049    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/559fa506-ed9a-4c61-9316-edc85d6c5399-crio-socket\") pod \"insights-runtime-extractor-qdkpz\" (UID: \"559fa506-ed9a-4c61-9316-edc85d6c5399\") " pod="openshift-insights/insights-runtime-extractor-qdkpz"
Oct 30 19:32:05.885612 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.884991    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-host-run-ovn-kubernetes\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:32:05.885612 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.884931    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/559fa506-ed9a-4c61-9316-edc85d6c5399-crio-socket\") pod \"insights-runtime-extractor-qdkpz\" (UID: \"559fa506-ed9a-4c61-9316-edc85d6c5399\") " pod="openshift-insights/insights-runtime-extractor-qdkpz"
Oct 30 19:32:05.885612 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.885107    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/bfd79074-21c8-4caf-8c63-a2c5ff45e26d-secret-telemeter-client\") pod \"telemeter-client-6d56c5d745-9xkj5\" (UID: \"bfd79074-21c8-4caf-8c63-a2c5ff45e26d\") " pod="openshift-monitoring/telemeter-client-6d56c5d745-9xkj5"
Oct 30 19:32:05.885612 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.885316    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-config\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:05.885612 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.885352    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/bfd79074-21c8-4caf-8c63-a2c5ff45e26d-secret-telemeter-client\") pod \"telemeter-client-6d56c5d745-9xkj5\" (UID: \"bfd79074-21c8-4caf-8c63-a2c5ff45e26d\") " pod="openshift-monitoring/telemeter-client-6d56c5d745-9xkj5"
Oct 30 19:32:05.885612 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.885364    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1f3e7563-7109-45a5-9a77-7676a0697bd1-service-ca\") pod \"console-6479499c8f-ptgvc\" (UID: \"1f3e7563-7109-45a5-9a77-7676a0697bd1\") " pod="openshift-console/console-6479499c8f-ptgvc"
Oct 30 19:32:05.885612 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.885400    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bfd79074-21c8-4caf-8c63-a2c5ff45e26d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6d56c5d745-9xkj5\" (UID: \"bfd79074-21c8-4caf-8c63-a2c5ff45e26d\") " pod="openshift-monitoring/telemeter-client-6d56c5d745-9xkj5"
Oct 30 19:32:05.885612 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.885435    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:05.885612 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.885492    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:05.885612 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.885597    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-host-kubelet\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:32:05.885612 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.885630    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/79ea5164-32e2-4505-b92a-1e50495a6492-system-cni-dir\") pod \"multus-additional-cni-plugins-9n92x\" (UID: \"79ea5164-32e2-4505-b92a-1e50495a6492\") " pod="openshift-multus/multus-additional-cni-plugins-9n92x"
Oct 30 19:32:05.886766 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.885646    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-console-config\") pod \"console-7c4f845644-d8mc5\" (UID: \"c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3\") " pod="openshift-console/console-7c4f845644-d8mc5"
Oct 30 19:32:05.886766 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.885662    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-console-serving-cert\") pod \"console-7c4f845644-d8mc5\" (UID: \"c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3\") " pod="openshift-console/console-7c4f845644-d8mc5"
Oct 30 19:32:05.886766 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.885646    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1f3e7563-7109-45a5-9a77-7676a0697bd1-service-ca\") pod \"console-6479499c8f-ptgvc\" (UID: \"1f3e7563-7109-45a5-9a77-7676a0697bd1\") " pod="openshift-console/console-6479499c8f-ptgvc"
Oct 30 19:32:05.886766 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.885695    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-host-kubelet\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:32:05.886766 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.885707    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1f3e7563-7109-45a5-9a77-7676a0697bd1-console-oauth-config\") pod \"console-6479499c8f-ptgvc\" (UID: \"1f3e7563-7109-45a5-9a77-7676a0697bd1\") " pod="openshift-console/console-6479499c8f-ptgvc"
Oct 30 19:32:05.886766 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.885729    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bfd79074-21c8-4caf-8c63-a2c5ff45e26d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6d56c5d745-9xkj5\" (UID: \"bfd79074-21c8-4caf-8c63-a2c5ff45e26d\") " pod="openshift-monitoring/telemeter-client-6d56c5d745-9xkj5"
Oct 30 19:32:05.886766 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.885741    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-web-config\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:05.886766 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.885744    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:05.886766 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.885784    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/79ea5164-32e2-4505-b92a-1e50495a6492-system-cni-dir\") pod \"multus-additional-cni-plugins-9n92x\" (UID: \"79ea5164-32e2-4505-b92a-1e50495a6492\") " pod="openshift-multus/multus-additional-cni-plugins-9n92x"
Oct 30 19:32:05.886766 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.885826    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/63effc7d-50b8-425f-b9a8-891ce2259283-console-config\") pod \"console-55994d7c46-sz78s\" (UID: \"63effc7d-50b8-425f-b9a8-891ce2259283\") " pod="openshift-console/console-55994d7c46-sz78s"
Oct 30 19:32:05.886766 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.885831    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:05.886766 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.885850    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/63effc7d-50b8-425f-b9a8-891ce2259283-trusted-ca-bundle\") pod \"console-55994d7c46-sz78s\" (UID: \"63effc7d-50b8-425f-b9a8-891ce2259283\") " pod="openshift-console/console-55994d7c46-sz78s"
Oct 30 19:32:05.886766 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.885898    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-cnibin\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:32:05.886766 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.885920    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1f3e7563-7109-45a5-9a77-7676a0697bd1-console-oauth-config\") pod \"console-6479499c8f-ptgvc\" (UID: \"1f3e7563-7109-45a5-9a77-7676a0697bd1\") " pod="openshift-console/console-6479499c8f-ptgvc"
Oct 30 19:32:05.886766 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.885951    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-web-config\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:05.886766 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.885962    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-cnibin\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:32:05.886766 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.885968    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-console-serving-cert\") pod \"console-7c4f845644-d8mc5\" (UID: \"c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3\") " pod="openshift-console/console-7c4f845644-d8mc5"
Oct 30 19:32:05.886766 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.886034    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/63effc7d-50b8-425f-b9a8-891ce2259283-console-config\") pod \"console-55994d7c46-sz78s\" (UID: \"63effc7d-50b8-425f-b9a8-891ce2259283\") " pod="openshift-console/console-55994d7c46-sz78s"
Oct 30 19:32:05.886766 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.886062    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-etc-openvswitch\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:32:05.886766 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.886096    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-console-oauth-config\") pod \"console-7c4f845644-d8mc5\" (UID: \"c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3\") " pod="openshift-console/console-7c4f845644-d8mc5"
Oct 30 19:32:05.886766 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.886131    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb-secret-metrics-server-tls\") pod \"metrics-server-5cb8677769-45w2n\" (UID: \"136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb\") " pod="openshift-monitoring/metrics-server-5cb8677769-45w2n"
Oct 30 19:32:05.886766 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.886204    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1f3e7563-7109-45a5-9a77-7676a0697bd1-console-serving-cert\") pod \"console-6479499c8f-ptgvc\" (UID: \"1f3e7563-7109-45a5-9a77-7676a0697bd1\") " pod="openshift-console/console-6479499c8f-ptgvc"
Oct 30 19:32:05.886766 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.886174    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-etc-openvswitch\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:32:05.886766 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.886226    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/63effc7d-50b8-425f-b9a8-891ce2259283-trusted-ca-bundle\") pod \"console-55994d7c46-sz78s\" (UID: \"63effc7d-50b8-425f-b9a8-891ce2259283\") " pod="openshift-console/console-55994d7c46-sz78s"
Oct 30 19:32:05.886766 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.886237    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-node-log\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:32:05.886766 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.886281    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-node-log\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:32:05.886766 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.886286    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bfd79074-21c8-4caf-8c63-a2c5ff45e26d-serving-certs-ca-bundle\") pod \"telemeter-client-6d56c5d745-9xkj5\" (UID: \"bfd79074-21c8-4caf-8c63-a2c5ff45e26d\") " pod="openshift-monitoring/telemeter-client-6d56c5d745-9xkj5"
Oct 30 19:32:05.886766 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.886414    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb-secret-metrics-server-tls\") pod \"metrics-server-5cb8677769-45w2n\" (UID: \"136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb\") " pod="openshift-monitoring/metrics-server-5cb8677769-45w2n"
Oct 30 19:32:05.886766 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.886326    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-console-oauth-config\") pod \"console-7c4f845644-d8mc5\" (UID: \"c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3\") " pod="openshift-console/console-7c4f845644-d8mc5"
Oct 30 19:32:05.886766 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.886327    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:05.886766 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.886469    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1f3e7563-7109-45a5-9a77-7676a0697bd1-console-serving-cert\") pod \"console-6479499c8f-ptgvc\" (UID: \"1f3e7563-7109-45a5-9a77-7676a0697bd1\") " pod="openshift-console/console-6479499c8f-ptgvc"
Oct 30 19:32:05.888083 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.886858    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bfd79074-21c8-4caf-8c63-a2c5ff45e26d-serving-certs-ca-bundle\") pod \"telemeter-client-6d56c5d745-9xkj5\" (UID: \"bfd79074-21c8-4caf-8c63-a2c5ff45e26d\") " pod="openshift-monitoring/telemeter-client-6d56c5d745-9xkj5"
Oct 30 19:32:05.888083 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.886861    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/63effc7d-50b8-425f-b9a8-891ce2259283-tmp\") pod \"console-55994d7c46-sz78s\" (UID: \"63effc7d-50b8-425f-b9a8-891ce2259283\") " pod="openshift-console/console-55994d7c46-sz78s"
Oct 30 19:32:05.888083 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.886925    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-multus-socket-dir-parent\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:32:05.888083 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.886938    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:05.888083 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.886947    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/63effc7d-50b8-425f-b9a8-891ce2259283-tmp\") pod \"console-55994d7c46-sz78s\" (UID: \"63effc7d-50b8-425f-b9a8-891ce2259283\") " pod="openshift-console/console-55994d7c46-sz78s"
Oct 30 19:32:05.888083 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.886955    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-tmp\") pod \"console-7c4f845644-d8mc5\" (UID: \"c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3\") " pod="openshift-console/console-7c4f845644-d8mc5"
Oct 30 19:32:05.888083 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.887000    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-multus-socket-dir-parent\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:32:05.888083 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.887025    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5cb8677769-45w2n\" (UID: \"136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb\") " pod="openshift-monitoring/metrics-server-5cb8677769-45w2n"
Oct 30 19:32:05.888083 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.887053    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-tmp\") pod \"console-7c4f845644-d8mc5\" (UID: \"c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3\") " pod="openshift-console/console-7c4f845644-d8mc5"
Oct 30 19:32:05.888083 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.887077    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bfd79074-21c8-4caf-8c63-a2c5ff45e26d-metrics-client-ca\") pod \"telemeter-client-6d56c5d745-9xkj5\" (UID: \"bfd79074-21c8-4caf-8c63-a2c5ff45e26d\") " pod="openshift-monitoring/telemeter-client-6d56c5d745-9xkj5"
Oct 30 19:32:05.888083 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.887098    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-system-cni-dir\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:32:05.888083 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.887113    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-etc-sysconfig\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:32:05.888083 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.887128    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd-node-exporter-accelerators-collector-config\") pod \"node-exporter-sx8t4\" (UID: \"0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd\") " pod="openshift-monitoring/node-exporter-sx8t4"
Oct 30 19:32:05.888083 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.887181    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/63678441-271f-46d7-ac1d-57a60bb78b12-system-cni-dir\") pod \"multus-97df4\" (UID: \"63678441-271f-46d7-ac1d-57a60bb78b12\") " pod="openshift-multus/multus-97df4"
Oct 30 19:32:05.888083 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.887215    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5cb8677769-45w2n\" (UID: \"136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb\") " pod="openshift-monitoring/metrics-server-5cb8677769-45w2n"
Oct 30 19:32:05.888083 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.887222    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/94ecba26-0809-414d-b084-8de0a693795e-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-996bc4dbd-wr65k\" (UID: \"94ecba26-0809-414d-b084-8de0a693795e\") " pod="openshift-monitoring/thanos-querier-996bc4dbd-wr65k"
Oct 30 19:32:05.888083 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.887279    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd-node-exporter-accelerators-collector-config\") pod \"node-exporter-sx8t4\" (UID: \"0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd\") " pod="openshift-monitoring/node-exporter-sx8t4"
Oct 30 19:32:05.888083 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.887279    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb-audit-log\") pod \"metrics-server-5cb8677769-45w2n\" (UID: \"136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb\") " pod="openshift-monitoring/metrics-server-5cb8677769-45w2n"
Oct 30 19:32:05.888083 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.887335    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb-audit-log\") pod \"metrics-server-5cb8677769-45w2n\" (UID: \"136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb\") " pod="openshift-monitoring/metrics-server-5cb8677769-45w2n"
Oct 30 19:32:05.888083 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.887363    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-etc-sysconfig\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:32:05.888083 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.887469    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/94ecba26-0809-414d-b084-8de0a693795e-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-996bc4dbd-wr65k\" (UID: \"94ecba26-0809-414d-b084-8de0a693795e\") " pod="openshift-monitoring/thanos-querier-996bc4dbd-wr65k"
Oct 30 19:32:05.888083 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.887559    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bfd79074-21c8-4caf-8c63-a2c5ff45e26d-metrics-client-ca\") pod \"telemeter-client-6d56c5d745-9xkj5\" (UID: \"bfd79074-21c8-4caf-8c63-a2c5ff45e26d\") " pod="openshift-monitoring/telemeter-client-6d56c5d745-9xkj5"
Oct 30 19:32:05.888083 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.887595    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:05.888083 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.887678    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-host\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:32:05.888083 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.887742    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x98xh\" (UniqueName: \"kubernetes.io/projected/94ecba26-0809-414d-b084-8de0a693795e-kube-api-access-x98xh\") pod \"thanos-querier-996bc4dbd-wr65k\" (UID: \"94ecba26-0809-414d-b084-8de0a693795e\") " pod="openshift-monitoring/thanos-querier-996bc4dbd-wr65k"
Oct 30 19:32:05.888083 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.887786    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1f3e7563-7109-45a5-9a77-7676a0697bd1-console-config\") pod \"console-6479499c8f-ptgvc\" (UID: \"1f3e7563-7109-45a5-9a77-7676a0697bd1\") " pod="openshift-console/console-6479499c8f-ptgvc"
Oct 30 19:32:05.888083 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.887812    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f3e7563-7109-45a5-9a77-7676a0697bd1-trusted-ca-bundle\") pod \"console-6479499c8f-ptgvc\" (UID: \"1f3e7563-7109-45a5-9a77-7676a0697bd1\") " pod="openshift-console/console-6479499c8f-ptgvc"
Oct 30 19:32:05.888083 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.887838    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-log-socket\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:32:05.888083 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.887862    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:05.888083 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.888084    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1cc9603f-1c61-43e4-b987-0e185cf26252-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-ph577\" (UID: \"1cc9603f-1c61-43e4-b987-0e185cf26252\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-ph577"
Oct 30 19:32:05.889073 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.888113    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/bfd79074-21c8-4caf-8c63-a2c5ff45e26d-telemeter-client-tls\") pod \"telemeter-client-6d56c5d745-9xkj5\" (UID: \"bfd79074-21c8-4caf-8c63-a2c5ff45e26d\") " pod="openshift-monitoring/telemeter-client-6d56c5d745-9xkj5"
Oct 30 19:32:05.889073 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.888121    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f3e7563-7109-45a5-9a77-7676a0697bd1-trusted-ca-bundle\") pod \"console-6479499c8f-ptgvc\" (UID: \"1f3e7563-7109-45a5-9a77-7676a0697bd1\") " pod="openshift-console/console-6479499c8f-ptgvc"
Oct 30 19:32:05.889073 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.888138    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlnsz\" (UniqueName: \"kubernetes.io/projected/bfd79074-21c8-4caf-8c63-a2c5ff45e26d-kube-api-access-xlnsz\") pod \"telemeter-client-6d56c5d745-9xkj5\" (UID: \"bfd79074-21c8-4caf-8c63-a2c5ff45e26d\") " pod="openshift-monitoring/telemeter-client-6d56c5d745-9xkj5"
Oct 30 19:32:05.889073 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.888171    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9wcq\" (UniqueName: \"kubernetes.io/projected/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-kube-api-access-f9wcq\") pod \"console-7c4f845644-d8mc5\" (UID: \"c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3\") " pod="openshift-console/console-7c4f845644-d8mc5"
Oct 30 19:32:05.889073 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.888174    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e-host\") pod \"tuned-g4c6r\" (UID: \"6d3b6ee1-6e9f-4ce5-8390-126cfd215f6e\") " pod="openshift-cluster-node-tuning-operator/tuned-g4c6r"
Oct 30 19:32:05.889073 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.888207    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:05.889073 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.888223    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mqpt\" (UniqueName: \"kubernetes.io/projected/1f3e7563-7109-45a5-9a77-7676a0697bd1-kube-api-access-2mqpt\") pod \"console-6479499c8f-ptgvc\" (UID: \"1f3e7563-7109-45a5-9a77-7676a0697bd1\") " pod="openshift-console/console-6479499c8f-ptgvc"
Oct 30 19:32:05.889073 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.888389    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1cc9603f-1c61-43e4-b987-0e185cf26252-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-ph577\" (UID: \"1cc9603f-1c61-43e4-b987-0e185cf26252\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-ph577"
Oct 30 19:32:05.889073 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.888430    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/bfd79074-21c8-4caf-8c63-a2c5ff45e26d-telemeter-client-tls\") pod \"telemeter-client-6d56c5d745-9xkj5\" (UID: \"bfd79074-21c8-4caf-8c63-a2c5ff45e26d\") " pod="openshift-monitoring/telemeter-client-6d56c5d745-9xkj5"
Oct 30 19:32:05.889073 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.888433    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d4d142b9-22f1-427f-a374-ba4918487946-log-socket\") pod \"ovnkube-node-74lps\" (UID: \"d4d142b9-22f1-427f-a374-ba4918487946\") " pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:32:05.889073 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.888459    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:05.889073 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.888498    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmz8l\" (UniqueName: \"kubernetes.io/projected/1cc9603f-1c61-43e4-b987-0e185cf26252-kube-api-access-bmz8l\") pod \"monitoring-plugin-77f4f76594-ph577\" (UID: \"1cc9603f-1c61-43e4-b987-0e185cf26252\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-ph577"
Oct 30 19:32:05.889073 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.888752    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdcn4\" (UniqueName: \"kubernetes.io/projected/136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb-kube-api-access-wdcn4\") pod \"metrics-server-5cb8677769-45w2n\" (UID: \"136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb\") " pod="openshift-monitoring/metrics-server-5cb8677769-45w2n"
Oct 30 19:32:05.889073 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.888923    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1f3e7563-7109-45a5-9a77-7676a0697bd1-console-config\") pod \"console-6479499c8f-ptgvc\" (UID: \"1f3e7563-7109-45a5-9a77-7676a0697bd1\") " pod="openshift-console/console-6479499c8f-ptgvc"
Oct 30 19:32:05.891440 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.891421    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5j8qc\" (UniqueName: \"kubernetes.io/projected/8cde7f30-1d5b-4bea-ad21-2264b567ca1d-kube-api-access-5j8qc\") pod \"openshift-state-metrics-7bd7489957-nq89t\" (UID: \"8cde7f30-1d5b-4bea-ad21-2264b567ca1d\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-nq89t"
Oct 30 19:32:05.891534 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.891436    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6kdm\" (UniqueName: \"kubernetes.io/projected/63effc7d-50b8-425f-b9a8-891ce2259283-kube-api-access-q6kdm\") pod \"console-55994d7c46-sz78s\" (UID: \"63effc7d-50b8-425f-b9a8-891ce2259283\") " pod="openshift-console/console-55994d7c46-sz78s"
Oct 30 19:32:05.892047 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.892031    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lphsg\" (UniqueName: \"kubernetes.io/projected/0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd-kube-api-access-lphsg\") pod \"node-exporter-sx8t4\" (UID: \"0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd\") " pod="openshift-monitoring/node-exporter-sx8t4"
Oct 30 19:32:05.892130 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.892111    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w94gr\" (UniqueName: \"kubernetes.io/projected/06b45974-f6ef-4b3a-b354-8103d7895fcf-kube-api-access-w94gr\") pod \"prometheus-k8s-0\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:05.898151 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.898129    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x98xh\" (UniqueName: \"kubernetes.io/projected/94ecba26-0809-414d-b084-8de0a693795e-kube-api-access-x98xh\") pod \"thanos-querier-996bc4dbd-wr65k\" (UID: \"94ecba26-0809-414d-b084-8de0a693795e\") " pod="openshift-monitoring/thanos-querier-996bc4dbd-wr65k"
Oct 30 19:32:05.898735 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.898718    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlnsz\" (UniqueName: \"kubernetes.io/projected/bfd79074-21c8-4caf-8c63-a2c5ff45e26d-kube-api-access-xlnsz\") pod \"telemeter-client-6d56c5d745-9xkj5\" (UID: \"bfd79074-21c8-4caf-8c63-a2c5ff45e26d\") " pod="openshift-monitoring/telemeter-client-6d56c5d745-9xkj5"
Oct 30 19:32:05.988911 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.988835    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd-node-exporter-wtmp\") pod \"node-exporter-sx8t4\" (UID: \"0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd\") " pod="openshift-monitoring/node-exporter-sx8t4"
Oct 30 19:32:05.988911 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.988876    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd-sys\") pod \"node-exporter-sx8t4\" (UID: \"0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd\") " pod="openshift-monitoring/node-exporter-sx8t4"
Oct 30 19:32:05.989063 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.988939    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd-sys\") pod \"node-exporter-sx8t4\" (UID: \"0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd\") " pod="openshift-monitoring/node-exporter-sx8t4"
Oct 30 19:32:05.989063 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.988954    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd-node-exporter-wtmp\") pod \"node-exporter-sx8t4\" (UID: \"0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd\") " pod="openshift-monitoring/node-exporter-sx8t4"
Oct 30 19:32:05.989063 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.988962    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd-root\") pod \"node-exporter-sx8t4\" (UID: \"0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd\") " pod="openshift-monitoring/node-exporter-sx8t4"
Oct 30 19:32:05.989063 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:05.988979    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd-root\") pod \"node-exporter-sx8t4\" (UID: \"0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd\") " pod="openshift-monitoring/node-exporter-sx8t4"
Oct 30 19:32:06.607108 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:06.607073    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:32:07.367782 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:07.367752    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-996bc4dbd-wr65k"
Oct 30 19:32:07.673119 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:07.673046    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-8dn55"
Oct 30 19:32:08.747839 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:08.747793    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-55994d7c46-sz78s"
Oct 30 19:32:08.830419 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:08.830387    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:10.521524 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:10.521499    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:32:10.521896 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:10.521688    6996 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 19:32:11.239612 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:11.239584    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-ph577"
Oct 30 19:32:13.249957 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:13.249927    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-74lps"
Oct 30 19:32:15.100175 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:15.100137    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-4f5sl"
Oct 30 19:32:15.440162 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:15.440075    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-55994d7c46-sz78s"
Oct 30 19:32:17.060017 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:17.059987    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5cb8677769-45w2n"
Oct 30 19:32:18.752714 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:18.752685    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-55994d7c46-sz78s"
Oct 30 19:32:18.756365 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:18.756340    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-55994d7c46-sz78s"
Oct 30 19:32:18.890630 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:18.890599    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-5cf7f796c9-hsqwz"]
Oct 30 19:32:18.913395 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:18.913370    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cf7f796c9-hsqwz"
Oct 30 19:32:18.913939 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:18.913914    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5cf7f796c9-hsqwz"]
Oct 30 19:32:19.013377 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:19.013310    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3a5d313c-b78d-4bba-aeae-218fa0cb2e85-console-serving-cert\") pod \"console-5cf7f796c9-hsqwz\" (UID: \"3a5d313c-b78d-4bba-aeae-218fa0cb2e85\") " pod="openshift-console/console-5cf7f796c9-hsqwz"
Oct 30 19:32:19.013377 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:19.013342    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3a5d313c-b78d-4bba-aeae-218fa0cb2e85-console-oauth-config\") pod \"console-5cf7f796c9-hsqwz\" (UID: \"3a5d313c-b78d-4bba-aeae-218fa0cb2e85\") " pod="openshift-console/console-5cf7f796c9-hsqwz"
Oct 30 19:32:19.013377 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:19.013364    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3a5d313c-b78d-4bba-aeae-218fa0cb2e85-console-config\") pod \"console-5cf7f796c9-hsqwz\" (UID: \"3a5d313c-b78d-4bba-aeae-218fa0cb2e85\") " pod="openshift-console/console-5cf7f796c9-hsqwz"
Oct 30 19:32:19.013627 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:19.013567    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3a5d313c-b78d-4bba-aeae-218fa0cb2e85-tmp\") pod \"console-5cf7f796c9-hsqwz\" (UID: \"3a5d313c-b78d-4bba-aeae-218fa0cb2e85\") " pod="openshift-console/console-5cf7f796c9-hsqwz"
Oct 30 19:32:19.013627 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:19.013591    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3a5d313c-b78d-4bba-aeae-218fa0cb2e85-service-ca\") pod \"console-5cf7f796c9-hsqwz\" (UID: \"3a5d313c-b78d-4bba-aeae-218fa0cb2e85\") " pod="openshift-console/console-5cf7f796c9-hsqwz"
Oct 30 19:32:19.013627 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:19.013609    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3a5d313c-b78d-4bba-aeae-218fa0cb2e85-oauth-serving-cert\") pod \"console-5cf7f796c9-hsqwz\" (UID: \"3a5d313c-b78d-4bba-aeae-218fa0cb2e85\") " pod="openshift-console/console-5cf7f796c9-hsqwz"
Oct 30 19:32:19.013869 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:19.013845    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a5d313c-b78d-4bba-aeae-218fa0cb2e85-trusted-ca-bundle\") pod \"console-5cf7f796c9-hsqwz\" (UID: \"3a5d313c-b78d-4bba-aeae-218fa0cb2e85\") " pod="openshift-console/console-5cf7f796c9-hsqwz"
Oct 30 19:32:19.013999 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:19.013979    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lr52p\" (UniqueName: \"kubernetes.io/projected/3a5d313c-b78d-4bba-aeae-218fa0cb2e85-kube-api-access-lr52p\") pod \"console-5cf7f796c9-hsqwz\" (UID: \"3a5d313c-b78d-4bba-aeae-218fa0cb2e85\") " pod="openshift-console/console-5cf7f796c9-hsqwz"
Oct 30 19:32:19.115030 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:19.114979    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3a5d313c-b78d-4bba-aeae-218fa0cb2e85-tmp\") pod \"console-5cf7f796c9-hsqwz\" (UID: \"3a5d313c-b78d-4bba-aeae-218fa0cb2e85\") " pod="openshift-console/console-5cf7f796c9-hsqwz"
Oct 30 19:32:19.115183 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:19.115034    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3a5d313c-b78d-4bba-aeae-218fa0cb2e85-service-ca\") pod \"console-5cf7f796c9-hsqwz\" (UID: \"3a5d313c-b78d-4bba-aeae-218fa0cb2e85\") " pod="openshift-console/console-5cf7f796c9-hsqwz"
Oct 30 19:32:19.115183 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:19.115061    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3a5d313c-b78d-4bba-aeae-218fa0cb2e85-oauth-serving-cert\") pod \"console-5cf7f796c9-hsqwz\" (UID: \"3a5d313c-b78d-4bba-aeae-218fa0cb2e85\") " pod="openshift-console/console-5cf7f796c9-hsqwz"
Oct 30 19:32:19.115183 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:19.115132    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a5d313c-b78d-4bba-aeae-218fa0cb2e85-trusted-ca-bundle\") pod \"console-5cf7f796c9-hsqwz\" (UID: \"3a5d313c-b78d-4bba-aeae-218fa0cb2e85\") " pod="openshift-console/console-5cf7f796c9-hsqwz"
Oct 30 19:32:19.115183 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:19.115161    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lr52p\" (UniqueName: \"kubernetes.io/projected/3a5d313c-b78d-4bba-aeae-218fa0cb2e85-kube-api-access-lr52p\") pod \"console-5cf7f796c9-hsqwz\" (UID: \"3a5d313c-b78d-4bba-aeae-218fa0cb2e85\") " pod="openshift-console/console-5cf7f796c9-hsqwz"
Oct 30 19:32:19.115398 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:19.115200    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3a5d313c-b78d-4bba-aeae-218fa0cb2e85-console-serving-cert\") pod \"console-5cf7f796c9-hsqwz\" (UID: \"3a5d313c-b78d-4bba-aeae-218fa0cb2e85\") " pod="openshift-console/console-5cf7f796c9-hsqwz"
Oct 30 19:32:19.115398 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:19.115231    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3a5d313c-b78d-4bba-aeae-218fa0cb2e85-console-oauth-config\") pod \"console-5cf7f796c9-hsqwz\" (UID: \"3a5d313c-b78d-4bba-aeae-218fa0cb2e85\") " pod="openshift-console/console-5cf7f796c9-hsqwz"
Oct 30 19:32:19.115398 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:19.115371    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3a5d313c-b78d-4bba-aeae-218fa0cb2e85-tmp\") pod \"console-5cf7f796c9-hsqwz\" (UID: \"3a5d313c-b78d-4bba-aeae-218fa0cb2e85\") " pod="openshift-console/console-5cf7f796c9-hsqwz"
Oct 30 19:32:19.115592 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:19.115417    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3a5d313c-b78d-4bba-aeae-218fa0cb2e85-console-config\") pod \"console-5cf7f796c9-hsqwz\" (UID: \"3a5d313c-b78d-4bba-aeae-218fa0cb2e85\") " pod="openshift-console/console-5cf7f796c9-hsqwz"
Oct 30 19:32:19.115647 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:19.115611    6996 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 19:32:19.115755 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:19.115735    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3a5d313c-b78d-4bba-aeae-218fa0cb2e85-service-ca\") pod \"console-5cf7f796c9-hsqwz\" (UID: \"3a5d313c-b78d-4bba-aeae-218fa0cb2e85\") " pod="openshift-console/console-5cf7f796c9-hsqwz"
Oct 30 19:32:19.115899 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:19.115865    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3a5d313c-b78d-4bba-aeae-218fa0cb2e85-oauth-serving-cert\") pod \"console-5cf7f796c9-hsqwz\" (UID: \"3a5d313c-b78d-4bba-aeae-218fa0cb2e85\") " pod="openshift-console/console-5cf7f796c9-hsqwz"
Oct 30 19:32:19.116084 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:19.116067    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3a5d313c-b78d-4bba-aeae-218fa0cb2e85-console-config\") pod \"console-5cf7f796c9-hsqwz\" (UID: \"3a5d313c-b78d-4bba-aeae-218fa0cb2e85\") " pod="openshift-console/console-5cf7f796c9-hsqwz"
Oct 30 19:32:19.116149 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:19.116071    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a5d313c-b78d-4bba-aeae-218fa0cb2e85-trusted-ca-bundle\") pod \"console-5cf7f796c9-hsqwz\" (UID: \"3a5d313c-b78d-4bba-aeae-218fa0cb2e85\") " pod="openshift-console/console-5cf7f796c9-hsqwz"
Oct 30 19:32:19.117825 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:19.117809    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3a5d313c-b78d-4bba-aeae-218fa0cb2e85-console-serving-cert\") pod \"console-5cf7f796c9-hsqwz\" (UID: \"3a5d313c-b78d-4bba-aeae-218fa0cb2e85\") " pod="openshift-console/console-5cf7f796c9-hsqwz"
Oct 30 19:32:19.117825 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:19.117819    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3a5d313c-b78d-4bba-aeae-218fa0cb2e85-console-oauth-config\") pod \"console-5cf7f796c9-hsqwz\" (UID: \"3a5d313c-b78d-4bba-aeae-218fa0cb2e85\") " pod="openshift-console/console-5cf7f796c9-hsqwz"
Oct 30 19:32:19.128188 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:19.128169    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lr52p\" (UniqueName: \"kubernetes.io/projected/3a5d313c-b78d-4bba-aeae-218fa0cb2e85-kube-api-access-lr52p\") pod \"console-5cf7f796c9-hsqwz\" (UID: \"3a5d313c-b78d-4bba-aeae-218fa0cb2e85\") " pod="openshift-console/console-5cf7f796c9-hsqwz"
Oct 30 19:32:19.242522 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:19.242496    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cf7f796c9-hsqwz"
Oct 30 19:32:19.558637 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:19.558582    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5cf7f796c9-hsqwz"]
Oct 30 19:32:19.561246 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:19.561219    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a5d313c_b78d_4bba_aeae_218fa0cb2e85.slice/crio-39c3a5e135e3f7542fd544e60df3d0b1a0cabda5c3e8eaea1d5f90bcf525e6df WatchSource:0}: Error finding container 39c3a5e135e3f7542fd544e60df3d0b1a0cabda5c3e8eaea1d5f90bcf525e6df: Status 404 returned error can't find the container with id 39c3a5e135e3f7542fd544e60df3d0b1a0cabda5c3e8eaea1d5f90bcf525e6df
Oct 30 19:32:19.923006 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:19.922956    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cf7f796c9-hsqwz" event={"ID":"3a5d313c-b78d-4bba-aeae-218fa0cb2e85","Type":"ContainerStarted","Data":"2863d7344c9237c289e11787aba00bc27ccb279d910df76f39d545149a473f60"}
Oct 30 19:32:19.923401 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:19.923027    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cf7f796c9-hsqwz" event={"ID":"3a5d313c-b78d-4bba-aeae-218fa0cb2e85","Type":"ContainerStarted","Data":"39c3a5e135e3f7542fd544e60df3d0b1a0cabda5c3e8eaea1d5f90bcf525e6df"}
Oct 30 19:32:19.944824 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:19.944777    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-5cf7f796c9-hsqwz" podStartSLOduration=1.944743965 podStartE2EDuration="1.944743965s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:32:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:32:19.944657814 +0000 UTC m=+15.272775966" watchObservedRunningTime="2026-10-30 19:32:19.944743965 +0000 UTC m=+15.272862110"
Oct 30 19:32:24.220017 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:24.219988    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-bx5vk"
Oct 30 19:32:27.466320 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.466289    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 19:32:27.466819 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.466673    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="06b45974-f6ef-4b3a-b354-8103d7895fcf" containerName="prometheus" containerID="cri-o://3760d224c3458dbc7a10ac1532d6d30e7d3156efe9d705342fc1f896360461e9" gracePeriod=600
Oct 30 19:32:27.466819 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.466734    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="06b45974-f6ef-4b3a-b354-8103d7895fcf" containerName="kube-rbac-proxy-thanos" containerID="cri-o://3d4d0dde720d320dd166a3af41fdbd89194f7a53c8a1ab753b25b58c53ee43bc" gracePeriod=600
Oct 30 19:32:27.466819 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.466790    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="06b45974-f6ef-4b3a-b354-8103d7895fcf" containerName="kube-rbac-proxy-web" containerID="cri-o://3d708174472383233ec9987dac6af32cfc3dc15dbcc8a56e1aa40e07bca81f96" gracePeriod=600
Oct 30 19:32:27.467000 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.466873    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="06b45974-f6ef-4b3a-b354-8103d7895fcf" containerName="config-reloader" containerID="cri-o://e568bf64d537e305e782d862c5636dea7f5a48a98e3054f7a8b24927051daa68" gracePeriod=600
Oct 30 19:32:27.467000 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.466734    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="06b45974-f6ef-4b3a-b354-8103d7895fcf" containerName="thanos-sidecar" containerID="cri-o://2cf12d0db579be0cb78758ca0b67edf3a7a06735a238750e6a05dbd2122c0ed0" gracePeriod=600
Oct 30 19:32:27.467095 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.467040    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="06b45974-f6ef-4b3a-b354-8103d7895fcf" containerName="kube-rbac-proxy" containerID="cri-o://c57dd406ebf9f954f23ff488744fa6907ed8865079c549ea006b707c0facc082" gracePeriod=600
Oct 30 19:32:27.700218 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.700196    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:27.796886 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.796802    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-prometheus-k8s-rulefiles-1\") pod \"06b45974-f6ef-4b3a-b354-8103d7895fcf\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") "
Oct 30 19:32:27.796886 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.796841    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-prometheus-k8s-rulefiles-0\") pod \"06b45974-f6ef-4b3a-b354-8103d7895fcf\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") "
Oct 30 19:32:27.797094 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.796925    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-config\" (UniqueName: \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-config\") pod \"06b45974-f6ef-4b3a-b354-8103d7895fcf\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") "
Oct 30 19:32:27.797094 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.796966    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/06b45974-f6ef-4b3a-b354-8103d7895fcf-config-out\" (UniqueName: \"kubernetes.io/empty-dir/06b45974-f6ef-4b3a-b354-8103d7895fcf-config-out\") pod \"06b45974-f6ef-4b3a-b354-8103d7895fcf\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") "
Oct 30 19:32:27.797094 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.796988    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-configmap-serving-certs-ca-bundle\") pod \"06b45974-f6ef-4b3a-b354-8103d7895fcf\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") "
Oct 30 19:32:27.797094 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.797006    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-secret-kube-rbac-proxy\") pod \"06b45974-f6ef-4b3a-b354-8103d7895fcf\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") "
Oct 30 19:32:27.797094 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.797057    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"06b45974-f6ef-4b3a-b354-8103d7895fcf\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") "
Oct 30 19:32:27.797094 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.797086    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-secret-prometheus-k8s-tls\") pod \"06b45974-f6ef-4b3a-b354-8103d7895fcf\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") "
Oct 30 19:32:27.797383 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.797119    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-secret-grpc-tls\") pod \"06b45974-f6ef-4b3a-b354-8103d7895fcf\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") "
Oct 30 19:32:27.797383 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.797151    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/06b45974-f6ef-4b3a-b354-8103d7895fcf-tls-assets\" (UniqueName: \"kubernetes.io/projected/06b45974-f6ef-4b3a-b354-8103d7895fcf-tls-assets\") pod \"06b45974-f6ef-4b3a-b354-8103d7895fcf\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") "
Oct 30 19:32:27.797383 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.797181    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-web-config\" (UniqueName: \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-web-config\") pod \"06b45974-f6ef-4b3a-b354-8103d7895fcf\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") "
Oct 30 19:32:27.797383 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.797226    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-configmap-metrics-client-ca\") pod \"06b45974-f6ef-4b3a-b354-8103d7895fcf\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") "
Oct 30 19:32:27.797383 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.797260    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-secret-metrics-client-certs\") pod \"06b45974-f6ef-4b3a-b354-8103d7895fcf\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") "
Oct 30 19:32:27.797383 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.797293    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-thanos-prometheus-http-client-file\") pod \"06b45974-f6ef-4b3a-b354-8103d7895fcf\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") "
Oct 30 19:32:27.797383 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.797331    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/06b45974-f6ef-4b3a-b354-8103d7895fcf-kube-api-access-w94gr\" (UniqueName: \"kubernetes.io/projected/06b45974-f6ef-4b3a-b354-8103d7895fcf-kube-api-access-w94gr\") pod \"06b45974-f6ef-4b3a-b354-8103d7895fcf\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") "
Oct 30 19:32:27.797740 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.797364    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/06b45974-f6ef-4b3a-b354-8103d7895fcf-prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/06b45974-f6ef-4b3a-b354-8103d7895fcf-prometheus-k8s-db\") pod \"06b45974-f6ef-4b3a-b354-8103d7895fcf\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") "
Oct 30 19:32:27.797740 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.797414    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-configmap-serving-certs-ca-bundle" pod "06b45974-f6ef-4b3a-b354-8103d7895fcf" (UID: "06b45974-f6ef-4b3a-b354-8103d7895fcf"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:32:27.797740 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.797435    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-prometheus-trusted-ca-bundle\") pod \"06b45974-f6ef-4b3a-b354-8103d7895fcf\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") "
Oct 30 19:32:27.797740 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.797587    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-configmap-kubelet-serving-ca-bundle\") pod \"06b45974-f6ef-4b3a-b354-8103d7895fcf\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") "
Oct 30 19:32:27.797740 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.797645    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-prometheus-k8s-rulefiles-2\") pod \"06b45974-f6ef-4b3a-b354-8103d7895fcf\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") "
Oct 30 19:32:27.797740 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.797689    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"06b45974-f6ef-4b3a-b354-8103d7895fcf\" (UID: \"06b45974-f6ef-4b3a-b354-8103d7895fcf\") "
Oct 30 19:32:27.798027 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.797990    6996 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:27.798568 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.798297    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-prometheus-k8s-rulefiles-1" pod "06b45974-f6ef-4b3a-b354-8103d7895fcf" (UID: "06b45974-f6ef-4b3a-b354-8103d7895fcf"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:32:27.798568 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.798456    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-prometheus-k8s-rulefiles-0" pod "06b45974-f6ef-4b3a-b354-8103d7895fcf" (UID: "06b45974-f6ef-4b3a-b354-8103d7895fcf"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:32:27.800499 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.799110    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-prometheus-k8s-rulefiles-2" pod "06b45974-f6ef-4b3a-b354-8103d7895fcf" (UID: "06b45974-f6ef-4b3a-b354-8103d7895fcf"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:32:27.800499 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.799407    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-configmap-kubelet-serving-ca-bundle" pod "06b45974-f6ef-4b3a-b354-8103d7895fcf" (UID: "06b45974-f6ef-4b3a-b354-8103d7895fcf"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:32:27.800499 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.799843    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-prometheus-trusted-ca-bundle" pod "06b45974-f6ef-4b3a-b354-8103d7895fcf" (UID: "06b45974-f6ef-4b3a-b354-8103d7895fcf"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:32:27.800499 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.800436    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-secret-metrics-client-certs" pod "06b45974-f6ef-4b3a-b354-8103d7895fcf" (UID: "06b45974-f6ef-4b3a-b354-8103d7895fcf"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:32:27.800761 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.800508    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-configmap-metrics-client-ca" pod "06b45974-f6ef-4b3a-b354-8103d7895fcf" (UID: "06b45974-f6ef-4b3a-b354-8103d7895fcf"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:32:27.800761 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.800548    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06b45974-f6ef-4b3a-b354-8103d7895fcf-prometheus-k8s-db" pod "06b45974-f6ef-4b3a-b354-8103d7895fcf" (UID: "06b45974-f6ef-4b3a-b354-8103d7895fcf"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:27.800863 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.800797    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-secret-prometheus-k8s-thanos-sidecar-tls" pod "06b45974-f6ef-4b3a-b354-8103d7895fcf" (UID: "06b45974-f6ef-4b3a-b354-8103d7895fcf"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:32:27.801357 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.801280    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-secret-prometheus-k8s-tls" pod "06b45974-f6ef-4b3a-b354-8103d7895fcf" (UID: "06b45974-f6ef-4b3a-b354-8103d7895fcf"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:32:27.801581 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.801548    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06b45974-f6ef-4b3a-b354-8103d7895fcf-config-out" pod "06b45974-f6ef-4b3a-b354-8103d7895fcf" (UID: "06b45974-f6ef-4b3a-b354-8103d7895fcf"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:27.802554 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.802527    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-config" pod "06b45974-f6ef-4b3a-b354-8103d7895fcf" (UID: "06b45974-f6ef-4b3a-b354-8103d7895fcf"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:32:27.802948 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.802905    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06b45974-f6ef-4b3a-b354-8103d7895fcf-tls-assets" pod "06b45974-f6ef-4b3a-b354-8103d7895fcf" (UID: "06b45974-f6ef-4b3a-b354-8103d7895fcf"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:32:27.803050 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.802959    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-secret-kube-rbac-proxy" pod "06b45974-f6ef-4b3a-b354-8103d7895fcf" (UID: "06b45974-f6ef-4b3a-b354-8103d7895fcf"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:32:27.803114 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.803085    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-secret-grpc-tls" pod "06b45974-f6ef-4b3a-b354-8103d7895fcf" (UID: "06b45974-f6ef-4b3a-b354-8103d7895fcf"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:32:27.803184 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.803165    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-thanos-prometheus-http-client-file" pod "06b45974-f6ef-4b3a-b354-8103d7895fcf" (UID: "06b45974-f6ef-4b3a-b354-8103d7895fcf"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:32:27.803239 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.803175    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06b45974-f6ef-4b3a-b354-8103d7895fcf-kube-api-access-w94gr" pod "06b45974-f6ef-4b3a-b354-8103d7895fcf" (UID: "06b45974-f6ef-4b3a-b354-8103d7895fcf"). InnerVolumeSpecName "kube-api-access-w94gr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:32:27.803239 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.803227    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-secret-prometheus-k8s-kube-rbac-proxy-web" pod "06b45974-f6ef-4b3a-b354-8103d7895fcf" (UID: "06b45974-f6ef-4b3a-b354-8103d7895fcf"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:32:27.812896 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.812873    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-web-config" pod "06b45974-f6ef-4b3a-b354-8103d7895fcf" (UID: "06b45974-f6ef-4b3a-b354-8103d7895fcf"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:32:27.898846 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.898820    6996 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:27.898846 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.898844    6996 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-secret-prometheus-k8s-tls\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:27.898985 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.898856    6996 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-secret-grpc-tls\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:27.898985 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.898864    6996 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/06b45974-f6ef-4b3a-b354-8103d7895fcf-tls-assets\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:27.898985 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.898875    6996 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-web-config\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:27.898985 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.898885    6996 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-configmap-metrics-client-ca\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:27.898985 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.898894    6996 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-secret-metrics-client-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:27.898985 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.898902    6996 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-thanos-prometheus-http-client-file\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:27.898985 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.898911    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w94gr\" (UniqueName: \"kubernetes.io/projected/06b45974-f6ef-4b3a-b354-8103d7895fcf-kube-api-access-w94gr\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:27.898985 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.898921    6996 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/06b45974-f6ef-4b3a-b354-8103d7895fcf-prometheus-k8s-db\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:27.898985 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.898929    6996 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-prometheus-trusted-ca-bundle\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:27.898985 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.898938    6996 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:27.898985 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.898946    6996 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:27.898985 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.898962    6996 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:27.898985 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.898971    6996 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:27.898985 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.898979    6996 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/06b45974-f6ef-4b3a-b354-8103d7895fcf-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:27.898985 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.898988    6996 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-config\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:27.899379 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.898997    6996 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/06b45974-f6ef-4b3a-b354-8103d7895fcf-config-out\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:27.899379 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.899005    6996 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/06b45974-f6ef-4b3a-b354-8103d7895fcf-secret-kube-rbac-proxy\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:27.954584 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.954558    6996 generic.go:356] "Generic (PLEG): container finished" podID="06b45974-f6ef-4b3a-b354-8103d7895fcf" containerID="3d4d0dde720d320dd166a3af41fdbd89194f7a53c8a1ab753b25b58c53ee43bc" exitCode=0
Oct 30 19:32:27.954584 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.954579    6996 generic.go:356] "Generic (PLEG): container finished" podID="06b45974-f6ef-4b3a-b354-8103d7895fcf" containerID="c57dd406ebf9f954f23ff488744fa6907ed8865079c549ea006b707c0facc082" exitCode=0
Oct 30 19:32:27.954584 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.954585    6996 generic.go:356] "Generic (PLEG): container finished" podID="06b45974-f6ef-4b3a-b354-8103d7895fcf" containerID="3d708174472383233ec9987dac6af32cfc3dc15dbcc8a56e1aa40e07bca81f96" exitCode=0
Oct 30 19:32:27.954854 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.954591    6996 generic.go:356] "Generic (PLEG): container finished" podID="06b45974-f6ef-4b3a-b354-8103d7895fcf" containerID="2cf12d0db579be0cb78758ca0b67edf3a7a06735a238750e6a05dbd2122c0ed0" exitCode=0
Oct 30 19:32:27.954854 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.954597    6996 generic.go:356] "Generic (PLEG): container finished" podID="06b45974-f6ef-4b3a-b354-8103d7895fcf" containerID="e568bf64d537e305e782d862c5636dea7f5a48a98e3054f7a8b24927051daa68" exitCode=0
Oct 30 19:32:27.954854 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.954602    6996 generic.go:356] "Generic (PLEG): container finished" podID="06b45974-f6ef-4b3a-b354-8103d7895fcf" containerID="3760d224c3458dbc7a10ac1532d6d30e7d3156efe9d705342fc1f896360461e9" exitCode=0
Oct 30 19:32:27.954854 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.954636    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:27.954854 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.954645    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"06b45974-f6ef-4b3a-b354-8103d7895fcf","Type":"ContainerDied","Data":"3d4d0dde720d320dd166a3af41fdbd89194f7a53c8a1ab753b25b58c53ee43bc"}
Oct 30 19:32:27.954854 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.954736    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"06b45974-f6ef-4b3a-b354-8103d7895fcf","Type":"ContainerDied","Data":"c57dd406ebf9f954f23ff488744fa6907ed8865079c549ea006b707c0facc082"}
Oct 30 19:32:27.954854 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.954780    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"06b45974-f6ef-4b3a-b354-8103d7895fcf","Type":"ContainerDied","Data":"3d708174472383233ec9987dac6af32cfc3dc15dbcc8a56e1aa40e07bca81f96"}
Oct 30 19:32:27.954854 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.954796    6996 scope.go:122] "RemoveContainer" containerID="3d4d0dde720d320dd166a3af41fdbd89194f7a53c8a1ab753b25b58c53ee43bc"
Oct 30 19:32:27.954854 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.954822    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"06b45974-f6ef-4b3a-b354-8103d7895fcf","Type":"ContainerDied","Data":"2cf12d0db579be0cb78758ca0b67edf3a7a06735a238750e6a05dbd2122c0ed0"}
Oct 30 19:32:27.955262 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.954861    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"06b45974-f6ef-4b3a-b354-8103d7895fcf","Type":"ContainerDied","Data":"e568bf64d537e305e782d862c5636dea7f5a48a98e3054f7a8b24927051daa68"}
Oct 30 19:32:27.955262 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.954901    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"06b45974-f6ef-4b3a-b354-8103d7895fcf","Type":"ContainerDied","Data":"3760d224c3458dbc7a10ac1532d6d30e7d3156efe9d705342fc1f896360461e9"}
Oct 30 19:32:27.955262 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.954941    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"06b45974-f6ef-4b3a-b354-8103d7895fcf","Type":"ContainerDied","Data":"32a5d73fd496680155bec45a71c11338e31bfbe4529f4f057c9ff41e97d1fbe7"}
Oct 30 19:32:27.962340 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.962282    6996 scope.go:122] "RemoveContainer" containerID="c57dd406ebf9f954f23ff488744fa6907ed8865079c549ea006b707c0facc082"
Oct 30 19:32:27.969251 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.969234    6996 scope.go:122] "RemoveContainer" containerID="3d708174472383233ec9987dac6af32cfc3dc15dbcc8a56e1aa40e07bca81f96"
Oct 30 19:32:27.975397 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.975381    6996 scope.go:122] "RemoveContainer" containerID="2cf12d0db579be0cb78758ca0b67edf3a7a06735a238750e6a05dbd2122c0ed0"
Oct 30 19:32:27.979880 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.979859    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 19:32:27.982266 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.982247    6996 scope.go:122] "RemoveContainer" containerID="e568bf64d537e305e782d862c5636dea7f5a48a98e3054f7a8b24927051daa68"
Oct 30 19:32:27.988440 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.988419    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 19:32:27.989006 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.988991    6996 scope.go:122] "RemoveContainer" containerID="3760d224c3458dbc7a10ac1532d6d30e7d3156efe9d705342fc1f896360461e9"
Oct 30 19:32:27.995638 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:27.995621    6996 scope.go:122] "RemoveContainer" containerID="67eff76e35d54c9bb9f908398b7e410860adfb0a1adc657d551d62f0fc21744a"
Oct 30 19:32:28.001821 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.001797    6996 scope.go:122] "RemoveContainer" containerID="3d4d0dde720d320dd166a3af41fdbd89194f7a53c8a1ab753b25b58c53ee43bc"
Oct 30 19:32:28.002083 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:32:28.002062    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d4d0dde720d320dd166a3af41fdbd89194f7a53c8a1ab753b25b58c53ee43bc\": container with ID starting with 3d4d0dde720d320dd166a3af41fdbd89194f7a53c8a1ab753b25b58c53ee43bc not found: ID does not exist" containerID="3d4d0dde720d320dd166a3af41fdbd89194f7a53c8a1ab753b25b58c53ee43bc"
Oct 30 19:32:28.002156 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.002097    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d4d0dde720d320dd166a3af41fdbd89194f7a53c8a1ab753b25b58c53ee43bc"} err="failed to get container status \"3d4d0dde720d320dd166a3af41fdbd89194f7a53c8a1ab753b25b58c53ee43bc\": rpc error: code = NotFound desc = could not find container \"3d4d0dde720d320dd166a3af41fdbd89194f7a53c8a1ab753b25b58c53ee43bc\": container with ID starting with 3d4d0dde720d320dd166a3af41fdbd89194f7a53c8a1ab753b25b58c53ee43bc not found: ID does not exist"
Oct 30 19:32:28.002156 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.002141    6996 scope.go:122] "RemoveContainer" containerID="c57dd406ebf9f954f23ff488744fa6907ed8865079c549ea006b707c0facc082"
Oct 30 19:32:28.002407 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:32:28.002389    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c57dd406ebf9f954f23ff488744fa6907ed8865079c549ea006b707c0facc082\": container with ID starting with c57dd406ebf9f954f23ff488744fa6907ed8865079c549ea006b707c0facc082 not found: ID does not exist" containerID="c57dd406ebf9f954f23ff488744fa6907ed8865079c549ea006b707c0facc082"
Oct 30 19:32:28.002449 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.002412    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c57dd406ebf9f954f23ff488744fa6907ed8865079c549ea006b707c0facc082"} err="failed to get container status \"c57dd406ebf9f954f23ff488744fa6907ed8865079c549ea006b707c0facc082\": rpc error: code = NotFound desc = could not find container \"c57dd406ebf9f954f23ff488744fa6907ed8865079c549ea006b707c0facc082\": container with ID starting with c57dd406ebf9f954f23ff488744fa6907ed8865079c549ea006b707c0facc082 not found: ID does not exist"
Oct 30 19:32:28.002449 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.002427    6996 scope.go:122] "RemoveContainer" containerID="3d708174472383233ec9987dac6af32cfc3dc15dbcc8a56e1aa40e07bca81f96"
Oct 30 19:32:28.002745 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:32:28.002729    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d708174472383233ec9987dac6af32cfc3dc15dbcc8a56e1aa40e07bca81f96\": container with ID starting with 3d708174472383233ec9987dac6af32cfc3dc15dbcc8a56e1aa40e07bca81f96 not found: ID does not exist" containerID="3d708174472383233ec9987dac6af32cfc3dc15dbcc8a56e1aa40e07bca81f96"
Oct 30 19:32:28.002818 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.002751    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d708174472383233ec9987dac6af32cfc3dc15dbcc8a56e1aa40e07bca81f96"} err="failed to get container status \"3d708174472383233ec9987dac6af32cfc3dc15dbcc8a56e1aa40e07bca81f96\": rpc error: code = NotFound desc = could not find container \"3d708174472383233ec9987dac6af32cfc3dc15dbcc8a56e1aa40e07bca81f96\": container with ID starting with 3d708174472383233ec9987dac6af32cfc3dc15dbcc8a56e1aa40e07bca81f96 not found: ID does not exist"
Oct 30 19:32:28.002818 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.002768    6996 scope.go:122] "RemoveContainer" containerID="2cf12d0db579be0cb78758ca0b67edf3a7a06735a238750e6a05dbd2122c0ed0"
Oct 30 19:32:28.002982 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:32:28.002964    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2cf12d0db579be0cb78758ca0b67edf3a7a06735a238750e6a05dbd2122c0ed0\": container with ID starting with 2cf12d0db579be0cb78758ca0b67edf3a7a06735a238750e6a05dbd2122c0ed0 not found: ID does not exist" containerID="2cf12d0db579be0cb78758ca0b67edf3a7a06735a238750e6a05dbd2122c0ed0"
Oct 30 19:32:28.003022 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.002987    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2cf12d0db579be0cb78758ca0b67edf3a7a06735a238750e6a05dbd2122c0ed0"} err="failed to get container status \"2cf12d0db579be0cb78758ca0b67edf3a7a06735a238750e6a05dbd2122c0ed0\": rpc error: code = NotFound desc = could not find container \"2cf12d0db579be0cb78758ca0b67edf3a7a06735a238750e6a05dbd2122c0ed0\": container with ID starting with 2cf12d0db579be0cb78758ca0b67edf3a7a06735a238750e6a05dbd2122c0ed0 not found: ID does not exist"
Oct 30 19:32:28.003022 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.002999    6996 scope.go:122] "RemoveContainer" containerID="e568bf64d537e305e782d862c5636dea7f5a48a98e3054f7a8b24927051daa68"
Oct 30 19:32:28.003216 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:32:28.003200    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e568bf64d537e305e782d862c5636dea7f5a48a98e3054f7a8b24927051daa68\": container with ID starting with e568bf64d537e305e782d862c5636dea7f5a48a98e3054f7a8b24927051daa68 not found: ID does not exist" containerID="e568bf64d537e305e782d862c5636dea7f5a48a98e3054f7a8b24927051daa68"
Oct 30 19:32:28.003255 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.003223    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e568bf64d537e305e782d862c5636dea7f5a48a98e3054f7a8b24927051daa68"} err="failed to get container status \"e568bf64d537e305e782d862c5636dea7f5a48a98e3054f7a8b24927051daa68\": rpc error: code = NotFound desc = could not find container \"e568bf64d537e305e782d862c5636dea7f5a48a98e3054f7a8b24927051daa68\": container with ID starting with e568bf64d537e305e782d862c5636dea7f5a48a98e3054f7a8b24927051daa68 not found: ID does not exist"
Oct 30 19:32:28.003255 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.003239    6996 scope.go:122] "RemoveContainer" containerID="3760d224c3458dbc7a10ac1532d6d30e7d3156efe9d705342fc1f896360461e9"
Oct 30 19:32:28.003443 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:32:28.003428    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3760d224c3458dbc7a10ac1532d6d30e7d3156efe9d705342fc1f896360461e9\": container with ID starting with 3760d224c3458dbc7a10ac1532d6d30e7d3156efe9d705342fc1f896360461e9 not found: ID does not exist" containerID="3760d224c3458dbc7a10ac1532d6d30e7d3156efe9d705342fc1f896360461e9"
Oct 30 19:32:28.003512 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.003446    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3760d224c3458dbc7a10ac1532d6d30e7d3156efe9d705342fc1f896360461e9"} err="failed to get container status \"3760d224c3458dbc7a10ac1532d6d30e7d3156efe9d705342fc1f896360461e9\": rpc error: code = NotFound desc = could not find container \"3760d224c3458dbc7a10ac1532d6d30e7d3156efe9d705342fc1f896360461e9\": container with ID starting with 3760d224c3458dbc7a10ac1532d6d30e7d3156efe9d705342fc1f896360461e9 not found: ID does not exist"
Oct 30 19:32:28.003512 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.003458    6996 scope.go:122] "RemoveContainer" containerID="67eff76e35d54c9bb9f908398b7e410860adfb0a1adc657d551d62f0fc21744a"
Oct 30 19:32:28.003809 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:32:28.003792    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67eff76e35d54c9bb9f908398b7e410860adfb0a1adc657d551d62f0fc21744a\": container with ID starting with 67eff76e35d54c9bb9f908398b7e410860adfb0a1adc657d551d62f0fc21744a not found: ID does not exist" containerID="67eff76e35d54c9bb9f908398b7e410860adfb0a1adc657d551d62f0fc21744a"
Oct 30 19:32:28.003854 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.003813    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67eff76e35d54c9bb9f908398b7e410860adfb0a1adc657d551d62f0fc21744a"} err="failed to get container status \"67eff76e35d54c9bb9f908398b7e410860adfb0a1adc657d551d62f0fc21744a\": rpc error: code = NotFound desc = could not find container \"67eff76e35d54c9bb9f908398b7e410860adfb0a1adc657d551d62f0fc21744a\": container with ID starting with 67eff76e35d54c9bb9f908398b7e410860adfb0a1adc657d551d62f0fc21744a not found: ID does not exist"
Oct 30 19:32:28.003854 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.003825    6996 scope.go:122] "RemoveContainer" containerID="3d4d0dde720d320dd166a3af41fdbd89194f7a53c8a1ab753b25b58c53ee43bc"
Oct 30 19:32:28.004059 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.004040    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d4d0dde720d320dd166a3af41fdbd89194f7a53c8a1ab753b25b58c53ee43bc"} err="failed to get container status \"3d4d0dde720d320dd166a3af41fdbd89194f7a53c8a1ab753b25b58c53ee43bc\": rpc error: code = NotFound desc = could not find container \"3d4d0dde720d320dd166a3af41fdbd89194f7a53c8a1ab753b25b58c53ee43bc\": container with ID starting with 3d4d0dde720d320dd166a3af41fdbd89194f7a53c8a1ab753b25b58c53ee43bc not found: ID does not exist"
Oct 30 19:32:28.004121 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.004061    6996 scope.go:122] "RemoveContainer" containerID="c57dd406ebf9f954f23ff488744fa6907ed8865079c549ea006b707c0facc082"
Oct 30 19:32:28.004277 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.004259    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c57dd406ebf9f954f23ff488744fa6907ed8865079c549ea006b707c0facc082"} err="failed to get container status \"c57dd406ebf9f954f23ff488744fa6907ed8865079c549ea006b707c0facc082\": rpc error: code = NotFound desc = could not find container \"c57dd406ebf9f954f23ff488744fa6907ed8865079c549ea006b707c0facc082\": container with ID starting with c57dd406ebf9f954f23ff488744fa6907ed8865079c549ea006b707c0facc082 not found: ID does not exist"
Oct 30 19:32:28.004277 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.004276    6996 scope.go:122] "RemoveContainer" containerID="3d708174472383233ec9987dac6af32cfc3dc15dbcc8a56e1aa40e07bca81f96"
Oct 30 19:32:28.004524 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.004510    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d708174472383233ec9987dac6af32cfc3dc15dbcc8a56e1aa40e07bca81f96"} err="failed to get container status \"3d708174472383233ec9987dac6af32cfc3dc15dbcc8a56e1aa40e07bca81f96\": rpc error: code = NotFound desc = could not find container \"3d708174472383233ec9987dac6af32cfc3dc15dbcc8a56e1aa40e07bca81f96\": container with ID starting with 3d708174472383233ec9987dac6af32cfc3dc15dbcc8a56e1aa40e07bca81f96 not found: ID does not exist"
Oct 30 19:32:28.004524 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.004523    6996 scope.go:122] "RemoveContainer" containerID="2cf12d0db579be0cb78758ca0b67edf3a7a06735a238750e6a05dbd2122c0ed0"
Oct 30 19:32:28.004742 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.004723    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2cf12d0db579be0cb78758ca0b67edf3a7a06735a238750e6a05dbd2122c0ed0"} err="failed to get container status \"2cf12d0db579be0cb78758ca0b67edf3a7a06735a238750e6a05dbd2122c0ed0\": rpc error: code = NotFound desc = could not find container \"2cf12d0db579be0cb78758ca0b67edf3a7a06735a238750e6a05dbd2122c0ed0\": container with ID starting with 2cf12d0db579be0cb78758ca0b67edf3a7a06735a238750e6a05dbd2122c0ed0 not found: ID does not exist"
Oct 30 19:32:28.004789 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.004743    6996 scope.go:122] "RemoveContainer" containerID="e568bf64d537e305e782d862c5636dea7f5a48a98e3054f7a8b24927051daa68"
Oct 30 19:32:28.004960 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.004944    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e568bf64d537e305e782d862c5636dea7f5a48a98e3054f7a8b24927051daa68"} err="failed to get container status \"e568bf64d537e305e782d862c5636dea7f5a48a98e3054f7a8b24927051daa68\": rpc error: code = NotFound desc = could not find container \"e568bf64d537e305e782d862c5636dea7f5a48a98e3054f7a8b24927051daa68\": container with ID starting with e568bf64d537e305e782d862c5636dea7f5a48a98e3054f7a8b24927051daa68 not found: ID does not exist"
Oct 30 19:32:28.005005 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.004959    6996 scope.go:122] "RemoveContainer" containerID="3760d224c3458dbc7a10ac1532d6d30e7d3156efe9d705342fc1f896360461e9"
Oct 30 19:32:28.005158 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.005141    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3760d224c3458dbc7a10ac1532d6d30e7d3156efe9d705342fc1f896360461e9"} err="failed to get container status \"3760d224c3458dbc7a10ac1532d6d30e7d3156efe9d705342fc1f896360461e9\": rpc error: code = NotFound desc = could not find container \"3760d224c3458dbc7a10ac1532d6d30e7d3156efe9d705342fc1f896360461e9\": container with ID starting with 3760d224c3458dbc7a10ac1532d6d30e7d3156efe9d705342fc1f896360461e9 not found: ID does not exist"
Oct 30 19:32:28.005195 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.005157    6996 scope.go:122] "RemoveContainer" containerID="67eff76e35d54c9bb9f908398b7e410860adfb0a1adc657d551d62f0fc21744a"
Oct 30 19:32:28.005345 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.005330    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67eff76e35d54c9bb9f908398b7e410860adfb0a1adc657d551d62f0fc21744a"} err="failed to get container status \"67eff76e35d54c9bb9f908398b7e410860adfb0a1adc657d551d62f0fc21744a\": rpc error: code = NotFound desc = could not find container \"67eff76e35d54c9bb9f908398b7e410860adfb0a1adc657d551d62f0fc21744a\": container with ID starting with 67eff76e35d54c9bb9f908398b7e410860adfb0a1adc657d551d62f0fc21744a not found: ID does not exist"
Oct 30 19:32:28.005388 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.005344    6996 scope.go:122] "RemoveContainer" containerID="3d4d0dde720d320dd166a3af41fdbd89194f7a53c8a1ab753b25b58c53ee43bc"
Oct 30 19:32:28.005573 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.005554    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d4d0dde720d320dd166a3af41fdbd89194f7a53c8a1ab753b25b58c53ee43bc"} err="failed to get container status \"3d4d0dde720d320dd166a3af41fdbd89194f7a53c8a1ab753b25b58c53ee43bc\": rpc error: code = NotFound desc = could not find container \"3d4d0dde720d320dd166a3af41fdbd89194f7a53c8a1ab753b25b58c53ee43bc\": container with ID starting with 3d4d0dde720d320dd166a3af41fdbd89194f7a53c8a1ab753b25b58c53ee43bc not found: ID does not exist"
Oct 30 19:32:28.005644 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.005574    6996 scope.go:122] "RemoveContainer" containerID="c57dd406ebf9f954f23ff488744fa6907ed8865079c549ea006b707c0facc082"
Oct 30 19:32:28.005801 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.005785    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c57dd406ebf9f954f23ff488744fa6907ed8865079c549ea006b707c0facc082"} err="failed to get container status \"c57dd406ebf9f954f23ff488744fa6907ed8865079c549ea006b707c0facc082\": rpc error: code = NotFound desc = could not find container \"c57dd406ebf9f954f23ff488744fa6907ed8865079c549ea006b707c0facc082\": container with ID starting with c57dd406ebf9f954f23ff488744fa6907ed8865079c549ea006b707c0facc082 not found: ID does not exist"
Oct 30 19:32:28.005865 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.005801    6996 scope.go:122] "RemoveContainer" containerID="3d708174472383233ec9987dac6af32cfc3dc15dbcc8a56e1aa40e07bca81f96"
Oct 30 19:32:28.005990 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.005972    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d708174472383233ec9987dac6af32cfc3dc15dbcc8a56e1aa40e07bca81f96"} err="failed to get container status \"3d708174472383233ec9987dac6af32cfc3dc15dbcc8a56e1aa40e07bca81f96\": rpc error: code = NotFound desc = could not find container \"3d708174472383233ec9987dac6af32cfc3dc15dbcc8a56e1aa40e07bca81f96\": container with ID starting with 3d708174472383233ec9987dac6af32cfc3dc15dbcc8a56e1aa40e07bca81f96 not found: ID does not exist"
Oct 30 19:32:28.006060 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.005992    6996 scope.go:122] "RemoveContainer" containerID="2cf12d0db579be0cb78758ca0b67edf3a7a06735a238750e6a05dbd2122c0ed0"
Oct 30 19:32:28.006187 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.006173    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2cf12d0db579be0cb78758ca0b67edf3a7a06735a238750e6a05dbd2122c0ed0"} err="failed to get container status \"2cf12d0db579be0cb78758ca0b67edf3a7a06735a238750e6a05dbd2122c0ed0\": rpc error: code = NotFound desc = could not find container \"2cf12d0db579be0cb78758ca0b67edf3a7a06735a238750e6a05dbd2122c0ed0\": container with ID starting with 2cf12d0db579be0cb78758ca0b67edf3a7a06735a238750e6a05dbd2122c0ed0 not found: ID does not exist"
Oct 30 19:32:28.006223 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.006187    6996 scope.go:122] "RemoveContainer" containerID="e568bf64d537e305e782d862c5636dea7f5a48a98e3054f7a8b24927051daa68"
Oct 30 19:32:28.006370 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.006356    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e568bf64d537e305e782d862c5636dea7f5a48a98e3054f7a8b24927051daa68"} err="failed to get container status \"e568bf64d537e305e782d862c5636dea7f5a48a98e3054f7a8b24927051daa68\": rpc error: code = NotFound desc = could not find container \"e568bf64d537e305e782d862c5636dea7f5a48a98e3054f7a8b24927051daa68\": container with ID starting with e568bf64d537e305e782d862c5636dea7f5a48a98e3054f7a8b24927051daa68 not found: ID does not exist"
Oct 30 19:32:28.006370 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.006369    6996 scope.go:122] "RemoveContainer" containerID="3760d224c3458dbc7a10ac1532d6d30e7d3156efe9d705342fc1f896360461e9"
Oct 30 19:32:28.006580 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.006562    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3760d224c3458dbc7a10ac1532d6d30e7d3156efe9d705342fc1f896360461e9"} err="failed to get container status \"3760d224c3458dbc7a10ac1532d6d30e7d3156efe9d705342fc1f896360461e9\": rpc error: code = NotFound desc = could not find container \"3760d224c3458dbc7a10ac1532d6d30e7d3156efe9d705342fc1f896360461e9\": container with ID starting with 3760d224c3458dbc7a10ac1532d6d30e7d3156efe9d705342fc1f896360461e9 not found: ID does not exist"
Oct 30 19:32:28.006631 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.006580    6996 scope.go:122] "RemoveContainer" containerID="67eff76e35d54c9bb9f908398b7e410860adfb0a1adc657d551d62f0fc21744a"
Oct 30 19:32:28.006774 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.006756    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67eff76e35d54c9bb9f908398b7e410860adfb0a1adc657d551d62f0fc21744a"} err="failed to get container status \"67eff76e35d54c9bb9f908398b7e410860adfb0a1adc657d551d62f0fc21744a\": rpc error: code = NotFound desc = could not find container \"67eff76e35d54c9bb9f908398b7e410860adfb0a1adc657d551d62f0fc21744a\": container with ID starting with 67eff76e35d54c9bb9f908398b7e410860adfb0a1adc657d551d62f0fc21744a not found: ID does not exist"
Oct 30 19:32:28.006813 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.006775    6996 scope.go:122] "RemoveContainer" containerID="3d4d0dde720d320dd166a3af41fdbd89194f7a53c8a1ab753b25b58c53ee43bc"
Oct 30 19:32:28.006976 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.006962    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d4d0dde720d320dd166a3af41fdbd89194f7a53c8a1ab753b25b58c53ee43bc"} err="failed to get container status \"3d4d0dde720d320dd166a3af41fdbd89194f7a53c8a1ab753b25b58c53ee43bc\": rpc error: code = NotFound desc = could not find container \"3d4d0dde720d320dd166a3af41fdbd89194f7a53c8a1ab753b25b58c53ee43bc\": container with ID starting with 3d4d0dde720d320dd166a3af41fdbd89194f7a53c8a1ab753b25b58c53ee43bc not found: ID does not exist"
Oct 30 19:32:28.006976 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.006975    6996 scope.go:122] "RemoveContainer" containerID="c57dd406ebf9f954f23ff488744fa6907ed8865079c549ea006b707c0facc082"
Oct 30 19:32:28.007183 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.007167    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c57dd406ebf9f954f23ff488744fa6907ed8865079c549ea006b707c0facc082"} err="failed to get container status \"c57dd406ebf9f954f23ff488744fa6907ed8865079c549ea006b707c0facc082\": rpc error: code = NotFound desc = could not find container \"c57dd406ebf9f954f23ff488744fa6907ed8865079c549ea006b707c0facc082\": container with ID starting with c57dd406ebf9f954f23ff488744fa6907ed8865079c549ea006b707c0facc082 not found: ID does not exist"
Oct 30 19:32:28.007222 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.007183    6996 scope.go:122] "RemoveContainer" containerID="3d708174472383233ec9987dac6af32cfc3dc15dbcc8a56e1aa40e07bca81f96"
Oct 30 19:32:28.007372 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.007357    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d708174472383233ec9987dac6af32cfc3dc15dbcc8a56e1aa40e07bca81f96"} err="failed to get container status \"3d708174472383233ec9987dac6af32cfc3dc15dbcc8a56e1aa40e07bca81f96\": rpc error: code = NotFound desc = could not find container \"3d708174472383233ec9987dac6af32cfc3dc15dbcc8a56e1aa40e07bca81f96\": container with ID starting with 3d708174472383233ec9987dac6af32cfc3dc15dbcc8a56e1aa40e07bca81f96 not found: ID does not exist"
Oct 30 19:32:28.007412 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.007372    6996 scope.go:122] "RemoveContainer" containerID="2cf12d0db579be0cb78758ca0b67edf3a7a06735a238750e6a05dbd2122c0ed0"
Oct 30 19:32:28.007615 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.007599    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2cf12d0db579be0cb78758ca0b67edf3a7a06735a238750e6a05dbd2122c0ed0"} err="failed to get container status \"2cf12d0db579be0cb78758ca0b67edf3a7a06735a238750e6a05dbd2122c0ed0\": rpc error: code = NotFound desc = could not find container \"2cf12d0db579be0cb78758ca0b67edf3a7a06735a238750e6a05dbd2122c0ed0\": container with ID starting with 2cf12d0db579be0cb78758ca0b67edf3a7a06735a238750e6a05dbd2122c0ed0 not found: ID does not exist"
Oct 30 19:32:28.007668 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.007614    6996 scope.go:122] "RemoveContainer" containerID="e568bf64d537e305e782d862c5636dea7f5a48a98e3054f7a8b24927051daa68"
Oct 30 19:32:28.007823 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.007805    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e568bf64d537e305e782d862c5636dea7f5a48a98e3054f7a8b24927051daa68"} err="failed to get container status \"e568bf64d537e305e782d862c5636dea7f5a48a98e3054f7a8b24927051daa68\": rpc error: code = NotFound desc = could not find container \"e568bf64d537e305e782d862c5636dea7f5a48a98e3054f7a8b24927051daa68\": container with ID starting with e568bf64d537e305e782d862c5636dea7f5a48a98e3054f7a8b24927051daa68 not found: ID does not exist"
Oct 30 19:32:28.007890 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.007825    6996 scope.go:122] "RemoveContainer" containerID="3760d224c3458dbc7a10ac1532d6d30e7d3156efe9d705342fc1f896360461e9"
Oct 30 19:32:28.008052 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.008038    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3760d224c3458dbc7a10ac1532d6d30e7d3156efe9d705342fc1f896360461e9"} err="failed to get container status \"3760d224c3458dbc7a10ac1532d6d30e7d3156efe9d705342fc1f896360461e9\": rpc error: code = NotFound desc = could not find container \"3760d224c3458dbc7a10ac1532d6d30e7d3156efe9d705342fc1f896360461e9\": container with ID starting with 3760d224c3458dbc7a10ac1532d6d30e7d3156efe9d705342fc1f896360461e9 not found: ID does not exist"
Oct 30 19:32:28.008103 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.008052    6996 scope.go:122] "RemoveContainer" containerID="67eff76e35d54c9bb9f908398b7e410860adfb0a1adc657d551d62f0fc21744a"
Oct 30 19:32:28.008251 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.008234    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67eff76e35d54c9bb9f908398b7e410860adfb0a1adc657d551d62f0fc21744a"} err="failed to get container status \"67eff76e35d54c9bb9f908398b7e410860adfb0a1adc657d551d62f0fc21744a\": rpc error: code = NotFound desc = could not find container \"67eff76e35d54c9bb9f908398b7e410860adfb0a1adc657d551d62f0fc21744a\": container with ID starting with 67eff76e35d54c9bb9f908398b7e410860adfb0a1adc657d551d62f0fc21744a not found: ID does not exist"
Oct 30 19:32:28.008296 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.008255    6996 scope.go:122] "RemoveContainer" containerID="3d4d0dde720d320dd166a3af41fdbd89194f7a53c8a1ab753b25b58c53ee43bc"
Oct 30 19:32:28.008474 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.008455    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d4d0dde720d320dd166a3af41fdbd89194f7a53c8a1ab753b25b58c53ee43bc"} err="failed to get container status \"3d4d0dde720d320dd166a3af41fdbd89194f7a53c8a1ab753b25b58c53ee43bc\": rpc error: code = NotFound desc = could not find container \"3d4d0dde720d320dd166a3af41fdbd89194f7a53c8a1ab753b25b58c53ee43bc\": container with ID starting with 3d4d0dde720d320dd166a3af41fdbd89194f7a53c8a1ab753b25b58c53ee43bc not found: ID does not exist"
Oct 30 19:32:28.008474 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.008475    6996 scope.go:122] "RemoveContainer" containerID="c57dd406ebf9f954f23ff488744fa6907ed8865079c549ea006b707c0facc082"
Oct 30 19:32:28.008675 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.008656    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c57dd406ebf9f954f23ff488744fa6907ed8865079c549ea006b707c0facc082"} err="failed to get container status \"c57dd406ebf9f954f23ff488744fa6907ed8865079c549ea006b707c0facc082\": rpc error: code = NotFound desc = could not find container \"c57dd406ebf9f954f23ff488744fa6907ed8865079c549ea006b707c0facc082\": container with ID starting with c57dd406ebf9f954f23ff488744fa6907ed8865079c549ea006b707c0facc082 not found: ID does not exist"
Oct 30 19:32:28.008730 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.008676    6996 scope.go:122] "RemoveContainer" containerID="3d708174472383233ec9987dac6af32cfc3dc15dbcc8a56e1aa40e07bca81f96"
Oct 30 19:32:28.008889 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.008873    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d708174472383233ec9987dac6af32cfc3dc15dbcc8a56e1aa40e07bca81f96"} err="failed to get container status \"3d708174472383233ec9987dac6af32cfc3dc15dbcc8a56e1aa40e07bca81f96\": rpc error: code = NotFound desc = could not find container \"3d708174472383233ec9987dac6af32cfc3dc15dbcc8a56e1aa40e07bca81f96\": container with ID starting with 3d708174472383233ec9987dac6af32cfc3dc15dbcc8a56e1aa40e07bca81f96 not found: ID does not exist"
Oct 30 19:32:28.008940 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.008890    6996 scope.go:122] "RemoveContainer" containerID="2cf12d0db579be0cb78758ca0b67edf3a7a06735a238750e6a05dbd2122c0ed0"
Oct 30 19:32:28.009090 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.009072    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2cf12d0db579be0cb78758ca0b67edf3a7a06735a238750e6a05dbd2122c0ed0"} err="failed to get container status \"2cf12d0db579be0cb78758ca0b67edf3a7a06735a238750e6a05dbd2122c0ed0\": rpc error: code = NotFound desc = could not find container \"2cf12d0db579be0cb78758ca0b67edf3a7a06735a238750e6a05dbd2122c0ed0\": container with ID starting with 2cf12d0db579be0cb78758ca0b67edf3a7a06735a238750e6a05dbd2122c0ed0 not found: ID does not exist"
Oct 30 19:32:28.009155 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.009090    6996 scope.go:122] "RemoveContainer" containerID="e568bf64d537e305e782d862c5636dea7f5a48a98e3054f7a8b24927051daa68"
Oct 30 19:32:28.009301 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.009284    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e568bf64d537e305e782d862c5636dea7f5a48a98e3054f7a8b24927051daa68"} err="failed to get container status \"e568bf64d537e305e782d862c5636dea7f5a48a98e3054f7a8b24927051daa68\": rpc error: code = NotFound desc = could not find container \"e568bf64d537e305e782d862c5636dea7f5a48a98e3054f7a8b24927051daa68\": container with ID starting with e568bf64d537e305e782d862c5636dea7f5a48a98e3054f7a8b24927051daa68 not found: ID does not exist"
Oct 30 19:32:28.009341 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.009302    6996 scope.go:122] "RemoveContainer" containerID="3760d224c3458dbc7a10ac1532d6d30e7d3156efe9d705342fc1f896360461e9"
Oct 30 19:32:28.009514 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.009496    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3760d224c3458dbc7a10ac1532d6d30e7d3156efe9d705342fc1f896360461e9"} err="failed to get container status \"3760d224c3458dbc7a10ac1532d6d30e7d3156efe9d705342fc1f896360461e9\": rpc error: code = NotFound desc = could not find container \"3760d224c3458dbc7a10ac1532d6d30e7d3156efe9d705342fc1f896360461e9\": container with ID starting with 3760d224c3458dbc7a10ac1532d6d30e7d3156efe9d705342fc1f896360461e9 not found: ID does not exist"
Oct 30 19:32:28.009562 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.009515    6996 scope.go:122] "RemoveContainer" containerID="67eff76e35d54c9bb9f908398b7e410860adfb0a1adc657d551d62f0fc21744a"
Oct 30 19:32:28.009719 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.009700    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67eff76e35d54c9bb9f908398b7e410860adfb0a1adc657d551d62f0fc21744a"} err="failed to get container status \"67eff76e35d54c9bb9f908398b7e410860adfb0a1adc657d551d62f0fc21744a\": rpc error: code = NotFound desc = could not find container \"67eff76e35d54c9bb9f908398b7e410860adfb0a1adc657d551d62f0fc21744a\": container with ID starting with 67eff76e35d54c9bb9f908398b7e410860adfb0a1adc657d551d62f0fc21744a not found: ID does not exist"
Oct 30 19:32:28.009759 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.009720    6996 scope.go:122] "RemoveContainer" containerID="3d4d0dde720d320dd166a3af41fdbd89194f7a53c8a1ab753b25b58c53ee43bc"
Oct 30 19:32:28.009924 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.009907    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d4d0dde720d320dd166a3af41fdbd89194f7a53c8a1ab753b25b58c53ee43bc"} err="failed to get container status \"3d4d0dde720d320dd166a3af41fdbd89194f7a53c8a1ab753b25b58c53ee43bc\": rpc error: code = NotFound desc = could not find container \"3d4d0dde720d320dd166a3af41fdbd89194f7a53c8a1ab753b25b58c53ee43bc\": container with ID starting with 3d4d0dde720d320dd166a3af41fdbd89194f7a53c8a1ab753b25b58c53ee43bc not found: ID does not exist"
Oct 30 19:32:28.009963 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.009925    6996 scope.go:122] "RemoveContainer" containerID="c57dd406ebf9f954f23ff488744fa6907ed8865079c549ea006b707c0facc082"
Oct 30 19:32:28.010125 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.010107    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c57dd406ebf9f954f23ff488744fa6907ed8865079c549ea006b707c0facc082"} err="failed to get container status \"c57dd406ebf9f954f23ff488744fa6907ed8865079c549ea006b707c0facc082\": rpc error: code = NotFound desc = could not find container \"c57dd406ebf9f954f23ff488744fa6907ed8865079c549ea006b707c0facc082\": container with ID starting with c57dd406ebf9f954f23ff488744fa6907ed8865079c549ea006b707c0facc082 not found: ID does not exist"
Oct 30 19:32:28.010161 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.010126    6996 scope.go:122] "RemoveContainer" containerID="3d708174472383233ec9987dac6af32cfc3dc15dbcc8a56e1aa40e07bca81f96"
Oct 30 19:32:28.010321 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.010302    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d708174472383233ec9987dac6af32cfc3dc15dbcc8a56e1aa40e07bca81f96"} err="failed to get container status \"3d708174472383233ec9987dac6af32cfc3dc15dbcc8a56e1aa40e07bca81f96\": rpc error: code = NotFound desc = could not find container \"3d708174472383233ec9987dac6af32cfc3dc15dbcc8a56e1aa40e07bca81f96\": container with ID starting with 3d708174472383233ec9987dac6af32cfc3dc15dbcc8a56e1aa40e07bca81f96 not found: ID does not exist"
Oct 30 19:32:28.010360 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.010322    6996 scope.go:122] "RemoveContainer" containerID="2cf12d0db579be0cb78758ca0b67edf3a7a06735a238750e6a05dbd2122c0ed0"
Oct 30 19:32:28.010531 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.010514    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2cf12d0db579be0cb78758ca0b67edf3a7a06735a238750e6a05dbd2122c0ed0"} err="failed to get container status \"2cf12d0db579be0cb78758ca0b67edf3a7a06735a238750e6a05dbd2122c0ed0\": rpc error: code = NotFound desc = could not find container \"2cf12d0db579be0cb78758ca0b67edf3a7a06735a238750e6a05dbd2122c0ed0\": container with ID starting with 2cf12d0db579be0cb78758ca0b67edf3a7a06735a238750e6a05dbd2122c0ed0 not found: ID does not exist"
Oct 30 19:32:28.010590 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.010532    6996 scope.go:122] "RemoveContainer" containerID="e568bf64d537e305e782d862c5636dea7f5a48a98e3054f7a8b24927051daa68"
Oct 30 19:32:28.010754 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.010737    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e568bf64d537e305e782d862c5636dea7f5a48a98e3054f7a8b24927051daa68"} err="failed to get container status \"e568bf64d537e305e782d862c5636dea7f5a48a98e3054f7a8b24927051daa68\": rpc error: code = NotFound desc = could not find container \"e568bf64d537e305e782d862c5636dea7f5a48a98e3054f7a8b24927051daa68\": container with ID starting with e568bf64d537e305e782d862c5636dea7f5a48a98e3054f7a8b24927051daa68 not found: ID does not exist"
Oct 30 19:32:28.010805 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.010755    6996 scope.go:122] "RemoveContainer" containerID="3760d224c3458dbc7a10ac1532d6d30e7d3156efe9d705342fc1f896360461e9"
Oct 30 19:32:28.010950 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.010933    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3760d224c3458dbc7a10ac1532d6d30e7d3156efe9d705342fc1f896360461e9"} err="failed to get container status \"3760d224c3458dbc7a10ac1532d6d30e7d3156efe9d705342fc1f896360461e9\": rpc error: code = NotFound desc = could not find container \"3760d224c3458dbc7a10ac1532d6d30e7d3156efe9d705342fc1f896360461e9\": container with ID starting with 3760d224c3458dbc7a10ac1532d6d30e7d3156efe9d705342fc1f896360461e9 not found: ID does not exist"
Oct 30 19:32:28.011018 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.010951    6996 scope.go:122] "RemoveContainer" containerID="67eff76e35d54c9bb9f908398b7e410860adfb0a1adc657d551d62f0fc21744a"
Oct 30 19:32:28.011154 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.011139    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67eff76e35d54c9bb9f908398b7e410860adfb0a1adc657d551d62f0fc21744a"} err="failed to get container status \"67eff76e35d54c9bb9f908398b7e410860adfb0a1adc657d551d62f0fc21744a\": rpc error: code = NotFound desc = could not find container \"67eff76e35d54c9bb9f908398b7e410860adfb0a1adc657d551d62f0fc21744a\": container with ID starting with 67eff76e35d54c9bb9f908398b7e410860adfb0a1adc657d551d62f0fc21744a not found: ID does not exist"
Oct 30 19:32:28.024444 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.024423    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 19:32:28.024739 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.024721    6996 cpu_manager.go:414] "containerMap: removing container" podUID="06b45974-f6ef-4b3a-b354-8103d7895fcf" containerName="kube-rbac-proxy-thanos"
Oct 30 19:32:28.024817 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.024739    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="06b45974-f6ef-4b3a-b354-8103d7895fcf" containerName="kube-rbac-proxy-thanos"
Oct 30 19:32:28.024817 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.024757    6996 cpu_manager.go:414] "containerMap: removing container" podUID="06b45974-f6ef-4b3a-b354-8103d7895fcf" containerName="kube-rbac-proxy"
Oct 30 19:32:28.024817 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.024762    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="06b45974-f6ef-4b3a-b354-8103d7895fcf" containerName="kube-rbac-proxy"
Oct 30 19:32:28.024817 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.024769    6996 cpu_manager.go:414] "containerMap: removing container" podUID="06b45974-f6ef-4b3a-b354-8103d7895fcf" containerName="config-reloader"
Oct 30 19:32:28.024817 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.024773    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="06b45974-f6ef-4b3a-b354-8103d7895fcf" containerName="config-reloader"
Oct 30 19:32:28.024817 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.024779    6996 cpu_manager.go:414] "containerMap: removing container" podUID="06b45974-f6ef-4b3a-b354-8103d7895fcf" containerName="prometheus"
Oct 30 19:32:28.024817 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.024784    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="06b45974-f6ef-4b3a-b354-8103d7895fcf" containerName="prometheus"
Oct 30 19:32:28.024817 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.024793    6996 cpu_manager.go:414] "containerMap: removing container" podUID="06b45974-f6ef-4b3a-b354-8103d7895fcf" containerName="kube-rbac-proxy-web"
Oct 30 19:32:28.024817 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.024797    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="06b45974-f6ef-4b3a-b354-8103d7895fcf" containerName="kube-rbac-proxy-web"
Oct 30 19:32:28.024817 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.024805    6996 cpu_manager.go:414] "containerMap: removing container" podUID="06b45974-f6ef-4b3a-b354-8103d7895fcf" containerName="init-config-reloader"
Oct 30 19:32:28.024817 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.024809    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="06b45974-f6ef-4b3a-b354-8103d7895fcf" containerName="init-config-reloader"
Oct 30 19:32:28.024817 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.024821    6996 cpu_manager.go:414] "containerMap: removing container" podUID="06b45974-f6ef-4b3a-b354-8103d7895fcf" containerName="thanos-sidecar"
Oct 30 19:32:28.025205 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.024828    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="06b45974-f6ef-4b3a-b354-8103d7895fcf" containerName="thanos-sidecar"
Oct 30 19:32:28.025205 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.024919    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="06b45974-f6ef-4b3a-b354-8103d7895fcf" containerName="kube-rbac-proxy"
Oct 30 19:32:28.025205 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.024929    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="06b45974-f6ef-4b3a-b354-8103d7895fcf" containerName="config-reloader"
Oct 30 19:32:28.025205 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.024937    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="06b45974-f6ef-4b3a-b354-8103d7895fcf" containerName="prometheus"
Oct 30 19:32:28.025205 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.024946    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="06b45974-f6ef-4b3a-b354-8103d7895fcf" containerName="kube-rbac-proxy-web"
Oct 30 19:32:28.025205 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.024956    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="06b45974-f6ef-4b3a-b354-8103d7895fcf" containerName="init-config-reloader"
Oct 30 19:32:28.025205 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.024964    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="06b45974-f6ef-4b3a-b354-8103d7895fcf" containerName="thanos-sidecar"
Oct 30 19:32:28.025205 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.024973    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="06b45974-f6ef-4b3a-b354-8103d7895fcf" containerName="kube-rbac-proxy-thanos"
Oct 30 19:32:28.028368 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.028353    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:28.034834 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.034811    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Oct 30 19:32:28.035333 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.035168    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Oct 30 19:32:28.035333 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.035211    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Oct 30 19:32:28.035333 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.035233    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Oct 30 19:32:28.035333 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.035232    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Oct 30 19:32:28.035333 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.035249    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Oct 30 19:32:28.035333 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.035286    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Oct 30 19:32:28.035333 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.035306    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Oct 30 19:32:28.035333 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.035308    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Oct 30 19:32:28.035333 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.035305    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Oct 30 19:32:28.035744 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.035513    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Oct 30 19:32:28.035744 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.035616    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-f6ia1krs5s708\""
Oct 30 19:32:28.038179 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.036120    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-gxpqn\""
Oct 30 19:32:28.038179 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.036293    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Oct 30 19:32:28.042666 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.042643    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Oct 30 19:32:28.042666 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.042653    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Oct 30 19:32:28.051783 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.051733    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 19:32:28.201999 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.201958    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c1e7407b-e78a-4b5c-b34f-44f5a24bc56f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c1e7407b-e78a-4b5c-b34f-44f5a24bc56f\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:28.202174 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.202004    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c1e7407b-e78a-4b5c-b34f-44f5a24bc56f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c1e7407b-e78a-4b5c-b34f-44f5a24bc56f\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:28.202174 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.202067    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c1e7407b-e78a-4b5c-b34f-44f5a24bc56f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c1e7407b-e78a-4b5c-b34f-44f5a24bc56f\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:28.202174 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.202086    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c1e7407b-e78a-4b5c-b34f-44f5a24bc56f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c1e7407b-e78a-4b5c-b34f-44f5a24bc56f\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:28.202174 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.202100    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c1e7407b-e78a-4b5c-b34f-44f5a24bc56f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c1e7407b-e78a-4b5c-b34f-44f5a24bc56f\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:28.202174 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.202115    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c1e7407b-e78a-4b5c-b34f-44f5a24bc56f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c1e7407b-e78a-4b5c-b34f-44f5a24bc56f\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:28.202174 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.202133    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1e7407b-e78a-4b5c-b34f-44f5a24bc56f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c1e7407b-e78a-4b5c-b34f-44f5a24bc56f\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:28.202348 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.202215    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c1e7407b-e78a-4b5c-b34f-44f5a24bc56f-web-config\") pod \"prometheus-k8s-0\" (UID: \"c1e7407b-e78a-4b5c-b34f-44f5a24bc56f\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:28.202348 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.202232    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dw29x\" (UniqueName: \"kubernetes.io/projected/c1e7407b-e78a-4b5c-b34f-44f5a24bc56f-kube-api-access-dw29x\") pod \"prometheus-k8s-0\" (UID: \"c1e7407b-e78a-4b5c-b34f-44f5a24bc56f\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:28.202348 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.202306    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c1e7407b-e78a-4b5c-b34f-44f5a24bc56f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c1e7407b-e78a-4b5c-b34f-44f5a24bc56f\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:28.202348 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.202346    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c1e7407b-e78a-4b5c-b34f-44f5a24bc56f-config\") pod \"prometheus-k8s-0\" (UID: \"c1e7407b-e78a-4b5c-b34f-44f5a24bc56f\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:28.202679 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.202373    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c1e7407b-e78a-4b5c-b34f-44f5a24bc56f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c1e7407b-e78a-4b5c-b34f-44f5a24bc56f\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:28.202679 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.202418    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c1e7407b-e78a-4b5c-b34f-44f5a24bc56f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c1e7407b-e78a-4b5c-b34f-44f5a24bc56f\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:28.202679 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.202466    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1e7407b-e78a-4b5c-b34f-44f5a24bc56f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c1e7407b-e78a-4b5c-b34f-44f5a24bc56f\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:28.202679 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.202510    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c1e7407b-e78a-4b5c-b34f-44f5a24bc56f-config-out\") pod \"prometheus-k8s-0\" (UID: \"c1e7407b-e78a-4b5c-b34f-44f5a24bc56f\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:28.202679 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.202533    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c1e7407b-e78a-4b5c-b34f-44f5a24bc56f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c1e7407b-e78a-4b5c-b34f-44f5a24bc56f\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:28.202679 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.202593    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c1e7407b-e78a-4b5c-b34f-44f5a24bc56f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c1e7407b-e78a-4b5c-b34f-44f5a24bc56f\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:28.202679 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.202620    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c1e7407b-e78a-4b5c-b34f-44f5a24bc56f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c1e7407b-e78a-4b5c-b34f-44f5a24bc56f\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:28.202679 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.202640    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1e7407b-e78a-4b5c-b34f-44f5a24bc56f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c1e7407b-e78a-4b5c-b34f-44f5a24bc56f\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:28.202679 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.202657    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c1e7407b-e78a-4b5c-b34f-44f5a24bc56f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c1e7407b-e78a-4b5c-b34f-44f5a24bc56f\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:28.303539 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.303444    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c1e7407b-e78a-4b5c-b34f-44f5a24bc56f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c1e7407b-e78a-4b5c-b34f-44f5a24bc56f\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:28.303539 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.303504    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c1e7407b-e78a-4b5c-b34f-44f5a24bc56f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c1e7407b-e78a-4b5c-b34f-44f5a24bc56f\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:28.303539 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.303531    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1e7407b-e78a-4b5c-b34f-44f5a24bc56f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c1e7407b-e78a-4b5c-b34f-44f5a24bc56f\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:28.303702 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.303622    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c1e7407b-e78a-4b5c-b34f-44f5a24bc56f-config-out\") pod \"prometheus-k8s-0\" (UID: \"c1e7407b-e78a-4b5c-b34f-44f5a24bc56f\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:28.303702 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.303660    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c1e7407b-e78a-4b5c-b34f-44f5a24bc56f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c1e7407b-e78a-4b5c-b34f-44f5a24bc56f\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:28.303702 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.303686    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c1e7407b-e78a-4b5c-b34f-44f5a24bc56f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c1e7407b-e78a-4b5c-b34f-44f5a24bc56f\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:28.303827 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.303725    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c1e7407b-e78a-4b5c-b34f-44f5a24bc56f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c1e7407b-e78a-4b5c-b34f-44f5a24bc56f\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:28.303827 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.303754    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1e7407b-e78a-4b5c-b34f-44f5a24bc56f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c1e7407b-e78a-4b5c-b34f-44f5a24bc56f\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:28.303827 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.303781    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c1e7407b-e78a-4b5c-b34f-44f5a24bc56f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c1e7407b-e78a-4b5c-b34f-44f5a24bc56f\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:28.303827 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.303821    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c1e7407b-e78a-4b5c-b34f-44f5a24bc56f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c1e7407b-e78a-4b5c-b34f-44f5a24bc56f\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:28.304014 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.303861    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c1e7407b-e78a-4b5c-b34f-44f5a24bc56f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c1e7407b-e78a-4b5c-b34f-44f5a24bc56f\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:28.304014 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.303886    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c1e7407b-e78a-4b5c-b34f-44f5a24bc56f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c1e7407b-e78a-4b5c-b34f-44f5a24bc56f\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:28.304014 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.303911    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c1e7407b-e78a-4b5c-b34f-44f5a24bc56f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c1e7407b-e78a-4b5c-b34f-44f5a24bc56f\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:28.304014 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.303935    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c1e7407b-e78a-4b5c-b34f-44f5a24bc56f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c1e7407b-e78a-4b5c-b34f-44f5a24bc56f\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:28.304014 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.303957    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c1e7407b-e78a-4b5c-b34f-44f5a24bc56f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c1e7407b-e78a-4b5c-b34f-44f5a24bc56f\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:28.304014 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.303988    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1e7407b-e78a-4b5c-b34f-44f5a24bc56f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c1e7407b-e78a-4b5c-b34f-44f5a24bc56f\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:28.304305 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.304019    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c1e7407b-e78a-4b5c-b34f-44f5a24bc56f-web-config\") pod \"prometheus-k8s-0\" (UID: \"c1e7407b-e78a-4b5c-b34f-44f5a24bc56f\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:28.304305 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.304040    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dw29x\" (UniqueName: \"kubernetes.io/projected/c1e7407b-e78a-4b5c-b34f-44f5a24bc56f-kube-api-access-dw29x\") pod \"prometheus-k8s-0\" (UID: \"c1e7407b-e78a-4b5c-b34f-44f5a24bc56f\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:28.304305 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.304070    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c1e7407b-e78a-4b5c-b34f-44f5a24bc56f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c1e7407b-e78a-4b5c-b34f-44f5a24bc56f\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:28.304305 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.304113    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c1e7407b-e78a-4b5c-b34f-44f5a24bc56f-config\") pod \"prometheus-k8s-0\" (UID: \"c1e7407b-e78a-4b5c-b34f-44f5a24bc56f\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:28.304510 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.304321    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c1e7407b-e78a-4b5c-b34f-44f5a24bc56f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c1e7407b-e78a-4b5c-b34f-44f5a24bc56f\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:28.304510 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.304395    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1e7407b-e78a-4b5c-b34f-44f5a24bc56f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c1e7407b-e78a-4b5c-b34f-44f5a24bc56f\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:28.304791 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.304761    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c1e7407b-e78a-4b5c-b34f-44f5a24bc56f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c1e7407b-e78a-4b5c-b34f-44f5a24bc56f\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:28.305410 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.305381    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1e7407b-e78a-4b5c-b34f-44f5a24bc56f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c1e7407b-e78a-4b5c-b34f-44f5a24bc56f\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:28.306577 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.306349    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c1e7407b-e78a-4b5c-b34f-44f5a24bc56f-config-out\") pod \"prometheus-k8s-0\" (UID: \"c1e7407b-e78a-4b5c-b34f-44f5a24bc56f\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:28.306577 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.306451    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c1e7407b-e78a-4b5c-b34f-44f5a24bc56f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c1e7407b-e78a-4b5c-b34f-44f5a24bc56f\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:28.306734 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.306690    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c1e7407b-e78a-4b5c-b34f-44f5a24bc56f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c1e7407b-e78a-4b5c-b34f-44f5a24bc56f\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:28.306924 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.306901    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c1e7407b-e78a-4b5c-b34f-44f5a24bc56f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c1e7407b-e78a-4b5c-b34f-44f5a24bc56f\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:28.307130 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.307107    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1e7407b-e78a-4b5c-b34f-44f5a24bc56f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c1e7407b-e78a-4b5c-b34f-44f5a24bc56f\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:28.307195 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.307167    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c1e7407b-e78a-4b5c-b34f-44f5a24bc56f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c1e7407b-e78a-4b5c-b34f-44f5a24bc56f\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:28.307287 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.307236    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c1e7407b-e78a-4b5c-b34f-44f5a24bc56f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c1e7407b-e78a-4b5c-b34f-44f5a24bc56f\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:28.307379 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.307360    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c1e7407b-e78a-4b5c-b34f-44f5a24bc56f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c1e7407b-e78a-4b5c-b34f-44f5a24bc56f\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:28.307833 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.307814    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c1e7407b-e78a-4b5c-b34f-44f5a24bc56f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c1e7407b-e78a-4b5c-b34f-44f5a24bc56f\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:28.308015 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.307990    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c1e7407b-e78a-4b5c-b34f-44f5a24bc56f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c1e7407b-e78a-4b5c-b34f-44f5a24bc56f\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:28.308094 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.308069    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/c1e7407b-e78a-4b5c-b34f-44f5a24bc56f-config\") pod \"prometheus-k8s-0\" (UID: \"c1e7407b-e78a-4b5c-b34f-44f5a24bc56f\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:28.309024 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.309004    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c1e7407b-e78a-4b5c-b34f-44f5a24bc56f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c1e7407b-e78a-4b5c-b34f-44f5a24bc56f\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:28.309419 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.309395    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c1e7407b-e78a-4b5c-b34f-44f5a24bc56f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c1e7407b-e78a-4b5c-b34f-44f5a24bc56f\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:28.309832 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.309813    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c1e7407b-e78a-4b5c-b34f-44f5a24bc56f-web-config\") pod \"prometheus-k8s-0\" (UID: \"c1e7407b-e78a-4b5c-b34f-44f5a24bc56f\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:28.309872 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.309813    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c1e7407b-e78a-4b5c-b34f-44f5a24bc56f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c1e7407b-e78a-4b5c-b34f-44f5a24bc56f\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:28.324294 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.324274    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dw29x\" (UniqueName: \"kubernetes.io/projected/c1e7407b-e78a-4b5c-b34f-44f5a24bc56f-kube-api-access-dw29x\") pod \"prometheus-k8s-0\" (UID: \"c1e7407b-e78a-4b5c-b34f-44f5a24bc56f\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:28.349096 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.349072    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:28.635239 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.635204    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 19:32:28.641857 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:32:28.641825    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1e7407b_e78a_4b5c_b34f_44f5a24bc56f.slice/crio-a37c3c430157107ae5f107a59aa1bcc2cbb37f2ba625c90badf57d872fcdaadc WatchSource:0}: Error finding container a37c3c430157107ae5f107a59aa1bcc2cbb37f2ba625c90badf57d872fcdaadc: Status 404 returned error can't find the container with id a37c3c430157107ae5f107a59aa1bcc2cbb37f2ba625c90badf57d872fcdaadc
Oct 30 19:32:28.789194 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.789159    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="06b45974-f6ef-4b3a-b354-8103d7895fcf" path="/var/lib/kubelet/pods/06b45974-f6ef-4b3a-b354-8103d7895fcf/volumes"
Oct 30 19:32:28.960315 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.960235    6996 generic.go:356] "Generic (PLEG): container finished" podID="c1e7407b-e78a-4b5c-b34f-44f5a24bc56f" containerID="7558a0a30a1f0cbc5cd43343a740b3ef4c4bd991e29a8e38fcf76d7970ca9cff" exitCode=0
Oct 30 19:32:28.960315 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.960290    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c1e7407b-e78a-4b5c-b34f-44f5a24bc56f","Type":"ContainerDied","Data":"7558a0a30a1f0cbc5cd43343a740b3ef4c4bd991e29a8e38fcf76d7970ca9cff"}
Oct 30 19:32:28.960533 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:28.960338    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c1e7407b-e78a-4b5c-b34f-44f5a24bc56f","Type":"ContainerStarted","Data":"a37c3c430157107ae5f107a59aa1bcc2cbb37f2ba625c90badf57d872fcdaadc"}
Oct 30 19:32:29.243698 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:29.243666    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5cf7f796c9-hsqwz"
Oct 30 19:32:29.243804 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:29.243774    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5cf7f796c9-hsqwz"
Oct 30 19:32:29.248618 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:29.248594    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5cf7f796c9-hsqwz"
Oct 30 19:32:29.967972 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:29.967937    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c1e7407b-e78a-4b5c-b34f-44f5a24bc56f","Type":"ContainerStarted","Data":"8d8ebf540a04dad5b17333b09d03dc18d31937d33808ead9d4b6fef80306f8fd"}
Oct 30 19:32:29.968384 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:29.968002    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c1e7407b-e78a-4b5c-b34f-44f5a24bc56f","Type":"ContainerStarted","Data":"48afd2c0b1b12caf8807769ebc95e020ad7e377545b0dae7548bbd40d8fec3c0"}
Oct 30 19:32:29.968384 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:29.968031    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c1e7407b-e78a-4b5c-b34f-44f5a24bc56f","Type":"ContainerStarted","Data":"98983446e2c9ffbd8c63d18e39043ae9ea0c89ba7f9c4a6732496a4160f908ba"}
Oct 30 19:32:29.968384 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:29.968055    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c1e7407b-e78a-4b5c-b34f-44f5a24bc56f","Type":"ContainerStarted","Data":"7f93f3e79c9a75fd5d74b821715261f9e2fe35e7a9f837a949fd3d4fb6f2c6c4"}
Oct 30 19:32:29.968384 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:29.968079    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c1e7407b-e78a-4b5c-b34f-44f5a24bc56f","Type":"ContainerStarted","Data":"d32a7b7197ec190c89dab4a7567bf6b84bc89263957aa009ad44833c9ffde4f0"}
Oct 30 19:32:29.968384 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:29.968100    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c1e7407b-e78a-4b5c-b34f-44f5a24bc56f","Type":"ContainerStarted","Data":"c2b48489ca9cb8a589efeebe647f74ab794fbc97676d2f8b59d5fd39cd17aef8"}
Oct 30 19:32:29.971755 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:29.971737    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5cf7f796c9-hsqwz"
Oct 30 19:32:30.003621 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:30.003571    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.989714992 podStartE2EDuration="3.003558773s" totalImagesPullingTime="0s" totalInitContainerRuntime="13.843781ms" isStatefulPod=false podCreationTimestamp="2026-10-30 19:32:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:32:30.002647823 +0000 UTC m=+25.330765978" watchObservedRunningTime="2026-10-30 19:32:30.003558773 +0000 UTC m=+25.331676926"
Oct 30 19:32:30.068975 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:30.068945    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-55994d7c46-sz78s"]
Oct 30 19:32:30.778908 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:30.778869    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-7c4f845644-d8mc5" podUID="c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3" containerName="console" containerID="cri-o://fca9ef27e3b7963e624d67c1297ec7b88e24a10df49955387f9a99896554fbc7" gracePeriod=15
Oct 30 19:32:30.782435 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:30.782408    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-6479499c8f-ptgvc" podUID="1f3e7563-7109-45a5-9a77-7676a0697bd1" containerName="console" containerID="cri-o://8e54b01508a82f4d9f02eb3b249f74aa8e7595af64edf0a09d57075abfbc7fbe" gracePeriod=15
Oct 30 19:32:30.975445 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:30.975409    6996 generic.go:356] "Generic (PLEG): container finished" podID="1f3e7563-7109-45a5-9a77-7676a0697bd1" containerID="8e54b01508a82f4d9f02eb3b249f74aa8e7595af64edf0a09d57075abfbc7fbe" exitCode=0
Oct 30 19:32:30.975874 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:30.975461    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6479499c8f-ptgvc" event={"ID":"1f3e7563-7109-45a5-9a77-7676a0697bd1","Type":"ContainerDied","Data":"8e54b01508a82f4d9f02eb3b249f74aa8e7595af64edf0a09d57075abfbc7fbe"}
Oct 30 19:32:30.977382 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:30.977357    6996 generic.go:356] "Generic (PLEG): container finished" podID="c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3" containerID="fca9ef27e3b7963e624d67c1297ec7b88e24a10df49955387f9a99896554fbc7" exitCode=0
Oct 30 19:32:30.977510 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:30.977421    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c4f845644-d8mc5" event={"ID":"c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3","Type":"ContainerDied","Data":"fca9ef27e3b7963e624d67c1297ec7b88e24a10df49955387f9a99896554fbc7"}
Oct 30 19:32:31.036916 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:31.036889    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c4f845644-d8mc5"
Oct 30 19:32:31.041714 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:31.041698    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6479499c8f-ptgvc"
Oct 30 19:32:31.129725 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:31.129695    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1f3e7563-7109-45a5-9a77-7676a0697bd1-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f3e7563-7109-45a5-9a77-7676a0697bd1-service-ca\") pod \"1f3e7563-7109-45a5-9a77-7676a0697bd1\" (UID: \"1f3e7563-7109-45a5-9a77-7676a0697bd1\") "
Oct 30 19:32:31.129887 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:31.129732    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-service-ca\" (UniqueName: \"kubernetes.io/configmap/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-service-ca\") pod \"c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3\" (UID: \"c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3\") "
Oct 30 19:32:31.129887 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:31.129771    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-console-serving-cert\") pod \"c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3\" (UID: \"c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3\") "
Oct 30 19:32:31.129887 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:31.129793    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-kube-api-access-f9wcq\" (UniqueName: \"kubernetes.io/projected/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-kube-api-access-f9wcq\") pod \"c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3\" (UID: \"c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3\") "
Oct 30 19:32:31.129887 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:31.129818    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1f3e7563-7109-45a5-9a77-7676a0697bd1-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1f3e7563-7109-45a5-9a77-7676a0697bd1-console-oauth-config\") pod \"1f3e7563-7109-45a5-9a77-7676a0697bd1\" (UID: \"1f3e7563-7109-45a5-9a77-7676a0697bd1\") "
Oct 30 19:32:31.129887 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:31.129839    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1f3e7563-7109-45a5-9a77-7676a0697bd1-tmp\" (UniqueName: \"kubernetes.io/empty-dir/1f3e7563-7109-45a5-9a77-7676a0697bd1-tmp\") pod \"1f3e7563-7109-45a5-9a77-7676a0697bd1\" (UID: \"1f3e7563-7109-45a5-9a77-7676a0697bd1\") "
Oct 30 19:32:31.129887 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:31.129871    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1f3e7563-7109-45a5-9a77-7676a0697bd1-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f3e7563-7109-45a5-9a77-7676a0697bd1-trusted-ca-bundle\") pod \"1f3e7563-7109-45a5-9a77-7676a0697bd1\" (UID: \"1f3e7563-7109-45a5-9a77-7676a0697bd1\") "
Oct 30 19:32:31.130172 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:31.129905    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-tmp\" (UniqueName: \"kubernetes.io/empty-dir/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-tmp\") pod \"c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3\" (UID: \"c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3\") "
Oct 30 19:32:31.130172 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:31.129944    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-oauth-serving-cert\") pod \"c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3\" (UID: \"c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3\") "
Oct 30 19:32:31.130172 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:31.129978    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-console-config\" (UniqueName: \"kubernetes.io/configmap/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-console-config\") pod \"c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3\" (UID: \"c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3\") "
Oct 30 19:32:31.130172 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:31.130034    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1f3e7563-7109-45a5-9a77-7676a0697bd1-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1f3e7563-7109-45a5-9a77-7676a0697bd1-oauth-serving-cert\") pod \"1f3e7563-7109-45a5-9a77-7676a0697bd1\" (UID: \"1f3e7563-7109-45a5-9a77-7676a0697bd1\") "
Oct 30 19:32:31.130172 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:31.130062    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1f3e7563-7109-45a5-9a77-7676a0697bd1-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1f3e7563-7109-45a5-9a77-7676a0697bd1-console-serving-cert\") pod \"1f3e7563-7109-45a5-9a77-7676a0697bd1\" (UID: \"1f3e7563-7109-45a5-9a77-7676a0697bd1\") "
Oct 30 19:32:31.130172 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:31.130094    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-trusted-ca-bundle\") pod \"c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3\" (UID: \"c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3\") "
Oct 30 19:32:31.130172 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:31.130132    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1f3e7563-7109-45a5-9a77-7676a0697bd1-kube-api-access-2mqpt\" (UniqueName: \"kubernetes.io/projected/1f3e7563-7109-45a5-9a77-7676a0697bd1-kube-api-access-2mqpt\") pod \"1f3e7563-7109-45a5-9a77-7676a0697bd1\" (UID: \"1f3e7563-7109-45a5-9a77-7676a0697bd1\") "
Oct 30 19:32:31.130172 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:31.130158    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1f3e7563-7109-45a5-9a77-7676a0697bd1-console-config\" (UniqueName: \"kubernetes.io/configmap/1f3e7563-7109-45a5-9a77-7676a0697bd1-console-config\") pod \"1f3e7563-7109-45a5-9a77-7676a0697bd1\" (UID: \"1f3e7563-7109-45a5-9a77-7676a0697bd1\") "
Oct 30 19:32:31.130576 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:31.130194    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-console-oauth-config\") pod \"c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3\" (UID: \"c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3\") "
Oct 30 19:32:31.130576 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:31.130196    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f3e7563-7109-45a5-9a77-7676a0697bd1-service-ca" pod "1f3e7563-7109-45a5-9a77-7676a0697bd1" (UID: "1f3e7563-7109-45a5-9a77-7676a0697bd1"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:32:31.130576 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:31.130381    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-service-ca" pod "c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3" (UID: "c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:32:31.130576 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:31.130528    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-tmp" pod "c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3" (UID: "c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:31.130576 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:31.130551    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-console-config" pod "c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3" (UID: "c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:32:31.130804 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:31.130753    6996 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-tmp\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:31.130804 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:31.130780    6996 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-console-config\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:31.130804 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:31.130797    6996 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1f3e7563-7109-45a5-9a77-7676a0697bd1-service-ca\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:31.130962 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:31.130811    6996 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-service-ca\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:31.130962 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:31.130857    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-trusted-ca-bundle" pod "c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3" (UID: "c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:32:31.130962 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:31.130922    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f3e7563-7109-45a5-9a77-7676a0697bd1-tmp" pod "1f3e7563-7109-45a5-9a77-7676a0697bd1" (UID: "1f3e7563-7109-45a5-9a77-7676a0697bd1"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:31.131499 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:31.131454    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-oauth-serving-cert" pod "c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3" (UID: "c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:32:31.131607 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:31.131508    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f3e7563-7109-45a5-9a77-7676a0697bd1-trusted-ca-bundle" pod "1f3e7563-7109-45a5-9a77-7676a0697bd1" (UID: "1f3e7563-7109-45a5-9a77-7676a0697bd1"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:32:31.131673 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:31.131624    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f3e7563-7109-45a5-9a77-7676a0697bd1-oauth-serving-cert" pod "1f3e7563-7109-45a5-9a77-7676a0697bd1" (UID: "1f3e7563-7109-45a5-9a77-7676a0697bd1"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:32:31.132029 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:31.131990    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f3e7563-7109-45a5-9a77-7676a0697bd1-console-config" pod "1f3e7563-7109-45a5-9a77-7676a0697bd1" (UID: "1f3e7563-7109-45a5-9a77-7676a0697bd1"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:32:31.132576 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:31.132557    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f3e7563-7109-45a5-9a77-7676a0697bd1-console-oauth-config" pod "1f3e7563-7109-45a5-9a77-7676a0697bd1" (UID: "1f3e7563-7109-45a5-9a77-7676a0697bd1"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:32:31.132655 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:31.132591    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-console-serving-cert" pod "c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3" (UID: "c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:32:31.132655 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:31.132607    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-kube-api-access-f9wcq" pod "c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3" (UID: "c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3"). InnerVolumeSpecName "kube-api-access-f9wcq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:32:31.133133 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:31.133114    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f3e7563-7109-45a5-9a77-7676a0697bd1-kube-api-access-2mqpt" pod "1f3e7563-7109-45a5-9a77-7676a0697bd1" (UID: "1f3e7563-7109-45a5-9a77-7676a0697bd1"). InnerVolumeSpecName "kube-api-access-2mqpt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:32:31.133419 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:31.133400    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f3e7563-7109-45a5-9a77-7676a0697bd1-console-serving-cert" pod "1f3e7563-7109-45a5-9a77-7676a0697bd1" (UID: "1f3e7563-7109-45a5-9a77-7676a0697bd1"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:32:31.133515 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:31.133470    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-console-oauth-config" pod "c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3" (UID: "c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:32:31.187231 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:32:31.187194    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 19:32:31.231402 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:31.231371    6996 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-console-serving-cert\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:31.231402 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:31.231396    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f9wcq\" (UniqueName: \"kubernetes.io/projected/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-kube-api-access-f9wcq\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:31.231607 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:31.231412    6996 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1f3e7563-7109-45a5-9a77-7676a0697bd1-console-oauth-config\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:31.231607 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:31.231426    6996 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1f3e7563-7109-45a5-9a77-7676a0697bd1-tmp\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:31.231607 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:31.231439    6996 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f3e7563-7109-45a5-9a77-7676a0697bd1-trusted-ca-bundle\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:31.231607 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:31.231452    6996 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-oauth-serving-cert\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:31.231607 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:31.231465    6996 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1f3e7563-7109-45a5-9a77-7676a0697bd1-oauth-serving-cert\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:31.231607 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:31.231493    6996 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1f3e7563-7109-45a5-9a77-7676a0697bd1-console-serving-cert\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:31.231607 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:31.231508    6996 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-trusted-ca-bundle\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:31.231607 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:31.231522    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2mqpt\" (UniqueName: \"kubernetes.io/projected/1f3e7563-7109-45a5-9a77-7676a0697bd1-kube-api-access-2mqpt\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:31.231607 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:31.231535    6996 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1f3e7563-7109-45a5-9a77-7676a0697bd1-console-config\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:31.231607 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:31.231548    6996 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3-console-oauth-config\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:31.982972 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:31.982941    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6479499c8f-ptgvc"
Oct 30 19:32:31.982972 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:31.982959    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6479499c8f-ptgvc" event={"ID":"1f3e7563-7109-45a5-9a77-7676a0697bd1","Type":"ContainerDied","Data":"e841b85a1079cb72d30538e44a703671f61625f93f932c45d8658bb417102983"}
Oct 30 19:32:31.983473 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:31.983038    6996 scope.go:122] "RemoveContainer" containerID="8e54b01508a82f4d9f02eb3b249f74aa8e7595af64edf0a09d57075abfbc7fbe"
Oct 30 19:32:31.984503 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:31.984410    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c4f845644-d8mc5"
Oct 30 19:32:31.984503 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:31.984408    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c4f845644-d8mc5" event={"ID":"c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3","Type":"ContainerDied","Data":"54b8f021991e26a526673f6dff681f418709cea4abb400a0abbdfec24cf722be"}
Oct 30 19:32:31.992539 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:31.992520    6996 scope.go:122] "RemoveContainer" containerID="fca9ef27e3b7963e624d67c1297ec7b88e24a10df49955387f9a99896554fbc7"
Oct 30 19:32:32.019874 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:32.019830    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6479499c8f-ptgvc"]
Oct 30 19:32:32.024316 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:32.024291    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6479499c8f-ptgvc"]
Oct 30 19:32:32.043748 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:32.043688    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c4f845644-d8mc5"]
Oct 30 19:32:32.056317 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:32.056289    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7c4f845644-d8mc5"]
Oct 30 19:32:32.788674 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:32.788644    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1f3e7563-7109-45a5-9a77-7676a0697bd1" path="/var/lib/kubelet/pods/1f3e7563-7109-45a5-9a77-7676a0697bd1/volumes"
Oct 30 19:32:32.789046 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:32.789034    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3" path="/var/lib/kubelet/pods/c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3/volumes"
Oct 30 19:32:33.350270 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:33.350235    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:35.102553 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:35.102526    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-4f5sl"
Oct 30 19:32:35.219156 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:35.219117    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:35.234218 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:35.234194    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:36.017080 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:36.017053    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 19:32:55.088333 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:55.088276    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-55994d7c46-sz78s" podUID="63effc7d-50b8-425f-b9a8-891ce2259283" containerName="console" containerID="cri-o://097b3d016393a37abfc842ff60ecf422d2b2f744fcadf541920f245588f8ec01" gracePeriod=15
Oct 30 19:32:55.332072 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:55.332048    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55994d7c46-sz78s"
Oct 30 19:32:55.448420 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:55.448327    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/63effc7d-50b8-425f-b9a8-891ce2259283-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/63effc7d-50b8-425f-b9a8-891ce2259283-oauth-serving-cert\") pod \"63effc7d-50b8-425f-b9a8-891ce2259283\" (UID: \"63effc7d-50b8-425f-b9a8-891ce2259283\") "
Oct 30 19:32:55.448420 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:55.448375    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/63effc7d-50b8-425f-b9a8-891ce2259283-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/63effc7d-50b8-425f-b9a8-891ce2259283-console-serving-cert\") pod \"63effc7d-50b8-425f-b9a8-891ce2259283\" (UID: \"63effc7d-50b8-425f-b9a8-891ce2259283\") "
Oct 30 19:32:55.448420 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:55.448419    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/63effc7d-50b8-425f-b9a8-891ce2259283-console-config\" (UniqueName: \"kubernetes.io/configmap/63effc7d-50b8-425f-b9a8-891ce2259283-console-config\") pod \"63effc7d-50b8-425f-b9a8-891ce2259283\" (UID: \"63effc7d-50b8-425f-b9a8-891ce2259283\") "
Oct 30 19:32:55.448688 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:55.448547    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/63effc7d-50b8-425f-b9a8-891ce2259283-kube-api-access-q6kdm\" (UniqueName: \"kubernetes.io/projected/63effc7d-50b8-425f-b9a8-891ce2259283-kube-api-access-q6kdm\") pod \"63effc7d-50b8-425f-b9a8-891ce2259283\" (UID: \"63effc7d-50b8-425f-b9a8-891ce2259283\") "
Oct 30 19:32:55.448688 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:55.448606    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/63effc7d-50b8-425f-b9a8-891ce2259283-service-ca\" (UniqueName: \"kubernetes.io/configmap/63effc7d-50b8-425f-b9a8-891ce2259283-service-ca\") pod \"63effc7d-50b8-425f-b9a8-891ce2259283\" (UID: \"63effc7d-50b8-425f-b9a8-891ce2259283\") "
Oct 30 19:32:55.448688 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:55.448656    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/63effc7d-50b8-425f-b9a8-891ce2259283-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/63effc7d-50b8-425f-b9a8-891ce2259283-console-oauth-config\") pod \"63effc7d-50b8-425f-b9a8-891ce2259283\" (UID: \"63effc7d-50b8-425f-b9a8-891ce2259283\") "
Oct 30 19:32:55.448847 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:55.448706    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/63effc7d-50b8-425f-b9a8-891ce2259283-tmp\" (UniqueName: \"kubernetes.io/empty-dir/63effc7d-50b8-425f-b9a8-891ce2259283-tmp\") pod \"63effc7d-50b8-425f-b9a8-891ce2259283\" (UID: \"63effc7d-50b8-425f-b9a8-891ce2259283\") "
Oct 30 19:32:55.448847 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:55.448736    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/63effc7d-50b8-425f-b9a8-891ce2259283-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/63effc7d-50b8-425f-b9a8-891ce2259283-trusted-ca-bundle\") pod \"63effc7d-50b8-425f-b9a8-891ce2259283\" (UID: \"63effc7d-50b8-425f-b9a8-891ce2259283\") "
Oct 30 19:32:55.448847 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:55.448824    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63effc7d-50b8-425f-b9a8-891ce2259283-oauth-serving-cert" pod "63effc7d-50b8-425f-b9a8-891ce2259283" (UID: "63effc7d-50b8-425f-b9a8-891ce2259283"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:32:55.448847 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:55.448832    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63effc7d-50b8-425f-b9a8-891ce2259283-console-config" pod "63effc7d-50b8-425f-b9a8-891ce2259283" (UID: "63effc7d-50b8-425f-b9a8-891ce2259283"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:32:55.449215 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:55.449005    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63effc7d-50b8-425f-b9a8-891ce2259283-tmp" pod "63effc7d-50b8-425f-b9a8-891ce2259283" (UID: "63effc7d-50b8-425f-b9a8-891ce2259283"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:55.449215 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:55.449022    6996 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/63effc7d-50b8-425f-b9a8-891ce2259283-oauth-serving-cert\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:55.449215 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:55.449061    6996 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/63effc7d-50b8-425f-b9a8-891ce2259283-console-config\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:55.449215 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:55.449106    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63effc7d-50b8-425f-b9a8-891ce2259283-service-ca" pod "63effc7d-50b8-425f-b9a8-891ce2259283" (UID: "63effc7d-50b8-425f-b9a8-891ce2259283"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:32:55.449215 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:55.449208    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63effc7d-50b8-425f-b9a8-891ce2259283-trusted-ca-bundle" pod "63effc7d-50b8-425f-b9a8-891ce2259283" (UID: "63effc7d-50b8-425f-b9a8-891ce2259283"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:32:55.450859 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:55.450835    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63effc7d-50b8-425f-b9a8-891ce2259283-console-serving-cert" pod "63effc7d-50b8-425f-b9a8-891ce2259283" (UID: "63effc7d-50b8-425f-b9a8-891ce2259283"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:32:55.450951 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:55.450876    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63effc7d-50b8-425f-b9a8-891ce2259283-console-oauth-config" pod "63effc7d-50b8-425f-b9a8-891ce2259283" (UID: "63effc7d-50b8-425f-b9a8-891ce2259283"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:32:55.450994 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:55.450957    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63effc7d-50b8-425f-b9a8-891ce2259283-kube-api-access-q6kdm" pod "63effc7d-50b8-425f-b9a8-891ce2259283" (UID: "63effc7d-50b8-425f-b9a8-891ce2259283"). InnerVolumeSpecName "kube-api-access-q6kdm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:32:55.549560 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:55.549527    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q6kdm\" (UniqueName: \"kubernetes.io/projected/63effc7d-50b8-425f-b9a8-891ce2259283-kube-api-access-q6kdm\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:55.549560 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:55.549555    6996 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/63effc7d-50b8-425f-b9a8-891ce2259283-service-ca\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:55.549560 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:55.549566    6996 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/63effc7d-50b8-425f-b9a8-891ce2259283-console-oauth-config\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:55.549779 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:55.549575    6996 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/63effc7d-50b8-425f-b9a8-891ce2259283-tmp\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:55.549779 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:55.549584    6996 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/63effc7d-50b8-425f-b9a8-891ce2259283-trusted-ca-bundle\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:55.549779 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:55.549593    6996 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/63effc7d-50b8-425f-b9a8-891ce2259283-console-serving-cert\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:56.078425 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:56.078390    6996 generic.go:356] "Generic (PLEG): container finished" podID="63effc7d-50b8-425f-b9a8-891ce2259283" containerID="097b3d016393a37abfc842ff60ecf422d2b2f744fcadf541920f245588f8ec01" exitCode=0
Oct 30 19:32:56.078607 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:56.078434    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55994d7c46-sz78s"
Oct 30 19:32:56.078607 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:56.078462    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55994d7c46-sz78s" event={"ID":"63effc7d-50b8-425f-b9a8-891ce2259283","Type":"ContainerDied","Data":"097b3d016393a37abfc842ff60ecf422d2b2f744fcadf541920f245588f8ec01"}
Oct 30 19:32:56.078607 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:56.078523    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55994d7c46-sz78s" event={"ID":"63effc7d-50b8-425f-b9a8-891ce2259283","Type":"ContainerDied","Data":"b628691bb3e641012cc218b5b75787f95203955ab8fadc0de9627fe887df7842"}
Oct 30 19:32:56.078607 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:56.078544    6996 scope.go:122] "RemoveContainer" containerID="097b3d016393a37abfc842ff60ecf422d2b2f744fcadf541920f245588f8ec01"
Oct 30 19:32:56.087618 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:56.087602    6996 scope.go:122] "RemoveContainer" containerID="097b3d016393a37abfc842ff60ecf422d2b2f744fcadf541920f245588f8ec01"
Oct 30 19:32:56.087875 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:32:56.087857    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"097b3d016393a37abfc842ff60ecf422d2b2f744fcadf541920f245588f8ec01\": container with ID starting with 097b3d016393a37abfc842ff60ecf422d2b2f744fcadf541920f245588f8ec01 not found: ID does not exist" containerID="097b3d016393a37abfc842ff60ecf422d2b2f744fcadf541920f245588f8ec01"
Oct 30 19:32:56.087929 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:56.087882    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"097b3d016393a37abfc842ff60ecf422d2b2f744fcadf541920f245588f8ec01"} err="failed to get container status \"097b3d016393a37abfc842ff60ecf422d2b2f744fcadf541920f245588f8ec01\": rpc error: code = NotFound desc = could not find container \"097b3d016393a37abfc842ff60ecf422d2b2f744fcadf541920f245588f8ec01\": container with ID starting with 097b3d016393a37abfc842ff60ecf422d2b2f744fcadf541920f245588f8ec01 not found: ID does not exist"
Oct 30 19:32:56.110387 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:56.110362    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-55994d7c46-sz78s"]
Oct 30 19:32:56.119210 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:56.119164    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-55994d7c46-sz78s"]
Oct 30 19:32:56.788911 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:32:56.788878    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="63effc7d-50b8-425f-b9a8-891ce2259283" path="/var/lib/kubelet/pods/63effc7d-50b8-425f-b9a8-891ce2259283/volumes"
Oct 30 19:33:01.188782 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:33:01.188737    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 19:33:31.188431 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:33:31.188393    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 19:34:01.188472 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:34:01.188397    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 19:34:31.187973 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:34:31.187939    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 19:34:41.958840 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:41.958804    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c7lhg8"]
Oct 30 19:34:41.959311 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:41.959254    6996 cpu_manager.go:414] "containerMap: removing container" podUID="c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3" containerName="console"
Oct 30 19:34:41.959311 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:41.959274    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3" containerName="console"
Oct 30 19:34:41.959311 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:41.959290    6996 cpu_manager.go:414] "containerMap: removing container" podUID="63effc7d-50b8-425f-b9a8-891ce2259283" containerName="console"
Oct 30 19:34:41.959311 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:41.959296    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="63effc7d-50b8-425f-b9a8-891ce2259283" containerName="console"
Oct 30 19:34:41.959447 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:41.959318    6996 cpu_manager.go:414] "containerMap: removing container" podUID="1f3e7563-7109-45a5-9a77-7676a0697bd1" containerName="console"
Oct 30 19:34:41.959447 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:41.959323    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1f3e7563-7109-45a5-9a77-7676a0697bd1" containerName="console"
Oct 30 19:34:41.959447 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:41.959383    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="1f3e7563-7109-45a5-9a77-7676a0697bd1" containerName="console"
Oct 30 19:34:41.959447 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:41.959393    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="c59434a3-f0f2-4807-bd5b-65b9b2bbc2e3" containerName="console"
Oct 30 19:34:41.959447 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:41.959401    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="63effc7d-50b8-425f-b9a8-891ce2259283" containerName="console"
Oct 30 19:34:41.961529 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:41.961508    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c7lhg8"
Oct 30 19:34:41.969781 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:41.969762    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Oct 30 19:34:41.970070 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:41.970056    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Oct 30 19:34:41.970115 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:41.970099    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c7537007944577f64df34960ace3c69136c416-139ec370-dockercfg-866xw\""
Oct 30 19:34:41.976899 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:41.976861    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c7lhg8"]
Oct 30 19:34:42.034503 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:42.034452    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ef844086-9e23-40dc-8828-148909797ed5-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c7lhg8\" (UID: \"ef844086-9e23-40dc-8828-148909797ed5\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c7lhg8"
Oct 30 19:34:42.034691 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:42.034615    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ef844086-9e23-40dc-8828-148909797ed5-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c7lhg8\" (UID: \"ef844086-9e23-40dc-8828-148909797ed5\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c7lhg8"
Oct 30 19:34:42.034691 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:42.034668    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdhdg\" (UniqueName: \"kubernetes.io/projected/ef844086-9e23-40dc-8828-148909797ed5-kube-api-access-mdhdg\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c7lhg8\" (UID: \"ef844086-9e23-40dc-8828-148909797ed5\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c7lhg8"
Oct 30 19:34:42.135520 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:42.135464    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ef844086-9e23-40dc-8828-148909797ed5-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c7lhg8\" (UID: \"ef844086-9e23-40dc-8828-148909797ed5\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c7lhg8"
Oct 30 19:34:42.135708 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:42.135535    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdhdg\" (UniqueName: \"kubernetes.io/projected/ef844086-9e23-40dc-8828-148909797ed5-kube-api-access-mdhdg\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c7lhg8\" (UID: \"ef844086-9e23-40dc-8828-148909797ed5\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c7lhg8"
Oct 30 19:34:42.135708 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:42.135562    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ef844086-9e23-40dc-8828-148909797ed5-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c7lhg8\" (UID: \"ef844086-9e23-40dc-8828-148909797ed5\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c7lhg8"
Oct 30 19:34:42.135882 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:42.135852    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ef844086-9e23-40dc-8828-148909797ed5-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c7lhg8\" (UID: \"ef844086-9e23-40dc-8828-148909797ed5\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c7lhg8"
Oct 30 19:34:42.135920 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:42.135881    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ef844086-9e23-40dc-8828-148909797ed5-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c7lhg8\" (UID: \"ef844086-9e23-40dc-8828-148909797ed5\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c7lhg8"
Oct 30 19:34:42.151078 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:42.151046    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdhdg\" (UniqueName: \"kubernetes.io/projected/ef844086-9e23-40dc-8828-148909797ed5-kube-api-access-mdhdg\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c7lhg8\" (UID: \"ef844086-9e23-40dc-8828-148909797ed5\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c7lhg8"
Oct 30 19:34:42.274676 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:42.274588    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c7lhg8"
Oct 30 19:34:42.548997 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:42.548912    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c7lhg8"]
Oct 30 19:34:42.553399 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:34:42.553369    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef844086_9e23_40dc_8828_148909797ed5.slice/crio-633eb085be5c32d25ee2cc1b3dcd498b956a4eec9606ea6fca4be1cf3f8b4edc WatchSource:0}: Error finding container 633eb085be5c32d25ee2cc1b3dcd498b956a4eec9606ea6fca4be1cf3f8b4edc: Status 404 returned error can't find the container with id 633eb085be5c32d25ee2cc1b3dcd498b956a4eec9606ea6fca4be1cf3f8b4edc
Oct 30 19:34:42.555445 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:42.555427    6996 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 19:34:43.456561 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:43.456521    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c7lhg8" event={"ID":"ef844086-9e23-40dc-8828-148909797ed5","Type":"ContainerStarted","Data":"633eb085be5c32d25ee2cc1b3dcd498b956a4eec9606ea6fca4be1cf3f8b4edc"}
Oct 30 19:34:43.782948 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:43.782856    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644qqb64"]
Oct 30 19:34:43.785778 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:43.785742    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644qqb64"
Oct 30 19:34:43.789807 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:43.789780    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c4959aeb960c8e20c88ac2de924a35b8221494-145dcd5a-dockercfg-sdk4b\""
Oct 30 19:34:43.798564 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:43.798537    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644qqb64"]
Oct 30 19:34:43.954259 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:43.954215    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhtg9\" (UniqueName: \"kubernetes.io/projected/db79bf16-8d00-45c6-9b06-3d4f9e261d6b-kube-api-access-hhtg9\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644qqb64\" (UID: \"db79bf16-8d00-45c6-9b06-3d4f9e261d6b\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644qqb64"
Oct 30 19:34:43.954457 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:43.954285    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/db79bf16-8d00-45c6-9b06-3d4f9e261d6b-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644qqb64\" (UID: \"db79bf16-8d00-45c6-9b06-3d4f9e261d6b\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644qqb64"
Oct 30 19:34:43.954556 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:43.954454    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/db79bf16-8d00-45c6-9b06-3d4f9e261d6b-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644qqb64\" (UID: \"db79bf16-8d00-45c6-9b06-3d4f9e261d6b\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644qqb64"
Oct 30 19:34:44.055896 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:44.055805    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhtg9\" (UniqueName: \"kubernetes.io/projected/db79bf16-8d00-45c6-9b06-3d4f9e261d6b-kube-api-access-hhtg9\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644qqb64\" (UID: \"db79bf16-8d00-45c6-9b06-3d4f9e261d6b\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644qqb64"
Oct 30 19:34:44.055896 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:44.055864    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/db79bf16-8d00-45c6-9b06-3d4f9e261d6b-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644qqb64\" (UID: \"db79bf16-8d00-45c6-9b06-3d4f9e261d6b\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644qqb64"
Oct 30 19:34:44.056134 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:44.055931    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/db79bf16-8d00-45c6-9b06-3d4f9e261d6b-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644qqb64\" (UID: \"db79bf16-8d00-45c6-9b06-3d4f9e261d6b\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644qqb64"
Oct 30 19:34:44.056318 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:44.056294    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/db79bf16-8d00-45c6-9b06-3d4f9e261d6b-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644qqb64\" (UID: \"db79bf16-8d00-45c6-9b06-3d4f9e261d6b\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644qqb64"
Oct 30 19:34:44.056394 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:44.056330    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/db79bf16-8d00-45c6-9b06-3d4f9e261d6b-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644qqb64\" (UID: \"db79bf16-8d00-45c6-9b06-3d4f9e261d6b\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644qqb64"
Oct 30 19:34:44.069544 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:44.069515    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhtg9\" (UniqueName: \"kubernetes.io/projected/db79bf16-8d00-45c6-9b06-3d4f9e261d6b-kube-api-access-hhtg9\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644qqb64\" (UID: \"db79bf16-8d00-45c6-9b06-3d4f9e261d6b\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644qqb64"
Oct 30 19:34:44.102253 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:44.101974    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644qqb64"
Oct 30 19:34:44.387889 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:44.387772    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644qqb64"]
Oct 30 19:34:44.392939 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:34:44.392904    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb79bf16_8d00_45c6_9b06_3d4f9e261d6b.slice/crio-0e0551902537c1e76a74735bcf4876da220955d4be8ee19f714af2e7c6c541d5 WatchSource:0}: Error finding container 0e0551902537c1e76a74735bcf4876da220955d4be8ee19f714af2e7c6c541d5: Status 404 returned error can't find the container with id 0e0551902537c1e76a74735bcf4876da220955d4be8ee19f714af2e7c6c541d5
Oct 30 19:34:44.462166 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:44.462121    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644qqb64" event={"ID":"db79bf16-8d00-45c6-9b06-3d4f9e261d6b","Type":"ContainerStarted","Data":"0e0551902537c1e76a74735bcf4876da220955d4be8ee19f714af2e7c6c541d5"}
Oct 30 19:34:46.085974 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:46.085937    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adzx9z"]
Oct 30 19:34:46.089791 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:46.089766    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adzx9z"
Oct 30 19:34:46.094709 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:46.094685    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"6541d93aaee335d6227a8778ab4e840bbe7f52-f98fbcf8-dockercfg-89wwn\""
Oct 30 19:34:46.107983 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:46.107949    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adzx9z"]
Oct 30 19:34:46.180106 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:46.180065    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/008d7f38-2a80-4901-af65-f08d2b88788c-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adzx9z\" (UID: \"008d7f38-2a80-4901-af65-f08d2b88788c\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adzx9z"
Oct 30 19:34:46.180294 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:46.180154    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/008d7f38-2a80-4901-af65-f08d2b88788c-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adzx9z\" (UID: \"008d7f38-2a80-4901-af65-f08d2b88788c\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adzx9z"
Oct 30 19:34:46.180294 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:46.180186    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrbzv\" (UniqueName: \"kubernetes.io/projected/008d7f38-2a80-4901-af65-f08d2b88788c-kube-api-access-nrbzv\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adzx9z\" (UID: \"008d7f38-2a80-4901-af65-f08d2b88788c\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adzx9z"
Oct 30 19:34:46.187663 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:46.187628    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cdrnw4"]
Oct 30 19:34:46.192908 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:46.192884    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cdrnw4"
Oct 30 19:34:46.211700 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:46.211647    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e3d011e69d9573cabb8c359e5ac709eb5ac240-be78d160-dockercfg-h8njm\""
Oct 30 19:34:46.212772 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:46.212739    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cdrnw4"]
Oct 30 19:34:46.280909 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:46.280862    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/008d7f38-2a80-4901-af65-f08d2b88788c-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adzx9z\" (UID: \"008d7f38-2a80-4901-af65-f08d2b88788c\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adzx9z"
Oct 30 19:34:46.281092 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:46.280988    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2416a420-aa57-4c6b-a17b-624f1451a046-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cdrnw4\" (UID: \"2416a420-aa57-4c6b-a17b-624f1451a046\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cdrnw4"
Oct 30 19:34:46.281092 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:46.281030    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdcn8\" (UniqueName: \"kubernetes.io/projected/2416a420-aa57-4c6b-a17b-624f1451a046-kube-api-access-bdcn8\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cdrnw4\" (UID: \"2416a420-aa57-4c6b-a17b-624f1451a046\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cdrnw4"
Oct 30 19:34:46.281092 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:46.281064    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2416a420-aa57-4c6b-a17b-624f1451a046-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cdrnw4\" (UID: \"2416a420-aa57-4c6b-a17b-624f1451a046\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cdrnw4"
Oct 30 19:34:46.281258 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:46.281235    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/008d7f38-2a80-4901-af65-f08d2b88788c-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adzx9z\" (UID: \"008d7f38-2a80-4901-af65-f08d2b88788c\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adzx9z"
Oct 30 19:34:46.281318 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:46.281280    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrbzv\" (UniqueName: \"kubernetes.io/projected/008d7f38-2a80-4901-af65-f08d2b88788c-kube-api-access-nrbzv\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adzx9z\" (UID: \"008d7f38-2a80-4901-af65-f08d2b88788c\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adzx9z"
Oct 30 19:34:46.281318 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:46.281302    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/008d7f38-2a80-4901-af65-f08d2b88788c-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adzx9z\" (UID: \"008d7f38-2a80-4901-af65-f08d2b88788c\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adzx9z"
Oct 30 19:34:46.281618 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:46.281598    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/008d7f38-2a80-4901-af65-f08d2b88788c-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adzx9z\" (UID: \"008d7f38-2a80-4901-af65-f08d2b88788c\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adzx9z"
Oct 30 19:34:46.286230 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:46.286207    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x5jts"]
Oct 30 19:34:46.290161 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:46.290135    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x5jts"
Oct 30 19:34:46.294538 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:46.294512    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrbzv\" (UniqueName: \"kubernetes.io/projected/008d7f38-2a80-4901-af65-f08d2b88788c-kube-api-access-nrbzv\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adzx9z\" (UID: \"008d7f38-2a80-4901-af65-f08d2b88788c\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adzx9z"
Oct 30 19:34:46.301520 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:46.301464    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"da112bd609bff87f7bacd2ba941225d7cff15c-bd73b2fc-dockercfg-7zfb6\""
Oct 30 19:34:46.301621 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:46.301517    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x5jts"]
Oct 30 19:34:46.382127 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:46.382031    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4fzn\" (UniqueName: \"kubernetes.io/projected/c5e94645-99f2-498d-bcda-57ea0d1d4aa9-kube-api-access-b4fzn\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x5jts\" (UID: \"c5e94645-99f2-498d-bcda-57ea0d1d4aa9\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x5jts"
Oct 30 19:34:46.382562 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:46.382154    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c5e94645-99f2-498d-bcda-57ea0d1d4aa9-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x5jts\" (UID: \"c5e94645-99f2-498d-bcda-57ea0d1d4aa9\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x5jts"
Oct 30 19:34:46.382562 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:46.382197    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c5e94645-99f2-498d-bcda-57ea0d1d4aa9-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x5jts\" (UID: \"c5e94645-99f2-498d-bcda-57ea0d1d4aa9\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x5jts"
Oct 30 19:34:46.382562 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:46.382243    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2416a420-aa57-4c6b-a17b-624f1451a046-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cdrnw4\" (UID: \"2416a420-aa57-4c6b-a17b-624f1451a046\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cdrnw4"
Oct 30 19:34:46.382562 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:46.382276    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdcn8\" (UniqueName: \"kubernetes.io/projected/2416a420-aa57-4c6b-a17b-624f1451a046-kube-api-access-bdcn8\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cdrnw4\" (UID: \"2416a420-aa57-4c6b-a17b-624f1451a046\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cdrnw4"
Oct 30 19:34:46.382562 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:46.382306    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2416a420-aa57-4c6b-a17b-624f1451a046-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cdrnw4\" (UID: \"2416a420-aa57-4c6b-a17b-624f1451a046\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cdrnw4"
Oct 30 19:34:46.382849 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:46.382775    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2416a420-aa57-4c6b-a17b-624f1451a046-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cdrnw4\" (UID: \"2416a420-aa57-4c6b-a17b-624f1451a046\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cdrnw4"
Oct 30 19:34:46.383116 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:46.383086    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2416a420-aa57-4c6b-a17b-624f1451a046-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cdrnw4\" (UID: \"2416a420-aa57-4c6b-a17b-624f1451a046\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cdrnw4"
Oct 30 19:34:46.398948 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:46.398886    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdcn8\" (UniqueName: \"kubernetes.io/projected/2416a420-aa57-4c6b-a17b-624f1451a046-kube-api-access-bdcn8\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cdrnw4\" (UID: \"2416a420-aa57-4c6b-a17b-624f1451a046\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cdrnw4"
Oct 30 19:34:46.408996 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:46.408968    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adzx9z"
Oct 30 19:34:46.483597 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:46.483163    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c5e94645-99f2-498d-bcda-57ea0d1d4aa9-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x5jts\" (UID: \"c5e94645-99f2-498d-bcda-57ea0d1d4aa9\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x5jts"
Oct 30 19:34:46.483597 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:46.483224    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4fzn\" (UniqueName: \"kubernetes.io/projected/c5e94645-99f2-498d-bcda-57ea0d1d4aa9-kube-api-access-b4fzn\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x5jts\" (UID: \"c5e94645-99f2-498d-bcda-57ea0d1d4aa9\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x5jts"
Oct 30 19:34:46.483597 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:46.483256    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c5e94645-99f2-498d-bcda-57ea0d1d4aa9-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x5jts\" (UID: \"c5e94645-99f2-498d-bcda-57ea0d1d4aa9\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x5jts"
Oct 30 19:34:46.483998 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:46.483860    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c5e94645-99f2-498d-bcda-57ea0d1d4aa9-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x5jts\" (UID: \"c5e94645-99f2-498d-bcda-57ea0d1d4aa9\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x5jts"
Oct 30 19:34:46.483998 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:46.483962    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c5e94645-99f2-498d-bcda-57ea0d1d4aa9-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x5jts\" (UID: \"c5e94645-99f2-498d-bcda-57ea0d1d4aa9\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x5jts"
Oct 30 19:34:46.496325 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:46.496298    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4fzn\" (UniqueName: \"kubernetes.io/projected/c5e94645-99f2-498d-bcda-57ea0d1d4aa9-kube-api-access-b4fzn\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x5jts\" (UID: \"c5e94645-99f2-498d-bcda-57ea0d1d4aa9\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x5jts"
Oct 30 19:34:46.511762 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:46.511543    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cdrnw4"
Oct 30 19:34:46.621891 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:46.621861    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x5jts"
Oct 30 19:34:46.702489 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:46.702439    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adzx9z"]
Oct 30 19:34:46.806592 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:46.806516    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cdrnw4"]
Oct 30 19:34:46.810869 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:34:46.810835    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2416a420_aa57_4c6b_a17b_624f1451a046.slice/crio-4f25a5ed9f857de2bcc37c957573b2f1137b424a21c93ddb957c6be71f1214ba WatchSource:0}: Error finding container 4f25a5ed9f857de2bcc37c957573b2f1137b424a21c93ddb957c6be71f1214ba: Status 404 returned error can't find the container with id 4f25a5ed9f857de2bcc37c957573b2f1137b424a21c93ddb957c6be71f1214ba
Oct 30 19:34:46.915026 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:46.914933    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x5jts"]
Oct 30 19:34:46.919765 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:34:46.919732    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5e94645_99f2_498d_bcda_57ea0d1d4aa9.slice/crio-34df93065a71a909ee8dbd4baa2c9c0ec7ceee29d64bacdc40adabfe690a701b WatchSource:0}: Error finding container 34df93065a71a909ee8dbd4baa2c9c0ec7ceee29d64bacdc40adabfe690a701b: Status 404 returned error can't find the container with id 34df93065a71a909ee8dbd4baa2c9c0ec7ceee29d64bacdc40adabfe690a701b
Oct 30 19:34:47.482512 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:47.481717    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adzx9z" event={"ID":"008d7f38-2a80-4901-af65-f08d2b88788c","Type":"ContainerStarted","Data":"018cd488f8683bbac918eb41de9712eae6769100a148f7a431f5b6a24d860ee5"}
Oct 30 19:34:47.485633 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:47.485558    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cdrnw4" event={"ID":"2416a420-aa57-4c6b-a17b-624f1451a046","Type":"ContainerStarted","Data":"4f25a5ed9f857de2bcc37c957573b2f1137b424a21c93ddb957c6be71f1214ba"}
Oct 30 19:34:47.513100 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:47.513061    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x5jts" event={"ID":"c5e94645-99f2-498d-bcda-57ea0d1d4aa9","Type":"ContainerStarted","Data":"34df93065a71a909ee8dbd4baa2c9c0ec7ceee29d64bacdc40adabfe690a701b"}
Oct 30 19:34:47.761183 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:47.759527    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be719mlk9"]
Oct 30 19:34:47.764722 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:47.764018    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be719mlk9"
Oct 30 19:34:47.768692 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:47.768649    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e644421527c169d876fdb59235c927986d6bd2-9d2815fc-dockercfg-pgm5r\""
Oct 30 19:34:47.778616 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:47.778554    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be719mlk9"]
Oct 30 19:34:47.906048 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:47.906014    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c5xlbj"]
Oct 30 19:34:47.913511 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:47.910162    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a7b5128e-f7c1-461f-a3f1-0c947208d0a5-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be719mlk9\" (UID: \"a7b5128e-f7c1-461f-a3f1-0c947208d0a5\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be719mlk9"
Oct 30 19:34:47.913511 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:47.910271    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n66sd\" (UniqueName: \"kubernetes.io/projected/a7b5128e-f7c1-461f-a3f1-0c947208d0a5-kube-api-access-n66sd\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be719mlk9\" (UID: \"a7b5128e-f7c1-461f-a3f1-0c947208d0a5\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be719mlk9"
Oct 30 19:34:47.913511 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:47.910333    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a7b5128e-f7c1-461f-a3f1-0c947208d0a5-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be719mlk9\" (UID: \"a7b5128e-f7c1-461f-a3f1-0c947208d0a5\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be719mlk9"
Oct 30 19:34:47.918507 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:47.918456    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c5xlbj"
Oct 30 19:34:47.928815 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:47.928728    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c5xlbj"]
Oct 30 19:34:48.012951 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:48.011757    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n66sd\" (UniqueName: \"kubernetes.io/projected/a7b5128e-f7c1-461f-a3f1-0c947208d0a5-kube-api-access-n66sd\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be719mlk9\" (UID: \"a7b5128e-f7c1-461f-a3f1-0c947208d0a5\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be719mlk9"
Oct 30 19:34:48.012951 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:48.011812    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a7b5128e-f7c1-461f-a3f1-0c947208d0a5-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be719mlk9\" (UID: \"a7b5128e-f7c1-461f-a3f1-0c947208d0a5\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be719mlk9"
Oct 30 19:34:48.012951 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:48.011844    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a7b5128e-f7c1-461f-a3f1-0c947208d0a5-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be719mlk9\" (UID: \"a7b5128e-f7c1-461f-a3f1-0c947208d0a5\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be719mlk9"
Oct 30 19:34:48.012951 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:48.012260    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a7b5128e-f7c1-461f-a3f1-0c947208d0a5-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be719mlk9\" (UID: \"a7b5128e-f7c1-461f-a3f1-0c947208d0a5\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be719mlk9"
Oct 30 19:34:48.012951 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:48.012838    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a7b5128e-f7c1-461f-a3f1-0c947208d0a5-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be719mlk9\" (UID: \"a7b5128e-f7c1-461f-a3f1-0c947208d0a5\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be719mlk9"
Oct 30 19:34:48.036647 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:48.036587    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n66sd\" (UniqueName: \"kubernetes.io/projected/a7b5128e-f7c1-461f-a3f1-0c947208d0a5-kube-api-access-n66sd\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be719mlk9\" (UID: \"a7b5128e-f7c1-461f-a3f1-0c947208d0a5\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be719mlk9"
Oct 30 19:34:48.089439 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:48.089400    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be719mlk9"
Oct 30 19:34:48.112766 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:48.112725    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ktb6v\" (UniqueName: \"kubernetes.io/projected/ff80e5e4-5ad0-40f6-ac21-91609e5b7c3b-kube-api-access-ktb6v\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c5xlbj\" (UID: \"ff80e5e4-5ad0-40f6-ac21-91609e5b7c3b\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c5xlbj"
Oct 30 19:34:48.112902 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:48.112857    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ff80e5e4-5ad0-40f6-ac21-91609e5b7c3b-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c5xlbj\" (UID: \"ff80e5e4-5ad0-40f6-ac21-91609e5b7c3b\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c5xlbj"
Oct 30 19:34:48.112987 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:48.112921    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ff80e5e4-5ad0-40f6-ac21-91609e5b7c3b-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c5xlbj\" (UID: \"ff80e5e4-5ad0-40f6-ac21-91609e5b7c3b\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c5xlbj"
Oct 30 19:34:48.214363 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:48.213964    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ff80e5e4-5ad0-40f6-ac21-91609e5b7c3b-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c5xlbj\" (UID: \"ff80e5e4-5ad0-40f6-ac21-91609e5b7c3b\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c5xlbj"
Oct 30 19:34:48.214363 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:48.214039    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ktb6v\" (UniqueName: \"kubernetes.io/projected/ff80e5e4-5ad0-40f6-ac21-91609e5b7c3b-kube-api-access-ktb6v\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c5xlbj\" (UID: \"ff80e5e4-5ad0-40f6-ac21-91609e5b7c3b\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c5xlbj"
Oct 30 19:34:48.214363 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:48.214144    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ff80e5e4-5ad0-40f6-ac21-91609e5b7c3b-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c5xlbj\" (UID: \"ff80e5e4-5ad0-40f6-ac21-91609e5b7c3b\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c5xlbj"
Oct 30 19:34:48.214727 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:48.214432    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ff80e5e4-5ad0-40f6-ac21-91609e5b7c3b-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c5xlbj\" (UID: \"ff80e5e4-5ad0-40f6-ac21-91609e5b7c3b\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c5xlbj"
Oct 30 19:34:48.215009 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:48.214951    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ff80e5e4-5ad0-40f6-ac21-91609e5b7c3b-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c5xlbj\" (UID: \"ff80e5e4-5ad0-40f6-ac21-91609e5b7c3b\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c5xlbj"
Oct 30 19:34:48.229064 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:48.228997    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktb6v\" (UniqueName: \"kubernetes.io/projected/ff80e5e4-5ad0-40f6-ac21-91609e5b7c3b-kube-api-access-ktb6v\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c5xlbj\" (UID: \"ff80e5e4-5ad0-40f6-ac21-91609e5b7c3b\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c5xlbj"
Oct 30 19:34:48.248022 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:48.247752    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c5xlbj"
Oct 30 19:34:48.410488 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:48.410440    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be719mlk9"]
Oct 30 19:34:48.520466 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:48.520348    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be719mlk9" event={"ID":"a7b5128e-f7c1-461f-a3f1-0c947208d0a5","Type":"ContainerStarted","Data":"03dcd6c0737aaea034e4fc3c743ad1befbe955666c6da3d50474e31c197a2916"}
Oct 30 19:34:48.559420 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:48.559344    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c5xlbj"]
Oct 30 19:34:49.534788 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:49.534682    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c5xlbj" event={"ID":"ff80e5e4-5ad0-40f6-ac21-91609e5b7c3b","Type":"ContainerStarted","Data":"566a2c50f778d6d7a03766256997e7d2c86979668c298a5d6c4b06fbbf288295"}
Oct 30 19:34:51.549498 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:51.548413    6996 generic.go:356] "Generic (PLEG): container finished" podID="ef844086-9e23-40dc-8828-148909797ed5" containerID="1f5e95fd3a4c007228fdbbab8b6b06bc5bcd0d027cb3592e67432faed54a9f71" exitCode=0
Oct 30 19:34:51.549498 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:51.548580    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c7lhg8" event={"ID":"ef844086-9e23-40dc-8828-148909797ed5","Type":"ContainerDied","Data":"1f5e95fd3a4c007228fdbbab8b6b06bc5bcd0d027cb3592e67432faed54a9f71"}
Oct 30 19:34:55.577820 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:55.577778    6996 generic.go:356] "Generic (PLEG): container finished" podID="c5e94645-99f2-498d-bcda-57ea0d1d4aa9" containerID="1368c24235bcdba67d2acc94ec73f43bff97b9f309c311976e1f8a3de0030a3d" exitCode=0
Oct 30 19:34:55.578201 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:55.577907    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x5jts" event={"ID":"c5e94645-99f2-498d-bcda-57ea0d1d4aa9","Type":"ContainerDied","Data":"1368c24235bcdba67d2acc94ec73f43bff97b9f309c311976e1f8a3de0030a3d"}
Oct 30 19:34:55.581252 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:55.581219    6996 generic.go:356] "Generic (PLEG): container finished" podID="008d7f38-2a80-4901-af65-f08d2b88788c" containerID="31107bc9461b06eb1bc2cf880b05030d194a4d85c7c7d04b4a6b7a9bfbff0d10" exitCode=0
Oct 30 19:34:55.581380 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:55.581316    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adzx9z" event={"ID":"008d7f38-2a80-4901-af65-f08d2b88788c","Type":"ContainerDied","Data":"31107bc9461b06eb1bc2cf880b05030d194a4d85c7c7d04b4a6b7a9bfbff0d10"}
Oct 30 19:34:55.585531 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:55.585006    6996 generic.go:356] "Generic (PLEG): container finished" podID="2416a420-aa57-4c6b-a17b-624f1451a046" containerID="b05750d0b9836c57e9fff4e4d34312665aec154aa890c76d901afbf5e25d7bfe" exitCode=0
Oct 30 19:34:55.585531 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:55.585034    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cdrnw4" event={"ID":"2416a420-aa57-4c6b-a17b-624f1451a046","Type":"ContainerDied","Data":"b05750d0b9836c57e9fff4e4d34312665aec154aa890c76d901afbf5e25d7bfe"}
Oct 30 19:34:56.589906 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:56.589879    6996 generic.go:356] "Generic (PLEG): container finished" podID="ff80e5e4-5ad0-40f6-ac21-91609e5b7c3b" containerID="7c472ba13a010ab795da16378318cbba46c7b41d83416101e4bda68c2afaa798" exitCode=0
Oct 30 19:34:56.590236 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:56.589950    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c5xlbj" event={"ID":"ff80e5e4-5ad0-40f6-ac21-91609e5b7c3b","Type":"ContainerDied","Data":"7c472ba13a010ab795da16378318cbba46c7b41d83416101e4bda68c2afaa798"}
Oct 30 19:34:56.592201 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:56.592179    6996 generic.go:356] "Generic (PLEG): container finished" podID="ef844086-9e23-40dc-8828-148909797ed5" containerID="7ffb701c9b7944c1b444590ccdc2eec63b0cc6af77b57f7af2b5afe2ed11a75b" exitCode=0
Oct 30 19:34:56.592311 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:56.592289    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c7lhg8" event={"ID":"ef844086-9e23-40dc-8828-148909797ed5","Type":"ContainerDied","Data":"7ffb701c9b7944c1b444590ccdc2eec63b0cc6af77b57f7af2b5afe2ed11a75b"}
Oct 30 19:34:56.597223 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:56.594792    6996 generic.go:356] "Generic (PLEG): container finished" podID="db79bf16-8d00-45c6-9b06-3d4f9e261d6b" containerID="aed903008f7ac7a503fb0f57bdbb6a763454000f9b9ae82b343db46ee124aa8a" exitCode=0
Oct 30 19:34:56.597223 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:56.594872    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644qqb64" event={"ID":"db79bf16-8d00-45c6-9b06-3d4f9e261d6b","Type":"ContainerDied","Data":"aed903008f7ac7a503fb0f57bdbb6a763454000f9b9ae82b343db46ee124aa8a"}
Oct 30 19:34:56.597223 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:56.596665    6996 generic.go:356] "Generic (PLEG): container finished" podID="a7b5128e-f7c1-461f-a3f1-0c947208d0a5" containerID="45ac2652484d72aeef86034839834a3a486a1041154008e8e50414e53c3f1de4" exitCode=0
Oct 30 19:34:56.597223 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:56.596710    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be719mlk9" event={"ID":"a7b5128e-f7c1-461f-a3f1-0c947208d0a5","Type":"ContainerDied","Data":"45ac2652484d72aeef86034839834a3a486a1041154008e8e50414e53c3f1de4"}
Oct 30 19:34:57.605017 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:57.604976    6996 generic.go:356] "Generic (PLEG): container finished" podID="c5e94645-99f2-498d-bcda-57ea0d1d4aa9" containerID="bb0916b626f590db143ad74f9b6244a07503dd7c636aa7809ddcc884dfc82ce3" exitCode=0
Oct 30 19:34:57.605624 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:57.605068    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x5jts" event={"ID":"c5e94645-99f2-498d-bcda-57ea0d1d4aa9","Type":"ContainerDied","Data":"bb0916b626f590db143ad74f9b6244a07503dd7c636aa7809ddcc884dfc82ce3"}
Oct 30 19:34:57.608805 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:57.608779    6996 generic.go:356] "Generic (PLEG): container finished" podID="a7b5128e-f7c1-461f-a3f1-0c947208d0a5" containerID="7ee9449fe6908e0edb269a1557908133d16a76824aea7db4491261454e521c31" exitCode=0
Oct 30 19:34:57.608930 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:57.608902    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be719mlk9" event={"ID":"a7b5128e-f7c1-461f-a3f1-0c947208d0a5","Type":"ContainerDied","Data":"7ee9449fe6908e0edb269a1557908133d16a76824aea7db4491261454e521c31"}
Oct 30 19:34:57.612761 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:57.612736    6996 generic.go:356] "Generic (PLEG): container finished" podID="ff80e5e4-5ad0-40f6-ac21-91609e5b7c3b" containerID="0696f66625f10b04b046cda89a5f6ecdc090944e43f0843a9fd890093329ad77" exitCode=0
Oct 30 19:34:57.612866 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:57.612814    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c5xlbj" event={"ID":"ff80e5e4-5ad0-40f6-ac21-91609e5b7c3b","Type":"ContainerDied","Data":"0696f66625f10b04b046cda89a5f6ecdc090944e43f0843a9fd890093329ad77"}
Oct 30 19:34:57.617532 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:57.617442    6996 generic.go:356] "Generic (PLEG): container finished" podID="008d7f38-2a80-4901-af65-f08d2b88788c" containerID="3ec675eace47dcd9c48cbb6e04797c87c5e7bdd5114969c20b52ec7142b8b5fa" exitCode=0
Oct 30 19:34:57.617532 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:57.617509    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adzx9z" event={"ID":"008d7f38-2a80-4901-af65-f08d2b88788c","Type":"ContainerDied","Data":"3ec675eace47dcd9c48cbb6e04797c87c5e7bdd5114969c20b52ec7142b8b5fa"}
Oct 30 19:34:57.620922 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:57.620897    6996 generic.go:356] "Generic (PLEG): container finished" podID="2416a420-aa57-4c6b-a17b-624f1451a046" containerID="58d3c0366a154ab6f04e568116eaf129da7b5bccaf42db37cd4baf0c5caa193f" exitCode=0
Oct 30 19:34:57.621019 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:34:57.620970    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cdrnw4" event={"ID":"2416a420-aa57-4c6b-a17b-624f1451a046","Type":"ContainerDied","Data":"58d3c0366a154ab6f04e568116eaf129da7b5bccaf42db37cd4baf0c5caa193f"}
Oct 30 19:35:00.652630 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:00.652309    6996 generic.go:356] "Generic (PLEG): container finished" podID="db79bf16-8d00-45c6-9b06-3d4f9e261d6b" containerID="d5d9b1e3e59731a4db43bf6866df02fc08258156c770939408d444110767ee15" exitCode=0
Oct 30 19:35:00.652630 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:00.652443    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644qqb64" event={"ID":"db79bf16-8d00-45c6-9b06-3d4f9e261d6b","Type":"ContainerDied","Data":"d5d9b1e3e59731a4db43bf6866df02fc08258156c770939408d444110767ee15"}
Oct 30 19:35:01.251364 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:35:01.251137    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 19:35:10.749981 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:10.749941    6996 generic.go:356] "Generic (PLEG): container finished" podID="ef844086-9e23-40dc-8828-148909797ed5" containerID="37312b7f25de073f44441c7900d521b3b64f836240d4fe626492384ce1aa4370" exitCode=0
Oct 30 19:35:10.750390 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:10.750078    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c7lhg8" event={"ID":"ef844086-9e23-40dc-8828-148909797ed5","Type":"ContainerDied","Data":"37312b7f25de073f44441c7900d521b3b64f836240d4fe626492384ce1aa4370"}
Oct 30 19:35:10.755689 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:10.755657    6996 generic.go:356] "Generic (PLEG): container finished" podID="db79bf16-8d00-45c6-9b06-3d4f9e261d6b" containerID="111b919645b5eacd036726a81197bac46f6073f6893c80537d280a72ed1b80a2" exitCode=0
Oct 30 19:35:10.755855 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:10.755731    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644qqb64" event={"ID":"db79bf16-8d00-45c6-9b06-3d4f9e261d6b","Type":"ContainerDied","Data":"111b919645b5eacd036726a81197bac46f6073f6893c80537d280a72ed1b80a2"}
Oct 30 19:35:11.979142 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:11.979016    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644qqb64"
Oct 30 19:35:12.015954 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:12.015915    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c7lhg8"
Oct 30 19:35:12.092147 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:12.092108    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/db79bf16-8d00-45c6-9b06-3d4f9e261d6b-kube-api-access-hhtg9\" (UniqueName: \"kubernetes.io/projected/db79bf16-8d00-45c6-9b06-3d4f9e261d6b-kube-api-access-hhtg9\") pod \"db79bf16-8d00-45c6-9b06-3d4f9e261d6b\" (UID: \"db79bf16-8d00-45c6-9b06-3d4f9e261d6b\") "
Oct 30 19:35:12.092342 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:12.092313    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/db79bf16-8d00-45c6-9b06-3d4f9e261d6b-util\" (UniqueName: \"kubernetes.io/empty-dir/db79bf16-8d00-45c6-9b06-3d4f9e261d6b-util\") pod \"db79bf16-8d00-45c6-9b06-3d4f9e261d6b\" (UID: \"db79bf16-8d00-45c6-9b06-3d4f9e261d6b\") "
Oct 30 19:35:12.092417 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:12.092399    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/db79bf16-8d00-45c6-9b06-3d4f9e261d6b-bundle\" (UniqueName: \"kubernetes.io/empty-dir/db79bf16-8d00-45c6-9b06-3d4f9e261d6b-bundle\") pod \"db79bf16-8d00-45c6-9b06-3d4f9e261d6b\" (UID: \"db79bf16-8d00-45c6-9b06-3d4f9e261d6b\") "
Oct 30 19:35:12.093029 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:12.092998    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db79bf16-8d00-45c6-9b06-3d4f9e261d6b-bundle" pod "db79bf16-8d00-45c6-9b06-3d4f9e261d6b" (UID: "db79bf16-8d00-45c6-9b06-3d4f9e261d6b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:35:12.095734 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:12.095705    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db79bf16-8d00-45c6-9b06-3d4f9e261d6b-kube-api-access-hhtg9" pod "db79bf16-8d00-45c6-9b06-3d4f9e261d6b" (UID: "db79bf16-8d00-45c6-9b06-3d4f9e261d6b"). InnerVolumeSpecName "kube-api-access-hhtg9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:35:12.098719 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:12.098684    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db79bf16-8d00-45c6-9b06-3d4f9e261d6b-util" pod "db79bf16-8d00-45c6-9b06-3d4f9e261d6b" (UID: "db79bf16-8d00-45c6-9b06-3d4f9e261d6b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:35:12.194047 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:12.194002    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ef844086-9e23-40dc-8828-148909797ed5-util\" (UniqueName: \"kubernetes.io/empty-dir/ef844086-9e23-40dc-8828-148909797ed5-util\") pod \"ef844086-9e23-40dc-8828-148909797ed5\" (UID: \"ef844086-9e23-40dc-8828-148909797ed5\") "
Oct 30 19:35:12.194243 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:12.194086    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ef844086-9e23-40dc-8828-148909797ed5-bundle\" (UniqueName: \"kubernetes.io/empty-dir/ef844086-9e23-40dc-8828-148909797ed5-bundle\") pod \"ef844086-9e23-40dc-8828-148909797ed5\" (UID: \"ef844086-9e23-40dc-8828-148909797ed5\") "
Oct 30 19:35:12.194243 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:12.194130    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ef844086-9e23-40dc-8828-148909797ed5-kube-api-access-mdhdg\" (UniqueName: \"kubernetes.io/projected/ef844086-9e23-40dc-8828-148909797ed5-kube-api-access-mdhdg\") pod \"ef844086-9e23-40dc-8828-148909797ed5\" (UID: \"ef844086-9e23-40dc-8828-148909797ed5\") "
Oct 30 19:35:12.194444 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:12.194420    6996 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/db79bf16-8d00-45c6-9b06-3d4f9e261d6b-util\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:12.194444 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:12.194441    6996 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/db79bf16-8d00-45c6-9b06-3d4f9e261d6b-bundle\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:12.194632 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:12.194458    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hhtg9\" (UniqueName: \"kubernetes.io/projected/db79bf16-8d00-45c6-9b06-3d4f9e261d6b-kube-api-access-hhtg9\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:12.195332 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:12.195280    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef844086-9e23-40dc-8828-148909797ed5-bundle" pod "ef844086-9e23-40dc-8828-148909797ed5" (UID: "ef844086-9e23-40dc-8828-148909797ed5"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:35:12.203039 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:12.203009    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef844086-9e23-40dc-8828-148909797ed5-util" pod "ef844086-9e23-40dc-8828-148909797ed5" (UID: "ef844086-9e23-40dc-8828-148909797ed5"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:35:12.205408 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:12.205383    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef844086-9e23-40dc-8828-148909797ed5-kube-api-access-mdhdg" pod "ef844086-9e23-40dc-8828-148909797ed5" (UID: "ef844086-9e23-40dc-8828-148909797ed5"). InnerVolumeSpecName "kube-api-access-mdhdg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:35:12.295122 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:12.295077    6996 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ef844086-9e23-40dc-8828-148909797ed5-util\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:12.295122 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:12.295116    6996 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ef844086-9e23-40dc-8828-148909797ed5-bundle\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:12.295122 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:12.295127    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mdhdg\" (UniqueName: \"kubernetes.io/projected/ef844086-9e23-40dc-8828-148909797ed5-kube-api-access-mdhdg\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:12.777887 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:12.777846    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644qqb64" event={"ID":"db79bf16-8d00-45c6-9b06-3d4f9e261d6b","Type":"ContainerDied","Data":"0e0551902537c1e76a74735bcf4876da220955d4be8ee19f714af2e7c6c541d5"}
Oct 30 19:35:12.778059 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:12.777933    6996 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0e0551902537c1e76a74735bcf4876da220955d4be8ee19f714af2e7c6c541d5"
Oct 30 19:35:12.778059 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:12.777864    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644qqb64"
Oct 30 19:35:12.780063 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:12.780035    6996 generic.go:356] "Generic (PLEG): container finished" podID="c5e94645-99f2-498d-bcda-57ea0d1d4aa9" containerID="ccdbbf0180fb25cf4dd8b0a2813fd13f0a40df803576687a57879c26620c976a" exitCode=0
Oct 30 19:35:12.780191 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:12.780115    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x5jts" event={"ID":"c5e94645-99f2-498d-bcda-57ea0d1d4aa9","Type":"ContainerDied","Data":"ccdbbf0180fb25cf4dd8b0a2813fd13f0a40df803576687a57879c26620c976a"}
Oct 30 19:35:12.782203 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:12.782179    6996 generic.go:356] "Generic (PLEG): container finished" podID="a7b5128e-f7c1-461f-a3f1-0c947208d0a5" containerID="c405b3bc35a2326f9d657c688383c5f10a08c70fc60bf28e2005136875f1dab0" exitCode=0
Oct 30 19:35:12.782286 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:12.782255    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be719mlk9" event={"ID":"a7b5128e-f7c1-461f-a3f1-0c947208d0a5","Type":"ContainerDied","Data":"c405b3bc35a2326f9d657c688383c5f10a08c70fc60bf28e2005136875f1dab0"}
Oct 30 19:35:12.784468 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:12.784446    6996 generic.go:356] "Generic (PLEG): container finished" podID="ff80e5e4-5ad0-40f6-ac21-91609e5b7c3b" containerID="c61d99a4737a89cc1b9f6045702ea8107c03409a5616c1d18584e87ff4a225c6" exitCode=0
Oct 30 19:35:12.786967 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:12.786943    6996 generic.go:356] "Generic (PLEG): container finished" podID="008d7f38-2a80-4901-af65-f08d2b88788c" containerID="e5e12d685b7436501d88639f9ad884d292a57b46725a2476aa7fb612d56acf45" exitCode=0
Oct 30 19:35:12.788901 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:12.788880    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c7lhg8"
Oct 30 19:35:12.789331 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:12.789312    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c5xlbj" event={"ID":"ff80e5e4-5ad0-40f6-ac21-91609e5b7c3b","Type":"ContainerDied","Data":"c61d99a4737a89cc1b9f6045702ea8107c03409a5616c1d18584e87ff4a225c6"}
Oct 30 19:35:12.789431 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:12.789347    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adzx9z" event={"ID":"008d7f38-2a80-4901-af65-f08d2b88788c","Type":"ContainerDied","Data":"e5e12d685b7436501d88639f9ad884d292a57b46725a2476aa7fb612d56acf45"}
Oct 30 19:35:12.789431 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:12.789363    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c7lhg8" event={"ID":"ef844086-9e23-40dc-8828-148909797ed5","Type":"ContainerDied","Data":"633eb085be5c32d25ee2cc1b3dcd498b956a4eec9606ea6fca4be1cf3f8b4edc"}
Oct 30 19:35:12.789431 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:12.789382    6996 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="633eb085be5c32d25ee2cc1b3dcd498b956a4eec9606ea6fca4be1cf3f8b4edc"
Oct 30 19:35:12.791262 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:12.791178    6996 generic.go:356] "Generic (PLEG): container finished" podID="2416a420-aa57-4c6b-a17b-624f1451a046" containerID="fede3beaa421703d9444f9f1d05b0504f558692e11e506e9c7d406431a6fb00f" exitCode=0
Oct 30 19:35:12.791262 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:12.791214    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cdrnw4" event={"ID":"2416a420-aa57-4c6b-a17b-624f1451a046","Type":"ContainerDied","Data":"fede3beaa421703d9444f9f1d05b0504f558692e11e506e9c7d406431a6fb00f"}
Oct 30 19:35:13.936695 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:13.936669    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cdrnw4"
Oct 30 19:35:14.003564 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:14.003537    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be719mlk9"
Oct 30 19:35:14.004527 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:14.004508    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x5jts"
Oct 30 19:35:14.023128 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:14.023104    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c5xlbj"
Oct 30 19:35:14.025828 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:14.025807    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adzx9z"
Oct 30 19:35:14.111267 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:14.111232    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c5e94645-99f2-498d-bcda-57ea0d1d4aa9-util\" (UniqueName: \"kubernetes.io/empty-dir/c5e94645-99f2-498d-bcda-57ea0d1d4aa9-util\") pod \"c5e94645-99f2-498d-bcda-57ea0d1d4aa9\" (UID: \"c5e94645-99f2-498d-bcda-57ea0d1d4aa9\") "
Oct 30 19:35:14.111463 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:14.111283    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2416a420-aa57-4c6b-a17b-624f1451a046-kube-api-access-bdcn8\" (UniqueName: \"kubernetes.io/projected/2416a420-aa57-4c6b-a17b-624f1451a046-kube-api-access-bdcn8\") pod \"2416a420-aa57-4c6b-a17b-624f1451a046\" (UID: \"2416a420-aa57-4c6b-a17b-624f1451a046\") "
Oct 30 19:35:14.111463 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:14.111322    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c5e94645-99f2-498d-bcda-57ea0d1d4aa9-kube-api-access-b4fzn\" (UniqueName: \"kubernetes.io/projected/c5e94645-99f2-498d-bcda-57ea0d1d4aa9-kube-api-access-b4fzn\") pod \"c5e94645-99f2-498d-bcda-57ea0d1d4aa9\" (UID: \"c5e94645-99f2-498d-bcda-57ea0d1d4aa9\") "
Oct 30 19:35:14.111463 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:14.111342    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a7b5128e-f7c1-461f-a3f1-0c947208d0a5-bundle\" (UniqueName: \"kubernetes.io/empty-dir/a7b5128e-f7c1-461f-a3f1-0c947208d0a5-bundle\") pod \"a7b5128e-f7c1-461f-a3f1-0c947208d0a5\" (UID: \"a7b5128e-f7c1-461f-a3f1-0c947208d0a5\") "
Oct 30 19:35:14.111463 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:14.111385    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2416a420-aa57-4c6b-a17b-624f1451a046-bundle\" (UniqueName: \"kubernetes.io/empty-dir/2416a420-aa57-4c6b-a17b-624f1451a046-bundle\") pod \"2416a420-aa57-4c6b-a17b-624f1451a046\" (UID: \"2416a420-aa57-4c6b-a17b-624f1451a046\") "
Oct 30 19:35:14.111463 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:14.111457    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c5e94645-99f2-498d-bcda-57ea0d1d4aa9-bundle\" (UniqueName: \"kubernetes.io/empty-dir/c5e94645-99f2-498d-bcda-57ea0d1d4aa9-bundle\") pod \"c5e94645-99f2-498d-bcda-57ea0d1d4aa9\" (UID: \"c5e94645-99f2-498d-bcda-57ea0d1d4aa9\") "
Oct 30 19:35:14.111759 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:14.111504    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a7b5128e-f7c1-461f-a3f1-0c947208d0a5-util\" (UniqueName: \"kubernetes.io/empty-dir/a7b5128e-f7c1-461f-a3f1-0c947208d0a5-util\") pod \"a7b5128e-f7c1-461f-a3f1-0c947208d0a5\" (UID: \"a7b5128e-f7c1-461f-a3f1-0c947208d0a5\") "
Oct 30 19:35:14.111759 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:14.111539    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2416a420-aa57-4c6b-a17b-624f1451a046-util\" (UniqueName: \"kubernetes.io/empty-dir/2416a420-aa57-4c6b-a17b-624f1451a046-util\") pod \"2416a420-aa57-4c6b-a17b-624f1451a046\" (UID: \"2416a420-aa57-4c6b-a17b-624f1451a046\") "
Oct 30 19:35:14.111759 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:14.111582    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a7b5128e-f7c1-461f-a3f1-0c947208d0a5-kube-api-access-n66sd\" (UniqueName: \"kubernetes.io/projected/a7b5128e-f7c1-461f-a3f1-0c947208d0a5-kube-api-access-n66sd\") pod \"a7b5128e-f7c1-461f-a3f1-0c947208d0a5\" (UID: \"a7b5128e-f7c1-461f-a3f1-0c947208d0a5\") "
Oct 30 19:35:14.112130 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:14.112099    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7b5128e-f7c1-461f-a3f1-0c947208d0a5-bundle" pod "a7b5128e-f7c1-461f-a3f1-0c947208d0a5" (UID: "a7b5128e-f7c1-461f-a3f1-0c947208d0a5"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:35:14.112848 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:14.112814    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2416a420-aa57-4c6b-a17b-624f1451a046-bundle" pod "2416a420-aa57-4c6b-a17b-624f1451a046" (UID: "2416a420-aa57-4c6b-a17b-624f1451a046"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:35:14.113366 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:14.113336    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5e94645-99f2-498d-bcda-57ea0d1d4aa9-bundle" pod "c5e94645-99f2-498d-bcda-57ea0d1d4aa9" (UID: "c5e94645-99f2-498d-bcda-57ea0d1d4aa9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:35:14.114063 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:14.114030    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2416a420-aa57-4c6b-a17b-624f1451a046-kube-api-access-bdcn8" pod "2416a420-aa57-4c6b-a17b-624f1451a046" (UID: "2416a420-aa57-4c6b-a17b-624f1451a046"). InnerVolumeSpecName "kube-api-access-bdcn8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:35:14.114147 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:14.114109    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5e94645-99f2-498d-bcda-57ea0d1d4aa9-kube-api-access-b4fzn" pod "c5e94645-99f2-498d-bcda-57ea0d1d4aa9" (UID: "c5e94645-99f2-498d-bcda-57ea0d1d4aa9"). InnerVolumeSpecName "kube-api-access-b4fzn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:35:14.114357 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:14.114332    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7b5128e-f7c1-461f-a3f1-0c947208d0a5-kube-api-access-n66sd" pod "a7b5128e-f7c1-461f-a3f1-0c947208d0a5" (UID: "a7b5128e-f7c1-461f-a3f1-0c947208d0a5"). InnerVolumeSpecName "kube-api-access-n66sd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:35:14.118144 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:14.118121    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7b5128e-f7c1-461f-a3f1-0c947208d0a5-util" pod "a7b5128e-f7c1-461f-a3f1-0c947208d0a5" (UID: "a7b5128e-f7c1-461f-a3f1-0c947208d0a5"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:35:14.118196 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:14.118136    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5e94645-99f2-498d-bcda-57ea0d1d4aa9-util" pod "c5e94645-99f2-498d-bcda-57ea0d1d4aa9" (UID: "c5e94645-99f2-498d-bcda-57ea0d1d4aa9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:35:14.118681 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:14.118662    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2416a420-aa57-4c6b-a17b-624f1451a046-util" pod "2416a420-aa57-4c6b-a17b-624f1451a046" (UID: "2416a420-aa57-4c6b-a17b-624f1451a046"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:35:14.212448 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:14.212392    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/008d7f38-2a80-4901-af65-f08d2b88788c-bundle\" (UniqueName: \"kubernetes.io/empty-dir/008d7f38-2a80-4901-af65-f08d2b88788c-bundle\") pod \"008d7f38-2a80-4901-af65-f08d2b88788c\" (UID: \"008d7f38-2a80-4901-af65-f08d2b88788c\") "
Oct 30 19:35:14.212667 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:14.212471    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/008d7f38-2a80-4901-af65-f08d2b88788c-kube-api-access-nrbzv\" (UniqueName: \"kubernetes.io/projected/008d7f38-2a80-4901-af65-f08d2b88788c-kube-api-access-nrbzv\") pod \"008d7f38-2a80-4901-af65-f08d2b88788c\" (UID: \"008d7f38-2a80-4901-af65-f08d2b88788c\") "
Oct 30 19:35:14.212667 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:14.212530    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/008d7f38-2a80-4901-af65-f08d2b88788c-util\" (UniqueName: \"kubernetes.io/empty-dir/008d7f38-2a80-4901-af65-f08d2b88788c-util\") pod \"008d7f38-2a80-4901-af65-f08d2b88788c\" (UID: \"008d7f38-2a80-4901-af65-f08d2b88788c\") "
Oct 30 19:35:14.212667 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:14.212583    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ff80e5e4-5ad0-40f6-ac21-91609e5b7c3b-bundle\" (UniqueName: \"kubernetes.io/empty-dir/ff80e5e4-5ad0-40f6-ac21-91609e5b7c3b-bundle\") pod \"ff80e5e4-5ad0-40f6-ac21-91609e5b7c3b\" (UID: \"ff80e5e4-5ad0-40f6-ac21-91609e5b7c3b\") "
Oct 30 19:35:14.212667 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:14.212628    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ff80e5e4-5ad0-40f6-ac21-91609e5b7c3b-util\" (UniqueName: \"kubernetes.io/empty-dir/ff80e5e4-5ad0-40f6-ac21-91609e5b7c3b-util\") pod \"ff80e5e4-5ad0-40f6-ac21-91609e5b7c3b\" (UID: \"ff80e5e4-5ad0-40f6-ac21-91609e5b7c3b\") "
Oct 30 19:35:14.212860 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:14.212674    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ff80e5e4-5ad0-40f6-ac21-91609e5b7c3b-kube-api-access-ktb6v\" (UniqueName: \"kubernetes.io/projected/ff80e5e4-5ad0-40f6-ac21-91609e5b7c3b-kube-api-access-ktb6v\") pod \"ff80e5e4-5ad0-40f6-ac21-91609e5b7c3b\" (UID: \"ff80e5e4-5ad0-40f6-ac21-91609e5b7c3b\") "
Oct 30 19:35:14.212935 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:14.212916    6996 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2416a420-aa57-4c6b-a17b-624f1451a046-bundle\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:14.212998 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:14.212942    6996 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c5e94645-99f2-498d-bcda-57ea0d1d4aa9-bundle\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:14.212998 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:14.212956    6996 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a7b5128e-f7c1-461f-a3f1-0c947208d0a5-util\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:14.212998 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:14.212969    6996 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2416a420-aa57-4c6b-a17b-624f1451a046-util\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:14.212998 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:14.212983    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n66sd\" (UniqueName: \"kubernetes.io/projected/a7b5128e-f7c1-461f-a3f1-0c947208d0a5-kube-api-access-n66sd\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:14.213149 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:14.212999    6996 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c5e94645-99f2-498d-bcda-57ea0d1d4aa9-util\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:14.213149 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:14.213013    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bdcn8\" (UniqueName: \"kubernetes.io/projected/2416a420-aa57-4c6b-a17b-624f1451a046-kube-api-access-bdcn8\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:14.213149 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:14.213028    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b4fzn\" (UniqueName: \"kubernetes.io/projected/c5e94645-99f2-498d-bcda-57ea0d1d4aa9-kube-api-access-b4fzn\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:14.213149 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:14.213043    6996 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a7b5128e-f7c1-461f-a3f1-0c947208d0a5-bundle\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:14.213346 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:14.213317    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/008d7f38-2a80-4901-af65-f08d2b88788c-bundle" pod "008d7f38-2a80-4901-af65-f08d2b88788c" (UID: "008d7f38-2a80-4901-af65-f08d2b88788c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:35:14.213381 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:14.213340    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff80e5e4-5ad0-40f6-ac21-91609e5b7c3b-bundle" pod "ff80e5e4-5ad0-40f6-ac21-91609e5b7c3b" (UID: "ff80e5e4-5ad0-40f6-ac21-91609e5b7c3b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:35:14.215083 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:14.215049    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/008d7f38-2a80-4901-af65-f08d2b88788c-kube-api-access-nrbzv" pod "008d7f38-2a80-4901-af65-f08d2b88788c" (UID: "008d7f38-2a80-4901-af65-f08d2b88788c"). InnerVolumeSpecName "kube-api-access-nrbzv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:35:14.215170 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:14.215110    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff80e5e4-5ad0-40f6-ac21-91609e5b7c3b-kube-api-access-ktb6v" pod "ff80e5e4-5ad0-40f6-ac21-91609e5b7c3b" (UID: "ff80e5e4-5ad0-40f6-ac21-91609e5b7c3b"). InnerVolumeSpecName "kube-api-access-ktb6v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:35:14.218739 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:14.218717    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/008d7f38-2a80-4901-af65-f08d2b88788c-util" pod "008d7f38-2a80-4901-af65-f08d2b88788c" (UID: "008d7f38-2a80-4901-af65-f08d2b88788c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:35:14.218959 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:14.218939    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff80e5e4-5ad0-40f6-ac21-91609e5b7c3b-util" pod "ff80e5e4-5ad0-40f6-ac21-91609e5b7c3b" (UID: "ff80e5e4-5ad0-40f6-ac21-91609e5b7c3b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:35:14.313983 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:14.313933    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ktb6v\" (UniqueName: \"kubernetes.io/projected/ff80e5e4-5ad0-40f6-ac21-91609e5b7c3b-kube-api-access-ktb6v\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:14.313983 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:14.313981    6996 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/008d7f38-2a80-4901-af65-f08d2b88788c-bundle\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:14.313983 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:14.313993    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nrbzv\" (UniqueName: \"kubernetes.io/projected/008d7f38-2a80-4901-af65-f08d2b88788c-kube-api-access-nrbzv\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:14.314184 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:14.314002    6996 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/008d7f38-2a80-4901-af65-f08d2b88788c-util\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:14.314184 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:14.314011    6996 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ff80e5e4-5ad0-40f6-ac21-91609e5b7c3b-bundle\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:14.314184 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:14.314018    6996 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ff80e5e4-5ad0-40f6-ac21-91609e5b7c3b-util\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:14.802938 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:14.802919    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cdrnw4"
Oct 30 19:35:14.803213 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:14.802941    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cdrnw4" event={"ID":"2416a420-aa57-4c6b-a17b-624f1451a046","Type":"ContainerDied","Data":"4f25a5ed9f857de2bcc37c957573b2f1137b424a21c93ddb957c6be71f1214ba"}
Oct 30 19:35:14.803213 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:14.803001    6996 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4f25a5ed9f857de2bcc37c957573b2f1137b424a21c93ddb957c6be71f1214ba"
Oct 30 19:35:14.805464 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:14.805444    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x5jts"
Oct 30 19:35:14.805604 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:14.805443    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x5jts" event={"ID":"c5e94645-99f2-498d-bcda-57ea0d1d4aa9","Type":"ContainerDied","Data":"34df93065a71a909ee8dbd4baa2c9c0ec7ceee29d64bacdc40adabfe690a701b"}
Oct 30 19:35:14.805684 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:14.805605    6996 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="34df93065a71a909ee8dbd4baa2c9c0ec7ceee29d64bacdc40adabfe690a701b"
Oct 30 19:35:14.808630 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:14.808613    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be719mlk9"
Oct 30 19:35:14.808719 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:14.808653    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be719mlk9" event={"ID":"a7b5128e-f7c1-461f-a3f1-0c947208d0a5","Type":"ContainerDied","Data":"03dcd6c0737aaea034e4fc3c743ad1befbe955666c6da3d50474e31c197a2916"}
Oct 30 19:35:14.808719 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:14.808698    6996 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="03dcd6c0737aaea034e4fc3c743ad1befbe955666c6da3d50474e31c197a2916"
Oct 30 19:35:14.810625 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:14.810604    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c5xlbj"
Oct 30 19:35:14.810726 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:14.810633    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c5xlbj" event={"ID":"ff80e5e4-5ad0-40f6-ac21-91609e5b7c3b","Type":"ContainerDied","Data":"566a2c50f778d6d7a03766256997e7d2c86979668c298a5d6c4b06fbbf288295"}
Oct 30 19:35:14.810726 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:14.810672    6996 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="566a2c50f778d6d7a03766256997e7d2c86979668c298a5d6c4b06fbbf288295"
Oct 30 19:35:14.812659 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:14.812636    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adzx9z" event={"ID":"008d7f38-2a80-4901-af65-f08d2b88788c","Type":"ContainerDied","Data":"018cd488f8683bbac918eb41de9712eae6769100a148f7a431f5b6a24d860ee5"}
Oct 30 19:35:14.812746 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:14.812677    6996 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="018cd488f8683bbac918eb41de9712eae6769100a148f7a431f5b6a24d860ee5"
Oct 30 19:35:14.812746 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:14.812681    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adzx9z"
Oct 30 19:35:16.412324 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:16.412241    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-25jst"]
Oct 30 19:35:16.412740 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:16.412597    6996 cpu_manager.go:414] "containerMap: removing container" podUID="db79bf16-8d00-45c6-9b06-3d4f9e261d6b" containerName="extract"
Oct 30 19:35:16.412740 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:16.412611    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="db79bf16-8d00-45c6-9b06-3d4f9e261d6b" containerName="extract"
Oct 30 19:35:16.412740 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:16.412622    6996 cpu_manager.go:414] "containerMap: removing container" podUID="c5e94645-99f2-498d-bcda-57ea0d1d4aa9" containerName="util"
Oct 30 19:35:16.412740 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:16.412627    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c5e94645-99f2-498d-bcda-57ea0d1d4aa9" containerName="util"
Oct 30 19:35:16.412740 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:16.412633    6996 cpu_manager.go:414] "containerMap: removing container" podUID="ef844086-9e23-40dc-8828-148909797ed5" containerName="pull"
Oct 30 19:35:16.412740 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:16.412638    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ef844086-9e23-40dc-8828-148909797ed5" containerName="pull"
Oct 30 19:35:16.412740 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:16.412644    6996 cpu_manager.go:414] "containerMap: removing container" podUID="008d7f38-2a80-4901-af65-f08d2b88788c" containerName="pull"
Oct 30 19:35:16.412740 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:16.412649    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="008d7f38-2a80-4901-af65-f08d2b88788c" containerName="pull"
Oct 30 19:35:16.412740 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:16.412658    6996 cpu_manager.go:414] "containerMap: removing container" podUID="ef844086-9e23-40dc-8828-148909797ed5" containerName="util"
Oct 30 19:35:16.412740 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:16.412662    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ef844086-9e23-40dc-8828-148909797ed5" containerName="util"
Oct 30 19:35:16.412740 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:16.412667    6996 cpu_manager.go:414] "containerMap: removing container" podUID="db79bf16-8d00-45c6-9b06-3d4f9e261d6b" containerName="util"
Oct 30 19:35:16.412740 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:16.412672    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="db79bf16-8d00-45c6-9b06-3d4f9e261d6b" containerName="util"
Oct 30 19:35:16.412740 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:16.412677    6996 cpu_manager.go:414] "containerMap: removing container" podUID="a7b5128e-f7c1-461f-a3f1-0c947208d0a5" containerName="pull"
Oct 30 19:35:16.412740 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:16.412682    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a7b5128e-f7c1-461f-a3f1-0c947208d0a5" containerName="pull"
Oct 30 19:35:16.412740 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:16.412688    6996 cpu_manager.go:414] "containerMap: removing container" podUID="008d7f38-2a80-4901-af65-f08d2b88788c" containerName="extract"
Oct 30 19:35:16.412740 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:16.412692    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="008d7f38-2a80-4901-af65-f08d2b88788c" containerName="extract"
Oct 30 19:35:16.412740 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:16.412703    6996 cpu_manager.go:414] "containerMap: removing container" podUID="008d7f38-2a80-4901-af65-f08d2b88788c" containerName="util"
Oct 30 19:35:16.412740 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:16.412707    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="008d7f38-2a80-4901-af65-f08d2b88788c" containerName="util"
Oct 30 19:35:16.412740 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:16.412715    6996 cpu_manager.go:414] "containerMap: removing container" podUID="c5e94645-99f2-498d-bcda-57ea0d1d4aa9" containerName="pull"
Oct 30 19:35:16.412740 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:16.412720    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c5e94645-99f2-498d-bcda-57ea0d1d4aa9" containerName="pull"
Oct 30 19:35:16.412740 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:16.412726    6996 cpu_manager.go:414] "containerMap: removing container" podUID="c5e94645-99f2-498d-bcda-57ea0d1d4aa9" containerName="extract"
Oct 30 19:35:16.412740 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:16.412731    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c5e94645-99f2-498d-bcda-57ea0d1d4aa9" containerName="extract"
Oct 30 19:35:16.412740 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:16.412739    6996 cpu_manager.go:414] "containerMap: removing container" podUID="ff80e5e4-5ad0-40f6-ac21-91609e5b7c3b" containerName="util"
Oct 30 19:35:16.412740 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:16.412743    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ff80e5e4-5ad0-40f6-ac21-91609e5b7c3b" containerName="util"
Oct 30 19:35:16.412740 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:16.412750    6996 cpu_manager.go:414] "containerMap: removing container" podUID="ef844086-9e23-40dc-8828-148909797ed5" containerName="extract"
Oct 30 19:35:16.412740 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:16.412755    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ef844086-9e23-40dc-8828-148909797ed5" containerName="extract"
Oct 30 19:35:16.413443 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:16.412762    6996 cpu_manager.go:414] "containerMap: removing container" podUID="a7b5128e-f7c1-461f-a3f1-0c947208d0a5" containerName="extract"
Oct 30 19:35:16.413443 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:16.412766    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a7b5128e-f7c1-461f-a3f1-0c947208d0a5" containerName="extract"
Oct 30 19:35:16.413443 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:16.412775    6996 cpu_manager.go:414] "containerMap: removing container" podUID="a7b5128e-f7c1-461f-a3f1-0c947208d0a5" containerName="util"
Oct 30 19:35:16.413443 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:16.412779    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a7b5128e-f7c1-461f-a3f1-0c947208d0a5" containerName="util"
Oct 30 19:35:16.413443 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:16.412786    6996 cpu_manager.go:414] "containerMap: removing container" podUID="2416a420-aa57-4c6b-a17b-624f1451a046" containerName="util"
Oct 30 19:35:16.413443 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:16.412790    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2416a420-aa57-4c6b-a17b-624f1451a046" containerName="util"
Oct 30 19:35:16.413443 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:16.412800    6996 cpu_manager.go:414] "containerMap: removing container" podUID="2416a420-aa57-4c6b-a17b-624f1451a046" containerName="pull"
Oct 30 19:35:16.413443 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:16.412803    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2416a420-aa57-4c6b-a17b-624f1451a046" containerName="pull"
Oct 30 19:35:16.413443 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:16.412810    6996 cpu_manager.go:414] "containerMap: removing container" podUID="2416a420-aa57-4c6b-a17b-624f1451a046" containerName="extract"
Oct 30 19:35:16.413443 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:16.412814    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2416a420-aa57-4c6b-a17b-624f1451a046" containerName="extract"
Oct 30 19:35:16.413443 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:16.412821    6996 cpu_manager.go:414] "containerMap: removing container" podUID="ff80e5e4-5ad0-40f6-ac21-91609e5b7c3b" containerName="extract"
Oct 30 19:35:16.413443 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:16.412825    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ff80e5e4-5ad0-40f6-ac21-91609e5b7c3b" containerName="extract"
Oct 30 19:35:16.413443 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:16.412835    6996 cpu_manager.go:414] "containerMap: removing container" podUID="ff80e5e4-5ad0-40f6-ac21-91609e5b7c3b" containerName="pull"
Oct 30 19:35:16.413443 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:16.412840    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ff80e5e4-5ad0-40f6-ac21-91609e5b7c3b" containerName="pull"
Oct 30 19:35:16.413443 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:16.412845    6996 cpu_manager.go:414] "containerMap: removing container" podUID="db79bf16-8d00-45c6-9b06-3d4f9e261d6b" containerName="pull"
Oct 30 19:35:16.413443 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:16.412849    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="db79bf16-8d00-45c6-9b06-3d4f9e261d6b" containerName="pull"
Oct 30 19:35:16.413443 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:16.412901    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="ef844086-9e23-40dc-8828-148909797ed5" containerName="extract"
Oct 30 19:35:16.413443 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:16.412909    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="a7b5128e-f7c1-461f-a3f1-0c947208d0a5" containerName="extract"
Oct 30 19:35:16.413443 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:16.412916    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="2416a420-aa57-4c6b-a17b-624f1451a046" containerName="extract"
Oct 30 19:35:16.413443 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:16.412922    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="ff80e5e4-5ad0-40f6-ac21-91609e5b7c3b" containerName="extract"
Oct 30 19:35:16.413443 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:16.412927    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="db79bf16-8d00-45c6-9b06-3d4f9e261d6b" containerName="extract"
Oct 30 19:35:16.413443 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:16.412933    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="008d7f38-2a80-4901-af65-f08d2b88788c" containerName="extract"
Oct 30 19:35:16.413443 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:16.412939    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="c5e94645-99f2-498d-bcda-57ea0d1d4aa9" containerName="extract"
Oct 30 19:35:16.414820 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:16.414806    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-25jst"
Oct 30 19:35:16.425211 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:16.425193    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"jobset-operator-dockercfg-dbvk5\""
Oct 30 19:35:16.425668 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:16.425655    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"openshift-service-ca.crt\""
Oct 30 19:35:16.425713 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:16.425656    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"kube-root-ca.crt\""
Oct 30 19:35:16.438001 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:16.437977    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-25jst"]
Oct 30 19:35:16.533499 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:16.533453    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxlbk\" (UniqueName: \"kubernetes.io/projected/41877ae0-20b9-4c29-aecf-970748aea29c-kube-api-access-rxlbk\") pod \"jobset-operator-674cd59dc5-25jst\" (UID: \"41877ae0-20b9-4c29-aecf-970748aea29c\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-25jst"
Oct 30 19:35:16.533688 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:16.533512    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/41877ae0-20b9-4c29-aecf-970748aea29c-tmp\") pod \"jobset-operator-674cd59dc5-25jst\" (UID: \"41877ae0-20b9-4c29-aecf-970748aea29c\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-25jst"
Oct 30 19:35:16.635021 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:16.634975    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxlbk\" (UniqueName: \"kubernetes.io/projected/41877ae0-20b9-4c29-aecf-970748aea29c-kube-api-access-rxlbk\") pod \"jobset-operator-674cd59dc5-25jst\" (UID: \"41877ae0-20b9-4c29-aecf-970748aea29c\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-25jst"
Oct 30 19:35:16.635021 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:16.635024    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/41877ae0-20b9-4c29-aecf-970748aea29c-tmp\") pod \"jobset-operator-674cd59dc5-25jst\" (UID: \"41877ae0-20b9-4c29-aecf-970748aea29c\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-25jst"
Oct 30 19:35:16.635538 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:16.635518    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/41877ae0-20b9-4c29-aecf-970748aea29c-tmp\") pod \"jobset-operator-674cd59dc5-25jst\" (UID: \"41877ae0-20b9-4c29-aecf-970748aea29c\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-25jst"
Oct 30 19:35:16.647726 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:16.647701    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxlbk\" (UniqueName: \"kubernetes.io/projected/41877ae0-20b9-4c29-aecf-970748aea29c-kube-api-access-rxlbk\") pod \"jobset-operator-674cd59dc5-25jst\" (UID: \"41877ae0-20b9-4c29-aecf-970748aea29c\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-25jst"
Oct 30 19:35:16.733182 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:16.733095    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-25jst"
Oct 30 19:35:17.018766 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:17.018686    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-25jst"]
Oct 30 19:35:17.026517 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:35:17.026451    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod41877ae0_20b9_4c29_aecf_970748aea29c.slice/crio-2658a5f3c1a085315a912d7f913ea5f0e0b2cc3db29fad366a11a815f8fd0688 WatchSource:0}: Error finding container 2658a5f3c1a085315a912d7f913ea5f0e0b2cc3db29fad366a11a815f8fd0688: Status 404 returned error can't find the container with id 2658a5f3c1a085315a912d7f913ea5f0e0b2cc3db29fad366a11a815f8fd0688
Oct 30 19:35:17.833842 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:17.833810    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-25jst" event={"ID":"41877ae0-20b9-4c29-aecf-970748aea29c","Type":"ContainerStarted","Data":"2658a5f3c1a085315a912d7f913ea5f0e0b2cc3db29fad366a11a815f8fd0688"}
Oct 30 19:35:19.845606 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:19.845576    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-25jst" event={"ID":"41877ae0-20b9-4c29-aecf-970748aea29c","Type":"ContainerStarted","Data":"551155d5327841ceb4c0c4c70c0604f2fe2b7df0b3b4346683c71a3802b78e65"}
Oct 30 19:35:19.874017 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:19.873957    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-25jst" podStartSLOduration=1.166030126 podStartE2EDuration="3.873936403s" totalImagesPullingTime="2.707906277s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:35:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:35:19.872362904 +0000 UTC m=+195.200481057" watchObservedRunningTime="2026-10-30 19:35:19.873936403 +0000 UTC m=+195.202054557"
Oct 30 19:35:29.744318 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:29.744269    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-78f5b8588d-p5vzw"]
Oct 30 19:35:29.747157 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:29.747136    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78f5b8588d-p5vzw"
Oct 30 19:35:29.779950 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:29.779919    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-bw5vt"]
Oct 30 19:35:29.782055 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:29.782040    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-bw5vt"
Oct 30 19:35:29.795677 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:29.795624    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-78f5b8588d-p5vzw"]
Oct 30 19:35:29.797983 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:29.797956    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-bw5vt"]
Oct 30 19:35:29.830874 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:29.830842    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\""
Oct 30 19:35:29.830874 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:29.830874    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\""
Oct 30 19:35:29.831113 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:29.830984    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-p8q9s\""
Oct 30 19:35:29.831113 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:29.831029    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\""
Oct 30 19:35:29.831113 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:29.831045    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Oct 30 19:35:29.831113 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:29.831041    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\""
Oct 30 19:35:29.831113 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:29.831061    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\""
Oct 30 19:35:29.831113 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:29.831077    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Oct 30 19:35:29.831320 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:29.831169    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-98qhj\""
Oct 30 19:35:29.842596 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:29.842573    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/217b4788-1432-49e7-a6f5-da0f1e2520cd-tmp\") pod \"opendatahub-operator-controller-manager-78f5b8588d-p5vzw\" (UID: \"217b4788-1432-49e7-a6f5-da0f1e2520cd\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78f5b8588d-p5vzw"
Oct 30 19:35:29.842663 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:29.842606    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/217b4788-1432-49e7-a6f5-da0f1e2520cd-webhook-cert\") pod \"opendatahub-operator-controller-manager-78f5b8588d-p5vzw\" (UID: \"217b4788-1432-49e7-a6f5-da0f1e2520cd\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78f5b8588d-p5vzw"
Oct 30 19:35:29.842663 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:29.842630    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/217b4788-1432-49e7-a6f5-da0f1e2520cd-manifests\") pod \"opendatahub-operator-controller-manager-78f5b8588d-p5vzw\" (UID: \"217b4788-1432-49e7-a6f5-da0f1e2520cd\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78f5b8588d-p5vzw"
Oct 30 19:35:29.842736 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:29.842701    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86l6v\" (UniqueName: \"kubernetes.io/projected/217b4788-1432-49e7-a6f5-da0f1e2520cd-kube-api-access-86l6v\") pod \"opendatahub-operator-controller-manager-78f5b8588d-p5vzw\" (UID: \"217b4788-1432-49e7-a6f5-da0f1e2520cd\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78f5b8588d-p5vzw"
Oct 30 19:35:29.842781 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:29.842764    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qgqx\" (UniqueName: \"kubernetes.io/projected/5e7e4cc1-bbcd-40f9-9daf-0f2051e23a04-kube-api-access-8qgqx\") pod \"cert-manager-webhook-5dbbbf75cc-bw5vt\" (UID: \"5e7e4cc1-bbcd-40f9-9daf-0f2051e23a04\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-bw5vt"
Oct 30 19:35:29.842819 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:29.842804    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/217b4788-1432-49e7-a6f5-da0f1e2520cd-metrics-cert\") pod \"opendatahub-operator-controller-manager-78f5b8588d-p5vzw\" (UID: \"217b4788-1432-49e7-a6f5-da0f1e2520cd\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78f5b8588d-p5vzw"
Oct 30 19:35:29.842879 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:29.842860    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/217b4788-1432-49e7-a6f5-da0f1e2520cd-apiservice-cert\") pod \"opendatahub-operator-controller-manager-78f5b8588d-p5vzw\" (UID: \"217b4788-1432-49e7-a6f5-da0f1e2520cd\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78f5b8588d-p5vzw"
Oct 30 19:35:29.842958 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:29.842911    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5e7e4cc1-bbcd-40f9-9daf-0f2051e23a04-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-bw5vt\" (UID: \"5e7e4cc1-bbcd-40f9-9daf-0f2051e23a04\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-bw5vt"
Oct 30 19:35:29.943901 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:29.943859    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86l6v\" (UniqueName: \"kubernetes.io/projected/217b4788-1432-49e7-a6f5-da0f1e2520cd-kube-api-access-86l6v\") pod \"opendatahub-operator-controller-manager-78f5b8588d-p5vzw\" (UID: \"217b4788-1432-49e7-a6f5-da0f1e2520cd\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78f5b8588d-p5vzw"
Oct 30 19:35:29.944093 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:29.943912    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qgqx\" (UniqueName: \"kubernetes.io/projected/5e7e4cc1-bbcd-40f9-9daf-0f2051e23a04-kube-api-access-8qgqx\") pod \"cert-manager-webhook-5dbbbf75cc-bw5vt\" (UID: \"5e7e4cc1-bbcd-40f9-9daf-0f2051e23a04\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-bw5vt"
Oct 30 19:35:29.944093 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:29.943961    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/217b4788-1432-49e7-a6f5-da0f1e2520cd-metrics-cert\") pod \"opendatahub-operator-controller-manager-78f5b8588d-p5vzw\" (UID: \"217b4788-1432-49e7-a6f5-da0f1e2520cd\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78f5b8588d-p5vzw"
Oct 30 19:35:29.944093 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:29.943986    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/217b4788-1432-49e7-a6f5-da0f1e2520cd-apiservice-cert\") pod \"opendatahub-operator-controller-manager-78f5b8588d-p5vzw\" (UID: \"217b4788-1432-49e7-a6f5-da0f1e2520cd\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78f5b8588d-p5vzw"
Oct 30 19:35:29.944093 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:29.944018    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5e7e4cc1-bbcd-40f9-9daf-0f2051e23a04-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-bw5vt\" (UID: \"5e7e4cc1-bbcd-40f9-9daf-0f2051e23a04\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-bw5vt"
Oct 30 19:35:29.944093 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:29.944070    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/217b4788-1432-49e7-a6f5-da0f1e2520cd-tmp\") pod \"opendatahub-operator-controller-manager-78f5b8588d-p5vzw\" (UID: \"217b4788-1432-49e7-a6f5-da0f1e2520cd\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78f5b8588d-p5vzw"
Oct 30 19:35:29.944339 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:29.944099    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/217b4788-1432-49e7-a6f5-da0f1e2520cd-webhook-cert\") pod \"opendatahub-operator-controller-manager-78f5b8588d-p5vzw\" (UID: \"217b4788-1432-49e7-a6f5-da0f1e2520cd\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78f5b8588d-p5vzw"
Oct 30 19:35:29.944339 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:29.944130    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/217b4788-1432-49e7-a6f5-da0f1e2520cd-manifests\") pod \"opendatahub-operator-controller-manager-78f5b8588d-p5vzw\" (UID: \"217b4788-1432-49e7-a6f5-da0f1e2520cd\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78f5b8588d-p5vzw"
Oct 30 19:35:29.944648 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:29.944615    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/217b4788-1432-49e7-a6f5-da0f1e2520cd-manifests\") pod \"opendatahub-operator-controller-manager-78f5b8588d-p5vzw\" (UID: \"217b4788-1432-49e7-a6f5-da0f1e2520cd\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78f5b8588d-p5vzw"
Oct 30 19:35:29.946471 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:29.946439    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/217b4788-1432-49e7-a6f5-da0f1e2520cd-tmp\") pod \"opendatahub-operator-controller-manager-78f5b8588d-p5vzw\" (UID: \"217b4788-1432-49e7-a6f5-da0f1e2520cd\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78f5b8588d-p5vzw"
Oct 30 19:35:29.946693 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:29.946676    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/217b4788-1432-49e7-a6f5-da0f1e2520cd-metrics-cert\") pod \"opendatahub-operator-controller-manager-78f5b8588d-p5vzw\" (UID: \"217b4788-1432-49e7-a6f5-da0f1e2520cd\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78f5b8588d-p5vzw"
Oct 30 19:35:29.946766 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:29.946735    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/217b4788-1432-49e7-a6f5-da0f1e2520cd-webhook-cert\") pod \"opendatahub-operator-controller-manager-78f5b8588d-p5vzw\" (UID: \"217b4788-1432-49e7-a6f5-da0f1e2520cd\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78f5b8588d-p5vzw"
Oct 30 19:35:29.946816 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:29.946759    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/217b4788-1432-49e7-a6f5-da0f1e2520cd-apiservice-cert\") pod \"opendatahub-operator-controller-manager-78f5b8588d-p5vzw\" (UID: \"217b4788-1432-49e7-a6f5-da0f1e2520cd\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78f5b8588d-p5vzw"
Oct 30 19:35:29.963402 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:29.963377    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5e7e4cc1-bbcd-40f9-9daf-0f2051e23a04-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-bw5vt\" (UID: \"5e7e4cc1-bbcd-40f9-9daf-0f2051e23a04\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-bw5vt"
Oct 30 19:35:29.963864 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:29.963845    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-86l6v\" (UniqueName: \"kubernetes.io/projected/217b4788-1432-49e7-a6f5-da0f1e2520cd-kube-api-access-86l6v\") pod \"opendatahub-operator-controller-manager-78f5b8588d-p5vzw\" (UID: \"217b4788-1432-49e7-a6f5-da0f1e2520cd\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78f5b8588d-p5vzw"
Oct 30 19:35:29.964007 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:29.963938    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qgqx\" (UniqueName: \"kubernetes.io/projected/5e7e4cc1-bbcd-40f9-9daf-0f2051e23a04-kube-api-access-8qgqx\") pod \"cert-manager-webhook-5dbbbf75cc-bw5vt\" (UID: \"5e7e4cc1-bbcd-40f9-9daf-0f2051e23a04\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-bw5vt"
Oct 30 19:35:30.059780 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:30.059696    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78f5b8588d-p5vzw"
Oct 30 19:35:30.092234 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:30.092200    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-bw5vt"
Oct 30 19:35:30.390950 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:30.390922    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-bw5vt"]
Oct 30 19:35:30.392466 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:35:30.392440    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5e7e4cc1_bbcd_40f9_9daf_0f2051e23a04.slice/crio-14b33a849b5b80949ba168c43586535f903983c3324a30b4502dc90a00748517 WatchSource:0}: Error finding container 14b33a849b5b80949ba168c43586535f903983c3324a30b4502dc90a00748517: Status 404 returned error can't find the container with id 14b33a849b5b80949ba168c43586535f903983c3324a30b4502dc90a00748517
Oct 30 19:35:30.433908 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:30.433868    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-78f5b8588d-p5vzw"]
Oct 30 19:35:30.438941 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:35:30.438905    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod217b4788_1432_49e7_a6f5_da0f1e2520cd.slice/crio-4d09fb537fd4bd5f10df8d3dc9beef6fbf126d7e84a006f22a967426428a4595 WatchSource:0}: Error finding container 4d09fb537fd4bd5f10df8d3dc9beef6fbf126d7e84a006f22a967426428a4595: Status 404 returned error can't find the container with id 4d09fb537fd4bd5f10df8d3dc9beef6fbf126d7e84a006f22a967426428a4595
Oct 30 19:35:30.892817 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:30.892780    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-bw5vt" event={"ID":"5e7e4cc1-bbcd-40f9-9daf-0f2051e23a04","Type":"ContainerStarted","Data":"14b33a849b5b80949ba168c43586535f903983c3324a30b4502dc90a00748517"}
Oct 30 19:35:30.894627 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:30.894595    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78f5b8588d-p5vzw" event={"ID":"217b4788-1432-49e7-a6f5-da0f1e2520cd","Type":"ContainerStarted","Data":"4d09fb537fd4bd5f10df8d3dc9beef6fbf126d7e84a006f22a967426428a4595"}
Oct 30 19:35:31.203033 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:35:31.202939    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 19:35:33.005309 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:33.005266    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-4g2vw"]
Oct 30 19:35:33.008199 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:33.008171    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-4g2vw"
Oct 30 19:35:33.030496 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:33.030448    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-4g2vw"]
Oct 30 19:35:33.069971 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:33.069931    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-p2kkb\""
Oct 30 19:35:33.073429 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:33.073400    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvdwg\" (UniqueName: \"kubernetes.io/projected/c0c1accd-2b41-4d31-82f0-089d338caf4c-kube-api-access-nvdwg\") pod \"cert-manager-cainjector-5c57bbd4fc-4g2vw\" (UID: \"c0c1accd-2b41-4d31-82f0-089d338caf4c\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-4g2vw"
Oct 30 19:35:33.073595 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:33.073463    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c0c1accd-2b41-4d31-82f0-089d338caf4c-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-4g2vw\" (UID: \"c0c1accd-2b41-4d31-82f0-089d338caf4c\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-4g2vw"
Oct 30 19:35:33.174621 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:33.174582    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvdwg\" (UniqueName: \"kubernetes.io/projected/c0c1accd-2b41-4d31-82f0-089d338caf4c-kube-api-access-nvdwg\") pod \"cert-manager-cainjector-5c57bbd4fc-4g2vw\" (UID: \"c0c1accd-2b41-4d31-82f0-089d338caf4c\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-4g2vw"
Oct 30 19:35:33.174788 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:33.174658    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c0c1accd-2b41-4d31-82f0-089d338caf4c-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-4g2vw\" (UID: \"c0c1accd-2b41-4d31-82f0-089d338caf4c\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-4g2vw"
Oct 30 19:35:33.191740 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:33.191710    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c0c1accd-2b41-4d31-82f0-089d338caf4c-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-4g2vw\" (UID: \"c0c1accd-2b41-4d31-82f0-089d338caf4c\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-4g2vw"
Oct 30 19:35:33.191909 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:33.191818    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvdwg\" (UniqueName: \"kubernetes.io/projected/c0c1accd-2b41-4d31-82f0-089d338caf4c-kube-api-access-nvdwg\") pod \"cert-manager-cainjector-5c57bbd4fc-4g2vw\" (UID: \"c0c1accd-2b41-4d31-82f0-089d338caf4c\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-4g2vw"
Oct 30 19:35:33.323254 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:33.323218    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-4g2vw"
Oct 30 19:35:33.616841 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:33.616814    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-4g2vw"]
Oct 30 19:35:33.620472 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:35:33.620447    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc0c1accd_2b41_4d31_82f0_089d338caf4c.slice/crio-734ab70ec71a8c8d2452043ce7ff8cb6836fdb79c7bd5553d96e4692fc613f2f WatchSource:0}: Error finding container 734ab70ec71a8c8d2452043ce7ff8cb6836fdb79c7bd5553d96e4692fc613f2f: Status 404 returned error can't find the container with id 734ab70ec71a8c8d2452043ce7ff8cb6836fdb79c7bd5553d96e4692fc613f2f
Oct 30 19:35:33.911207 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:33.911123    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-4g2vw" event={"ID":"c0c1accd-2b41-4d31-82f0-089d338caf4c","Type":"ContainerStarted","Data":"734ab70ec71a8c8d2452043ce7ff8cb6836fdb79c7bd5553d96e4692fc613f2f"}
Oct 30 19:35:33.912684 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:33.912651    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-bw5vt" event={"ID":"5e7e4cc1-bbcd-40f9-9daf-0f2051e23a04","Type":"ContainerStarted","Data":"0628dd1f50d979386fd795abee562e0fd824cc21c059faab1dcf92a9d0edc530"}
Oct 30 19:35:33.912829 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:33.912701    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-bw5vt"
Oct 30 19:35:33.914211 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:33.914187    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78f5b8588d-p5vzw" event={"ID":"217b4788-1432-49e7-a6f5-da0f1e2520cd","Type":"ContainerStarted","Data":"27ca3375a1885df2d126d4ebd1e16ce503b34d36214247201ad7e3bd6e666762"}
Oct 30 19:35:33.937428 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:33.937372    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-bw5vt" podStartSLOduration=1.657509572 podStartE2EDuration="4.93735495s" totalImagesPullingTime="3.279845378s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:35:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:35:33.93655252 +0000 UTC m=+209.264670686" watchObservedRunningTime="2026-10-30 19:35:33.93735495 +0000 UTC m=+209.265473104"
Oct 30 19:35:34.920758 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:34.920713    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-4g2vw" event={"ID":"c0c1accd-2b41-4d31-82f0-089d338caf4c","Type":"ContainerStarted","Data":"ed17342a9ec6388ee1ff56b5c53b1b27c377d153789ab5ffbb085c6684a648cf"}
Oct 30 19:35:34.922405 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:34.922379    6996 generic.go:356] "Generic (PLEG): container finished" podID="217b4788-1432-49e7-a6f5-da0f1e2520cd" containerID="27ca3375a1885df2d126d4ebd1e16ce503b34d36214247201ad7e3bd6e666762" exitCode=0
Oct 30 19:35:34.922582 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:34.922451    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78f5b8588d-p5vzw" event={"ID":"217b4788-1432-49e7-a6f5-da0f1e2520cd","Type":"ContainerDied","Data":"27ca3375a1885df2d126d4ebd1e16ce503b34d36214247201ad7e3bd6e666762"}
Oct 30 19:35:34.937149 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:34.937111    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-4g2vw" podStartSLOduration=2.177901752 podStartE2EDuration="2.937096268s" totalImagesPullingTime="759.194516ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:35:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:35:34.936862328 +0000 UTC m=+210.264980482" watchObservedRunningTime="2026-10-30 19:35:34.937096268 +0000 UTC m=+210.265214422"
Oct 30 19:35:35.929746 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:35.929704    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78f5b8588d-p5vzw" event={"ID":"217b4788-1432-49e7-a6f5-da0f1e2520cd","Type":"ContainerStarted","Data":"5465dc2e0e58953d16b9cbf6a84391b1f7a5bd371837a9c6fbecf1b78e6c98dd"}
Oct 30 19:35:35.930186 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:35.930010    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78f5b8588d-p5vzw"
Oct 30 19:35:35.978944 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:35.978895    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78f5b8588d-p5vzw" podStartSLOduration=2.857701848 podStartE2EDuration="6.978880475s" totalImagesPullingTime="3.612013895s" totalInitContainerRuntime="509.164732ms" isStatefulPod=false podCreationTimestamp="2026-10-30 19:35:29 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:35:35.97753859 +0000 UTC m=+211.305656745" watchObservedRunningTime="2026-10-30 19:35:35.978880475 +0000 UTC m=+211.306998628"
Oct 30 19:35:39.924980 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:39.924943    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-bw5vt"
Oct 30 19:35:40.754652 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:40.754610    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-597b95cb84-ljcm7"]
Oct 30 19:35:40.761285 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:40.761255    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-597b95cb84-ljcm7"
Oct 30 19:35:40.766867 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:40.766843    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-kchq2\""
Oct 30 19:35:40.774758 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:40.774734    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-597b95cb84-ljcm7"]
Oct 30 19:35:40.841268 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:40.841228    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96brd\" (UniqueName: \"kubernetes.io/projected/1857e8be-daf9-4361-834a-178c88ce5ff0-kube-api-access-96brd\") pod \"cert-manager-597b95cb84-ljcm7\" (UID: \"1857e8be-daf9-4361-834a-178c88ce5ff0\") " pod="cert-manager/cert-manager-597b95cb84-ljcm7"
Oct 30 19:35:40.841440 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:40.841275    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1857e8be-daf9-4361-834a-178c88ce5ff0-bound-sa-token\") pod \"cert-manager-597b95cb84-ljcm7\" (UID: \"1857e8be-daf9-4361-834a-178c88ce5ff0\") " pod="cert-manager/cert-manager-597b95cb84-ljcm7"
Oct 30 19:35:40.942225 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:40.942188    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96brd\" (UniqueName: \"kubernetes.io/projected/1857e8be-daf9-4361-834a-178c88ce5ff0-kube-api-access-96brd\") pod \"cert-manager-597b95cb84-ljcm7\" (UID: \"1857e8be-daf9-4361-834a-178c88ce5ff0\") " pod="cert-manager/cert-manager-597b95cb84-ljcm7"
Oct 30 19:35:40.942680 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:40.942241    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1857e8be-daf9-4361-834a-178c88ce5ff0-bound-sa-token\") pod \"cert-manager-597b95cb84-ljcm7\" (UID: \"1857e8be-daf9-4361-834a-178c88ce5ff0\") " pod="cert-manager/cert-manager-597b95cb84-ljcm7"
Oct 30 19:35:40.955747 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:40.955719    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-96brd\" (UniqueName: \"kubernetes.io/projected/1857e8be-daf9-4361-834a-178c88ce5ff0-kube-api-access-96brd\") pod \"cert-manager-597b95cb84-ljcm7\" (UID: \"1857e8be-daf9-4361-834a-178c88ce5ff0\") " pod="cert-manager/cert-manager-597b95cb84-ljcm7"
Oct 30 19:35:40.956842 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:40.956822    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1857e8be-daf9-4361-834a-178c88ce5ff0-bound-sa-token\") pod \"cert-manager-597b95cb84-ljcm7\" (UID: \"1857e8be-daf9-4361-834a-178c88ce5ff0\") " pod="cert-manager/cert-manager-597b95cb84-ljcm7"
Oct 30 19:35:41.072402 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:41.072368    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-597b95cb84-ljcm7"
Oct 30 19:35:41.358875 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:41.358784    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-597b95cb84-ljcm7"]
Oct 30 19:35:41.362646 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:35:41.362618    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1857e8be_daf9_4361_834a_178c88ce5ff0.slice/crio-e8b8db827134e2d64c0c89bfc19e6f371897132a78932135ec04c753f4bfee2c WatchSource:0}: Error finding container e8b8db827134e2d64c0c89bfc19e6f371897132a78932135ec04c753f4bfee2c: Status 404 returned error can't find the container with id e8b8db827134e2d64c0c89bfc19e6f371897132a78932135ec04c753f4bfee2c
Oct 30 19:35:41.960428 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:41.960390    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-597b95cb84-ljcm7" event={"ID":"1857e8be-daf9-4361-834a-178c88ce5ff0","Type":"ContainerStarted","Data":"bc59075e25a2c860815090c8fb0a3a7dc8b86dd004f939a5583fb5e273d0f05a"}
Oct 30 19:35:41.960428 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:41.960434    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-597b95cb84-ljcm7" event={"ID":"1857e8be-daf9-4361-834a-178c88ce5ff0","Type":"ContainerStarted","Data":"e8b8db827134e2d64c0c89bfc19e6f371897132a78932135ec04c753f4bfee2c"}
Oct 30 19:35:41.988069 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:41.988017    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-597b95cb84-ljcm7" podStartSLOduration=1.988002396 podStartE2EDuration="1.988002396s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:35:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:35:41.984929993 +0000 UTC m=+217.313048146" watchObservedRunningTime="2026-10-30 19:35:41.988002396 +0000 UTC m=+217.316120550"
Oct 30 19:35:46.938737 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:46.938707    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78f5b8588d-p5vzw"
Oct 30 19:35:50.052592 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:50.052557    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-gmgtn"]
Oct 30 19:35:50.056152 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:50.056134    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-gmgtn"
Oct 30 19:35:50.061198 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:50.061176    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Oct 30 19:35:50.061779 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:50.061755    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Oct 30 19:35:50.061779 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:50.061777    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Oct 30 19:35:50.062028 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:50.062012    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Oct 30 19:35:50.062028 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:50.062025    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-kcgcv\""
Oct 30 19:35:50.107548 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:50.107513    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-gmgtn"]
Oct 30 19:35:50.124980 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:50.124936    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glqkj\" (UniqueName: \"kubernetes.io/projected/bede9e81-8025-4db4-b21d-91f8c4b233e0-kube-api-access-glqkj\") pod \"kuadrant-console-plugin-66c4fb575d-gmgtn\" (UID: \"bede9e81-8025-4db4-b21d-91f8c4b233e0\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-gmgtn"
Oct 30 19:35:50.125181 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:50.124990    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bede9e81-8025-4db4-b21d-91f8c4b233e0-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-gmgtn\" (UID: \"bede9e81-8025-4db4-b21d-91f8c4b233e0\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-gmgtn"
Oct 30 19:35:50.125181 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:50.125017    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/bede9e81-8025-4db4-b21d-91f8c4b233e0-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-gmgtn\" (UID: \"bede9e81-8025-4db4-b21d-91f8c4b233e0\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-gmgtn"
Oct 30 19:35:50.226535 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:50.226499    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glqkj\" (UniqueName: \"kubernetes.io/projected/bede9e81-8025-4db4-b21d-91f8c4b233e0-kube-api-access-glqkj\") pod \"kuadrant-console-plugin-66c4fb575d-gmgtn\" (UID: \"bede9e81-8025-4db4-b21d-91f8c4b233e0\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-gmgtn"
Oct 30 19:35:50.226743 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:50.226560    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bede9e81-8025-4db4-b21d-91f8c4b233e0-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-gmgtn\" (UID: \"bede9e81-8025-4db4-b21d-91f8c4b233e0\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-gmgtn"
Oct 30 19:35:50.226743 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:50.226589    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/bede9e81-8025-4db4-b21d-91f8c4b233e0-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-gmgtn\" (UID: \"bede9e81-8025-4db4-b21d-91f8c4b233e0\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-gmgtn"
Oct 30 19:35:50.227337 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:50.227309    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bede9e81-8025-4db4-b21d-91f8c4b233e0-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-gmgtn\" (UID: \"bede9e81-8025-4db4-b21d-91f8c4b233e0\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-gmgtn"
Oct 30 19:35:50.228931 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:50.228899    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/bede9e81-8025-4db4-b21d-91f8c4b233e0-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-gmgtn\" (UID: \"bede9e81-8025-4db4-b21d-91f8c4b233e0\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-gmgtn"
Oct 30 19:35:50.240964 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:50.240938    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-glqkj\" (UniqueName: \"kubernetes.io/projected/bede9e81-8025-4db4-b21d-91f8c4b233e0-kube-api-access-glqkj\") pod \"kuadrant-console-plugin-66c4fb575d-gmgtn\" (UID: \"bede9e81-8025-4db4-b21d-91f8c4b233e0\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-gmgtn"
Oct 30 19:35:50.367695 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:50.367661    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-gmgtn"
Oct 30 19:35:50.646450 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:50.646424    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-gmgtn"]
Oct 30 19:35:50.649216 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:35:50.649186    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbede9e81_8025_4db4_b21d_91f8c4b233e0.slice/crio-d90b22139095b5a382f245648afbf80bc0b08121f456dc92a080f5f900933fdf WatchSource:0}: Error finding container d90b22139095b5a382f245648afbf80bc0b08121f456dc92a080f5f900933fdf: Status 404 returned error can't find the container with id d90b22139095b5a382f245648afbf80bc0b08121f456dc92a080f5f900933fdf
Oct 30 19:35:51.003242 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:51.003158    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-gmgtn" event={"ID":"bede9e81-8025-4db4-b21d-91f8c4b233e0","Type":"ContainerStarted","Data":"d90b22139095b5a382f245648afbf80bc0b08121f456dc92a080f5f900933fdf"}
Oct 30 19:35:56.032778 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:56.032742    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-gmgtn" event={"ID":"bede9e81-8025-4db4-b21d-91f8c4b233e0","Type":"ContainerStarted","Data":"281091ed6cc5eb2d7e8019bd265bde615790397d7dd373556c2f19bd85141684"}
Oct 30 19:35:56.054224 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:56.054177    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-gmgtn" podStartSLOduration=1.232223109 podStartE2EDuration="6.054159432s" totalImagesPullingTime="4.821936323s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:35:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:35:56.053613059 +0000 UTC m=+231.381731213" watchObservedRunningTime="2026-10-30 19:35:56.054159432 +0000 UTC m=+231.382277587"
Oct 30 19:35:59.068683 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:59.068640    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5b99845f6f-qb5pq"]
Oct 30 19:35:59.072516 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:59.072473    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5b99845f6f-qb5pq"
Oct 30 19:35:59.083752 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:59.083729    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Oct 30 19:35:59.084005 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:59.083987    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Oct 30 19:35:59.084240 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:59.084226    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Oct 30 19:35:59.089245 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:59.089221    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5b99845f6f-qb5pq"]
Oct 30 19:35:59.183738 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:59.183707    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Oct 30 19:35:59.184298 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:59.184280    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-stlgn\""
Oct 30 19:35:59.211742 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:59.211704    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wgrlb\" (UniqueName: \"kubernetes.io/projected/b2e83193-5313-47da-96aa-d3e85a5d4e29-kube-api-access-wgrlb\") pod \"kube-auth-proxy-5b99845f6f-qb5pq\" (UID: \"b2e83193-5313-47da-96aa-d3e85a5d4e29\") " pod="openshift-ingress/kube-auth-proxy-5b99845f6f-qb5pq"
Oct 30 19:35:59.211742 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:59.211746    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b2e83193-5313-47da-96aa-d3e85a5d4e29-tmp\") pod \"kube-auth-proxy-5b99845f6f-qb5pq\" (UID: \"b2e83193-5313-47da-96aa-d3e85a5d4e29\") " pod="openshift-ingress/kube-auth-proxy-5b99845f6f-qb5pq"
Oct 30 19:35:59.211944 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:59.211894    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2e83193-5313-47da-96aa-d3e85a5d4e29-tls-certs\") pod \"kube-auth-proxy-5b99845f6f-qb5pq\" (UID: \"b2e83193-5313-47da-96aa-d3e85a5d4e29\") " pod="openshift-ingress/kube-auth-proxy-5b99845f6f-qb5pq"
Oct 30 19:35:59.313359 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:59.313316    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2e83193-5313-47da-96aa-d3e85a5d4e29-tls-certs\") pod \"kube-auth-proxy-5b99845f6f-qb5pq\" (UID: \"b2e83193-5313-47da-96aa-d3e85a5d4e29\") " pod="openshift-ingress/kube-auth-proxy-5b99845f6f-qb5pq"
Oct 30 19:35:59.313568 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:59.313373    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wgrlb\" (UniqueName: \"kubernetes.io/projected/b2e83193-5313-47da-96aa-d3e85a5d4e29-kube-api-access-wgrlb\") pod \"kube-auth-proxy-5b99845f6f-qb5pq\" (UID: \"b2e83193-5313-47da-96aa-d3e85a5d4e29\") " pod="openshift-ingress/kube-auth-proxy-5b99845f6f-qb5pq"
Oct 30 19:35:59.313568 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:59.313406    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b2e83193-5313-47da-96aa-d3e85a5d4e29-tmp\") pod \"kube-auth-proxy-5b99845f6f-qb5pq\" (UID: \"b2e83193-5313-47da-96aa-d3e85a5d4e29\") " pod="openshift-ingress/kube-auth-proxy-5b99845f6f-qb5pq"
Oct 30 19:35:59.315825 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:59.315797    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b2e83193-5313-47da-96aa-d3e85a5d4e29-tmp\") pod \"kube-auth-proxy-5b99845f6f-qb5pq\" (UID: \"b2e83193-5313-47da-96aa-d3e85a5d4e29\") " pod="openshift-ingress/kube-auth-proxy-5b99845f6f-qb5pq"
Oct 30 19:35:59.316033 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:59.316013    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2e83193-5313-47da-96aa-d3e85a5d4e29-tls-certs\") pod \"kube-auth-proxy-5b99845f6f-qb5pq\" (UID: \"b2e83193-5313-47da-96aa-d3e85a5d4e29\") " pod="openshift-ingress/kube-auth-proxy-5b99845f6f-qb5pq"
Oct 30 19:35:59.339681 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:59.339605    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wgrlb\" (UniqueName: \"kubernetes.io/projected/b2e83193-5313-47da-96aa-d3e85a5d4e29-kube-api-access-wgrlb\") pod \"kube-auth-proxy-5b99845f6f-qb5pq\" (UID: \"b2e83193-5313-47da-96aa-d3e85a5d4e29\") " pod="openshift-ingress/kube-auth-proxy-5b99845f6f-qb5pq"
Oct 30 19:35:59.385296 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:59.385258    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5b99845f6f-qb5pq"
Oct 30 19:35:59.689803 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:35:59.689719    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5b99845f6f-qb5pq"]
Oct 30 19:35:59.692695 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:35:59.692666    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2e83193_5313_47da_96aa_d3e85a5d4e29.slice/crio-5394273d33fd1c8ca22f64c0c4f2938940d5fd213d6486061d1bb3860e4cc665 WatchSource:0}: Error finding container 5394273d33fd1c8ca22f64c0c4f2938940d5fd213d6486061d1bb3860e4cc665: Status 404 returned error can't find the container with id 5394273d33fd1c8ca22f64c0c4f2938940d5fd213d6486061d1bb3860e4cc665
Oct 30 19:36:00.052796 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:00.052699    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5b99845f6f-qb5pq" event={"ID":"b2e83193-5313-47da-96aa-d3e85a5d4e29","Type":"ContainerStarted","Data":"5394273d33fd1c8ca22f64c0c4f2938940d5fd213d6486061d1bb3860e4cc665"}
Oct 30 19:36:00.626885 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:00.626843    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 19:36:00.630778 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:00.630751    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7c9b87d574-2kq47"
Oct 30 19:36:00.644099 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:00.644042    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Oct 30 19:36:00.645196 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:00.645129    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 19:36:00.725613 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:00.725523    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 19:36:00.726963 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:00.726935    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f1319006-f8c2-4f14-9d3d-e7d44737c884-config-file\") pod \"limitador-limitador-7c9b87d574-2kq47\" (UID: \"f1319006-f8c2-4f14-9d3d-e7d44737c884\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-2kq47"
Oct 30 19:36:00.727108 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:00.726975    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xz4mc\" (UniqueName: \"kubernetes.io/projected/f1319006-f8c2-4f14-9d3d-e7d44737c884-kube-api-access-xz4mc\") pod \"limitador-limitador-7c9b87d574-2kq47\" (UID: \"f1319006-f8c2-4f14-9d3d-e7d44737c884\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-2kq47"
Oct 30 19:36:00.828769 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:00.828731    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f1319006-f8c2-4f14-9d3d-e7d44737c884-config-file\") pod \"limitador-limitador-7c9b87d574-2kq47\" (UID: \"f1319006-f8c2-4f14-9d3d-e7d44737c884\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-2kq47"
Oct 30 19:36:00.828964 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:00.828780    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xz4mc\" (UniqueName: \"kubernetes.io/projected/f1319006-f8c2-4f14-9d3d-e7d44737c884-kube-api-access-xz4mc\") pod \"limitador-limitador-7c9b87d574-2kq47\" (UID: \"f1319006-f8c2-4f14-9d3d-e7d44737c884\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-2kq47"
Oct 30 19:36:00.829606 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:00.829576    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f1319006-f8c2-4f14-9d3d-e7d44737c884-config-file\") pod \"limitador-limitador-7c9b87d574-2kq47\" (UID: \"f1319006-f8c2-4f14-9d3d-e7d44737c884\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-2kq47"
Oct 30 19:36:00.874024 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:00.873989    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xz4mc\" (UniqueName: \"kubernetes.io/projected/f1319006-f8c2-4f14-9d3d-e7d44737c884-kube-api-access-xz4mc\") pod \"limitador-limitador-7c9b87d574-2kq47\" (UID: \"f1319006-f8c2-4f14-9d3d-e7d44737c884\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-2kq47"
Oct 30 19:36:00.948743 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:00.948666    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7c9b87d574-2kq47"
Oct 30 19:36:01.194815 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:36:01.194776    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 19:36:01.266564 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:01.265896    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 19:36:01.270916 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:36:01.270879    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1319006_f8c2_4f14_9d3d_e7d44737c884.slice/crio-abb5c5a5dc42f473cdd0c9227c5c7dad531339e5e85191103d51cd9af9b965c1 WatchSource:0}: Error finding container abb5c5a5dc42f473cdd0c9227c5c7dad531339e5e85191103d51cd9af9b965c1: Status 404 returned error can't find the container with id abb5c5a5dc42f473cdd0c9227c5c7dad531339e5e85191103d51cd9af9b965c1
Oct 30 19:36:01.635418 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:01.635382    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-59fbh"]
Oct 30 19:36:01.638980 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:01.638962    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-59fbh"
Oct 30 19:36:01.650241 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:01.650216    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-gj5tc\""
Oct 30 19:36:01.657041 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:01.657006    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-59fbh"]
Oct 30 19:36:01.735699 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:01.735663    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jq8n\" (UniqueName: \"kubernetes.io/projected/ae663b82-a97f-4d25-abfb-81c987110994-kube-api-access-5jq8n\") pod \"authorino-56c6c9ddd5-59fbh\" (UID: \"ae663b82-a97f-4d25-abfb-81c987110994\") " pod="kuadrant-system/authorino-56c6c9ddd5-59fbh"
Oct 30 19:36:01.837073 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:01.837029    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jq8n\" (UniqueName: \"kubernetes.io/projected/ae663b82-a97f-4d25-abfb-81c987110994-kube-api-access-5jq8n\") pod \"authorino-56c6c9ddd5-59fbh\" (UID: \"ae663b82-a97f-4d25-abfb-81c987110994\") " pod="kuadrant-system/authorino-56c6c9ddd5-59fbh"
Oct 30 19:36:01.854734 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:01.854701    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jq8n\" (UniqueName: \"kubernetes.io/projected/ae663b82-a97f-4d25-abfb-81c987110994-kube-api-access-5jq8n\") pod \"authorino-56c6c9ddd5-59fbh\" (UID: \"ae663b82-a97f-4d25-abfb-81c987110994\") " pod="kuadrant-system/authorino-56c6c9ddd5-59fbh"
Oct 30 19:36:01.955560 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:01.955463    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-59fbh"
Oct 30 19:36:02.065210 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:02.065174    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7c9b87d574-2kq47" event={"ID":"f1319006-f8c2-4f14-9d3d-e7d44737c884","Type":"ContainerStarted","Data":"abb5c5a5dc42f473cdd0c9227c5c7dad531339e5e85191103d51cd9af9b965c1"}
Oct 30 19:36:02.239903 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:02.239820    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-59fbh"]
Oct 30 19:36:02.243109 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:36:02.243082    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podae663b82_a97f_4d25_abfb_81c987110994.slice/crio-0bbb9d8af2e751f4ee9c15f7b37abc0f0fa6c0b5d45832c33c9cca5608473e16 WatchSource:0}: Error finding container 0bbb9d8af2e751f4ee9c15f7b37abc0f0fa6c0b5d45832c33c9cca5608473e16: Status 404 returned error can't find the container with id 0bbb9d8af2e751f4ee9c15f7b37abc0f0fa6c0b5d45832c33c9cca5608473e16
Oct 30 19:36:03.074945 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:03.074797    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5b99845f6f-qb5pq" event={"ID":"b2e83193-5313-47da-96aa-d3e85a5d4e29","Type":"ContainerStarted","Data":"9dfe5b594363a3ee66aee09e1405fbf5bf7373be1fc8651d7328c7e87edf6d94"}
Oct 30 19:36:03.078150 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:03.078116    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-59fbh" event={"ID":"ae663b82-a97f-4d25-abfb-81c987110994","Type":"ContainerStarted","Data":"0bbb9d8af2e751f4ee9c15f7b37abc0f0fa6c0b5d45832c33c9cca5608473e16"}
Oct 30 19:36:05.091887 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:05.091842    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-59fbh" event={"ID":"ae663b82-a97f-4d25-abfb-81c987110994","Type":"ContainerStarted","Data":"199ac20eb67543424adcdd8c338d5472ea9da5c5888ecca4321c08bc01dcbff1"}
Oct 30 19:36:05.093543 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:05.093514    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7c9b87d574-2kq47" event={"ID":"f1319006-f8c2-4f14-9d3d-e7d44737c884","Type":"ContainerStarted","Data":"deb41da68ff7be90122b230afe94e931d39f6ce1b87e457390a54815595a0691"}
Oct 30 19:36:05.093654 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:05.093616    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-7c9b87d574-2kq47"
Oct 30 19:36:05.119565 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:05.119511    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5b99845f6f-qb5pq" podStartSLOduration=3.792292089 podStartE2EDuration="6.119473358s" totalImagesPullingTime="2.327181269s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:35:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:36:03.104295329 +0000 UTC m=+238.432413483" watchObservedRunningTime="2026-10-30 19:36:05.119473358 +0000 UTC m=+240.447591509"
Oct 30 19:36:05.120470 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:05.120433    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-56c6c9ddd5-59fbh" podStartSLOduration=1.409813421 podStartE2EDuration="4.120426081s" totalImagesPullingTime="2.71061266s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:36:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:36:05.120239525 +0000 UTC m=+240.448357679" watchObservedRunningTime="2026-10-30 19:36:05.120426081 +0000 UTC m=+240.448544235"
Oct 30 19:36:05.157246 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:05.157189    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-7c9b87d574-2kq47" podStartSLOduration=2.345734944 podStartE2EDuration="5.157175357s" totalImagesPullingTime="2.811440413s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:36:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:36:05.15581239 +0000 UTC m=+240.483930544" watchObservedRunningTime="2026-10-30 19:36:05.157175357 +0000 UTC m=+240.485293510"
Oct 30 19:36:16.099804 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:16.099773    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-7c9b87d574-2kq47"
Oct 30 19:36:31.188415 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:36:31.188373    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 19:36:55.905692 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:55.905619    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-wwgvc"]
Oct 30 19:36:55.908964 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:55.908943    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-75bd89fdfc-wwgvc"
Oct 30 19:36:55.913981 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:55.913965    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Oct 30 19:36:55.916688 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:55.916665    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Oct 30 19:36:55.924735 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:55.924709    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-wwgvc"]
Oct 30 19:36:55.930632 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:55.930612    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-769875db56-blshw"]
Oct 30 19:36:55.933743 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:55.933726    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-769875db56-blshw"
Oct 30 19:36:55.950121 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:55.950099    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-769875db56-blshw"]
Oct 30 19:36:55.962855 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:55.962834    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-metrics-tls\""
Oct 30 19:36:55.962948 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:55.962896    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-ktljb\""
Oct 30 19:36:55.962948 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:55.962930    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Oct 30 19:36:55.962948 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:55.962939    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-q6hh5\""
Oct 30 19:36:55.963060 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:55.962958    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Oct 30 19:36:56.006899 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:56.006867    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppjs7\" (UniqueName: \"kubernetes.io/projected/77b41da1-d9be-4723-886a-fd568313384b-kube-api-access-ppjs7\") pod \"model-serving-api-75bd89fdfc-wwgvc\" (UID: \"77b41da1-d9be-4723-886a-fd568313384b\") " pod="opendatahub/model-serving-api-75bd89fdfc-wwgvc"
Oct 30 19:36:56.007028 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:56.006916    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77b41da1-d9be-4723-886a-fd568313384b-tls-certs\") pod \"model-serving-api-75bd89fdfc-wwgvc\" (UID: \"77b41da1-d9be-4723-886a-fd568313384b\") " pod="opendatahub/model-serving-api-75bd89fdfc-wwgvc"
Oct 30 19:36:56.108201 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:56.108170    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppjs7\" (UniqueName: \"kubernetes.io/projected/77b41da1-d9be-4723-886a-fd568313384b-kube-api-access-ppjs7\") pod \"model-serving-api-75bd89fdfc-wwgvc\" (UID: \"77b41da1-d9be-4723-886a-fd568313384b\") " pod="opendatahub/model-serving-api-75bd89fdfc-wwgvc"
Oct 30 19:36:56.108339 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:56.108207    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-588v5\" (UniqueName: \"kubernetes.io/projected/4e58facf-d3be-4f40-94b2-13cc119c8651-kube-api-access-588v5\") pod \"odh-model-controller-769875db56-blshw\" (UID: \"4e58facf-d3be-4f40-94b2-13cc119c8651\") " pod="opendatahub/odh-model-controller-769875db56-blshw"
Oct 30 19:36:56.108339 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:56.108234    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77b41da1-d9be-4723-886a-fd568313384b-tls-certs\") pod \"model-serving-api-75bd89fdfc-wwgvc\" (UID: \"77b41da1-d9be-4723-886a-fd568313384b\") " pod="opendatahub/model-serving-api-75bd89fdfc-wwgvc"
Oct 30 19:36:56.108339 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:56.108272    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4e58facf-d3be-4f40-94b2-13cc119c8651-cert\") pod \"odh-model-controller-769875db56-blshw\" (UID: \"4e58facf-d3be-4f40-94b2-13cc119c8651\") " pod="opendatahub/odh-model-controller-769875db56-blshw"
Oct 30 19:36:56.108339 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:56.108302    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e58facf-d3be-4f40-94b2-13cc119c8651-metrics-certs\") pod \"odh-model-controller-769875db56-blshw\" (UID: \"4e58facf-d3be-4f40-94b2-13cc119c8651\") " pod="opendatahub/odh-model-controller-769875db56-blshw"
Oct 30 19:36:56.110588 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:56.110570    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77b41da1-d9be-4723-886a-fd568313384b-tls-certs\") pod \"model-serving-api-75bd89fdfc-wwgvc\" (UID: \"77b41da1-d9be-4723-886a-fd568313384b\") " pod="opendatahub/model-serving-api-75bd89fdfc-wwgvc"
Oct 30 19:36:56.124174 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:56.124151    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppjs7\" (UniqueName: \"kubernetes.io/projected/77b41da1-d9be-4723-886a-fd568313384b-kube-api-access-ppjs7\") pod \"model-serving-api-75bd89fdfc-wwgvc\" (UID: \"77b41da1-d9be-4723-886a-fd568313384b\") " pod="opendatahub/model-serving-api-75bd89fdfc-wwgvc"
Oct 30 19:36:56.209269 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:56.209205    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4e58facf-d3be-4f40-94b2-13cc119c8651-cert\") pod \"odh-model-controller-769875db56-blshw\" (UID: \"4e58facf-d3be-4f40-94b2-13cc119c8651\") " pod="opendatahub/odh-model-controller-769875db56-blshw"
Oct 30 19:36:56.209269 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:56.209242    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e58facf-d3be-4f40-94b2-13cc119c8651-metrics-certs\") pod \"odh-model-controller-769875db56-blshw\" (UID: \"4e58facf-d3be-4f40-94b2-13cc119c8651\") " pod="opendatahub/odh-model-controller-769875db56-blshw"
Oct 30 19:36:56.209444 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:56.209357    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-588v5\" (UniqueName: \"kubernetes.io/projected/4e58facf-d3be-4f40-94b2-13cc119c8651-kube-api-access-588v5\") pod \"odh-model-controller-769875db56-blshw\" (UID: \"4e58facf-d3be-4f40-94b2-13cc119c8651\") " pod="opendatahub/odh-model-controller-769875db56-blshw"
Oct 30 19:36:56.211351 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:56.211328    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4e58facf-d3be-4f40-94b2-13cc119c8651-cert\") pod \"odh-model-controller-769875db56-blshw\" (UID: \"4e58facf-d3be-4f40-94b2-13cc119c8651\") " pod="opendatahub/odh-model-controller-769875db56-blshw"
Oct 30 19:36:56.211436 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:56.211421    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e58facf-d3be-4f40-94b2-13cc119c8651-metrics-certs\") pod \"odh-model-controller-769875db56-blshw\" (UID: \"4e58facf-d3be-4f40-94b2-13cc119c8651\") " pod="opendatahub/odh-model-controller-769875db56-blshw"
Oct 30 19:36:56.221546 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:56.221522    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-75bd89fdfc-wwgvc"
Oct 30 19:36:56.223578 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:56.223559    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-588v5\" (UniqueName: \"kubernetes.io/projected/4e58facf-d3be-4f40-94b2-13cc119c8651-kube-api-access-588v5\") pod \"odh-model-controller-769875db56-blshw\" (UID: \"4e58facf-d3be-4f40-94b2-13cc119c8651\") " pod="opendatahub/odh-model-controller-769875db56-blshw"
Oct 30 19:36:56.245901 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:56.245872    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-769875db56-blshw"
Oct 30 19:36:56.513089 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:56.513015    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-wwgvc"]
Oct 30 19:36:56.546069 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:56.546047    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-769875db56-blshw"]
Oct 30 19:36:56.549218 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:36:56.549196    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e58facf_d3be_4f40_94b2_13cc119c8651.slice/crio-40c1e928a454250ae4cff90dd87b201d502cfcd7dda439f7eda1978fad609a53 WatchSource:0}: Error finding container 40c1e928a454250ae4cff90dd87b201d502cfcd7dda439f7eda1978fad609a53: Status 404 returned error can't find the container with id 40c1e928a454250ae4cff90dd87b201d502cfcd7dda439f7eda1978fad609a53
Oct 30 19:36:57.347579 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:57.347537    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-769875db56-blshw" event={"ID":"4e58facf-d3be-4f40-94b2-13cc119c8651","Type":"ContainerStarted","Data":"40c1e928a454250ae4cff90dd87b201d502cfcd7dda439f7eda1978fad609a53"}
Oct 30 19:36:57.351158 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:57.351095    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-75bd89fdfc-wwgvc" event={"ID":"77b41da1-d9be-4723-886a-fd568313384b","Type":"ContainerStarted","Data":"f07630a0d4fbc8674ca2d27c55ceeecacee1110e7da6d557cde2bef368b114fa"}
Oct 30 19:36:59.363900 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:59.363873    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-75bd89fdfc-wwgvc" event={"ID":"77b41da1-d9be-4723-886a-fd568313384b","Type":"ContainerStarted","Data":"bd3dd869665c5d457d825ee9a5a38da6fb3f573b8168bc256148c2ec9831fa85"}
Oct 30 19:36:59.364203 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:59.364008    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-75bd89fdfc-wwgvc"
Oct 30 19:36:59.416522 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:36:59.416447    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/model-serving-api-75bd89fdfc-wwgvc" podStartSLOduration=2.4238423510000002 podStartE2EDuration="4.416433489s" totalImagesPullingTime="1.992591138s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:36:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:36:59.414166198 +0000 UTC m=+294.742284353" watchObservedRunningTime="2026-10-30 19:36:59.416433489 +0000 UTC m=+294.744551642"
Oct 30 19:37:00.370713 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:00.370679    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-769875db56-blshw" event={"ID":"4e58facf-d3be-4f40-94b2-13cc119c8651","Type":"ContainerStarted","Data":"563ae5e9a074ed71ea2d5e536016cecac605c49733429f9477a108af250878a1"}
Oct 30 19:37:00.371195 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:00.370762    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-769875db56-blshw"
Oct 30 19:37:00.406585 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:00.406535    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/odh-model-controller-769875db56-blshw" podStartSLOduration=2.6239791649999997 podStartE2EDuration="5.406521287s" totalImagesPullingTime="2.782542122s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:36:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:37:00.405599034 +0000 UTC m=+295.733717188" watchObservedRunningTime="2026-10-30 19:37:00.406521287 +0000 UTC m=+295.734639441"
Oct 30 19:37:01.188920 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:37:01.188879    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 19:37:10.375495 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:10.375449    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-75bd89fdfc-wwgvc"
Oct 30 19:37:11.378233 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:11.378202    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-769875db56-blshw"
Oct 30 19:37:21.319539 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:21.319499    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 19:37:31.189092 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:37:31.189051    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 19:37:33.203576 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:33.203543    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6dd76875b5-2ws5q"]
Oct 30 19:37:33.251176 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:33.251148    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6dd76875b5-2ws5q"]
Oct 30 19:37:33.251313 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:33.251244    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6dd76875b5-2ws5q"
Oct 30 19:37:33.259557 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:33.259534    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Oct 30 19:37:33.268236 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:33.268213    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6dd76875b5-2ws5q"]
Oct 30 19:37:33.279472 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:37:33.279449    6996 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-c7lkw tls-cert], unattached volumes=[], failed to process volumes=[kube-api-access-c7lkw]: context canceled" pod="kuadrant-system/authorino-6dd76875b5-2ws5q" podUID="17c39354-e52c-400e-99dd-d2ba0b2a73a3"
Oct 30 19:37:33.332097 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:33.332075    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/17c39354-e52c-400e-99dd-d2ba0b2a73a3-tls-cert\") pod \"authorino-6dd76875b5-2ws5q\" (UID: \"17c39354-e52c-400e-99dd-d2ba0b2a73a3\") " pod="kuadrant-system/authorino-6dd76875b5-2ws5q"
Oct 30 19:37:33.332195 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:33.332125    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7lkw\" (UniqueName: \"kubernetes.io/projected/17c39354-e52c-400e-99dd-d2ba0b2a73a3-kube-api-access-c7lkw\") pod \"authorino-6dd76875b5-2ws5q\" (UID: \"17c39354-e52c-400e-99dd-d2ba0b2a73a3\") " pod="kuadrant-system/authorino-6dd76875b5-2ws5q"
Oct 30 19:37:33.408519 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:33.408470    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5cc5c6d754-r8xbw"]
Oct 30 19:37:33.433451 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:33.433417    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7lkw\" (UniqueName: \"kubernetes.io/projected/17c39354-e52c-400e-99dd-d2ba0b2a73a3-kube-api-access-c7lkw\") pod \"authorino-6dd76875b5-2ws5q\" (UID: \"17c39354-e52c-400e-99dd-d2ba0b2a73a3\") " pod="kuadrant-system/authorino-6dd76875b5-2ws5q"
Oct 30 19:37:33.433605 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:33.433557    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/17c39354-e52c-400e-99dd-d2ba0b2a73a3-tls-cert\") pod \"authorino-6dd76875b5-2ws5q\" (UID: \"17c39354-e52c-400e-99dd-d2ba0b2a73a3\") " pod="kuadrant-system/authorino-6dd76875b5-2ws5q"
Oct 30 19:37:33.435895 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:33.435874    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/17c39354-e52c-400e-99dd-d2ba0b2a73a3-tls-cert\") pod \"authorino-6dd76875b5-2ws5q\" (UID: \"17c39354-e52c-400e-99dd-d2ba0b2a73a3\") " pod="kuadrant-system/authorino-6dd76875b5-2ws5q"
Oct 30 19:37:33.437366 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:33.437342    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5cc5c6d754-r8xbw"]
Oct 30 19:37:33.437470 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:33.437455    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5cc5c6d754-r8xbw"
Oct 30 19:37:33.444669 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:33.444647    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 19:37:33.444760 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:33.444727    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 19:37:33.452178 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:33.452157    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7lkw\" (UniqueName: \"kubernetes.io/projected/17c39354-e52c-400e-99dd-d2ba0b2a73a3-kube-api-access-c7lkw\") pod \"authorino-6dd76875b5-2ws5q\" (UID: \"17c39354-e52c-400e-99dd-d2ba0b2a73a3\") " pod="kuadrant-system/authorino-6dd76875b5-2ws5q"
Oct 30 19:37:33.460272 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:33.460222    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Oct 30 19:37:33.460272 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:33.460232    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-d892r\""
Oct 30 19:37:33.460272 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:33.460263    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Oct 30 19:37:33.534760 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:33.534734    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9df8a8aa-54f9-43a0-b047-4b6e121259ed-openshift-service-ca\") pod \"maas-api-5cc5c6d754-r8xbw\" (UID: \"9df8a8aa-54f9-43a0-b047-4b6e121259ed\") " pod="odh-ai-gateway-infra/maas-api-5cc5c6d754-r8xbw"
Oct 30 19:37:33.534880 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:33.534777    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2bvm\" (UniqueName: \"kubernetes.io/projected/9df8a8aa-54f9-43a0-b047-4b6e121259ed-kube-api-access-g2bvm\") pod \"maas-api-5cc5c6d754-r8xbw\" (UID: \"9df8a8aa-54f9-43a0-b047-4b6e121259ed\") " pod="odh-ai-gateway-infra/maas-api-5cc5c6d754-r8xbw"
Oct 30 19:37:33.534939 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:33.534920    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9df8a8aa-54f9-43a0-b047-4b6e121259ed-maas-api-tls\") pod \"maas-api-5cc5c6d754-r8xbw\" (UID: \"9df8a8aa-54f9-43a0-b047-4b6e121259ed\") " pod="odh-ai-gateway-infra/maas-api-5cc5c6d754-r8xbw"
Oct 30 19:37:33.535004 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:33.534947    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9df8a8aa-54f9-43a0-b047-4b6e121259ed-metrics-certs\") pod \"maas-api-5cc5c6d754-r8xbw\" (UID: \"9df8a8aa-54f9-43a0-b047-4b6e121259ed\") " pod="odh-ai-gateway-infra/maas-api-5cc5c6d754-r8xbw"
Oct 30 19:37:33.549615 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:33.549593    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6dd76875b5-2ws5q"
Oct 30 19:37:33.565353 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:33.565330    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6dd76875b5-2ws5q"
Oct 30 19:37:33.616367 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:33.616339    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-84f585c5-n9z4n"]
Oct 30 19:37:33.635539 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:33.635513    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/17c39354-e52c-400e-99dd-d2ba0b2a73a3-kube-api-access-c7lkw\" (UniqueName: \"kubernetes.io/projected/17c39354-e52c-400e-99dd-d2ba0b2a73a3-kube-api-access-c7lkw\") pod \"17c39354-e52c-400e-99dd-d2ba0b2a73a3\" (UID: \"17c39354-e52c-400e-99dd-d2ba0b2a73a3\") "
Oct 30 19:37:33.635671 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:33.635601    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/17c39354-e52c-400e-99dd-d2ba0b2a73a3-tls-cert\" (UniqueName: \"kubernetes.io/secret/17c39354-e52c-400e-99dd-d2ba0b2a73a3-tls-cert\") pod \"17c39354-e52c-400e-99dd-d2ba0b2a73a3\" (UID: \"17c39354-e52c-400e-99dd-d2ba0b2a73a3\") "
Oct 30 19:37:33.635825 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:33.635806    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9df8a8aa-54f9-43a0-b047-4b6e121259ed-maas-api-tls\") pod \"maas-api-5cc5c6d754-r8xbw\" (UID: \"9df8a8aa-54f9-43a0-b047-4b6e121259ed\") " pod="odh-ai-gateway-infra/maas-api-5cc5c6d754-r8xbw"
Oct 30 19:37:33.635933 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:33.635917    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9df8a8aa-54f9-43a0-b047-4b6e121259ed-metrics-certs\") pod \"maas-api-5cc5c6d754-r8xbw\" (UID: \"9df8a8aa-54f9-43a0-b047-4b6e121259ed\") " pod="odh-ai-gateway-infra/maas-api-5cc5c6d754-r8xbw"
Oct 30 19:37:33.635994 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:33.635970    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9df8a8aa-54f9-43a0-b047-4b6e121259ed-openshift-service-ca\") pod \"maas-api-5cc5c6d754-r8xbw\" (UID: \"9df8a8aa-54f9-43a0-b047-4b6e121259ed\") " pod="odh-ai-gateway-infra/maas-api-5cc5c6d754-r8xbw"
Oct 30 19:37:33.636051 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:33.636007    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2bvm\" (UniqueName: \"kubernetes.io/projected/9df8a8aa-54f9-43a0-b047-4b6e121259ed-kube-api-access-g2bvm\") pod \"maas-api-5cc5c6d754-r8xbw\" (UID: \"9df8a8aa-54f9-43a0-b047-4b6e121259ed\") " pod="odh-ai-gateway-infra/maas-api-5cc5c6d754-r8xbw"
Oct 30 19:37:33.636890 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:33.636863    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9df8a8aa-54f9-43a0-b047-4b6e121259ed-openshift-service-ca\") pod \"maas-api-5cc5c6d754-r8xbw\" (UID: \"9df8a8aa-54f9-43a0-b047-4b6e121259ed\") " pod="odh-ai-gateway-infra/maas-api-5cc5c6d754-r8xbw"
Oct 30 19:37:33.637698 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:33.637671    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/17c39354-e52c-400e-99dd-d2ba0b2a73a3-tls-cert" pod "17c39354-e52c-400e-99dd-d2ba0b2a73a3" (UID: "17c39354-e52c-400e-99dd-d2ba0b2a73a3"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:37:33.637881 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:33.637860    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17c39354-e52c-400e-99dd-d2ba0b2a73a3-kube-api-access-c7lkw" pod "17c39354-e52c-400e-99dd-d2ba0b2a73a3" (UID: "17c39354-e52c-400e-99dd-d2ba0b2a73a3"). InnerVolumeSpecName "kube-api-access-c7lkw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:37:33.637963 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:33.637933    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-84f585c5-n9z4n"]
Oct 30 19:37:33.638043 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:33.638029    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-84f585c5-n9z4n"
Oct 30 19:37:33.638549 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:33.638521    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9df8a8aa-54f9-43a0-b047-4b6e121259ed-maas-api-tls\") pod \"maas-api-5cc5c6d754-r8xbw\" (UID: \"9df8a8aa-54f9-43a0-b047-4b6e121259ed\") " pod="odh-ai-gateway-infra/maas-api-5cc5c6d754-r8xbw"
Oct 30 19:37:33.639397 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:33.639377    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9df8a8aa-54f9-43a0-b047-4b6e121259ed-metrics-certs\") pod \"maas-api-5cc5c6d754-r8xbw\" (UID: \"9df8a8aa-54f9-43a0-b047-4b6e121259ed\") " pod="odh-ai-gateway-infra/maas-api-5cc5c6d754-r8xbw"
Oct 30 19:37:33.650967 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:33.650949    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2bvm\" (UniqueName: \"kubernetes.io/projected/9df8a8aa-54f9-43a0-b047-4b6e121259ed-kube-api-access-g2bvm\") pod \"maas-api-5cc5c6d754-r8xbw\" (UID: \"9df8a8aa-54f9-43a0-b047-4b6e121259ed\") " pod="odh-ai-gateway-infra/maas-api-5cc5c6d754-r8xbw"
Oct 30 19:37:33.736666 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:33.736611    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vms5n\" (UniqueName: \"kubernetes.io/projected/cce9d43a-54e7-4417-b19d-292afa15e88b-kube-api-access-vms5n\") pod \"authorino-84f585c5-n9z4n\" (UID: \"cce9d43a-54e7-4417-b19d-292afa15e88b\") " pod="kuadrant-system/authorino-84f585c5-n9z4n"
Oct 30 19:37:33.736753 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:33.736714    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/cce9d43a-54e7-4417-b19d-292afa15e88b-tls-cert\") pod \"authorino-84f585c5-n9z4n\" (UID: \"cce9d43a-54e7-4417-b19d-292afa15e88b\") " pod="kuadrant-system/authorino-84f585c5-n9z4n"
Oct 30 19:37:33.736795 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:33.736785    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c7lkw\" (UniqueName: \"kubernetes.io/projected/17c39354-e52c-400e-99dd-d2ba0b2a73a3-kube-api-access-c7lkw\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:37:33.736834 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:33.736796    6996 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/17c39354-e52c-400e-99dd-d2ba0b2a73a3-tls-cert\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:37:33.774668 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:33.774646    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5cc5c6d754-r8xbw"
Oct 30 19:37:33.837900 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:33.837870    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/cce9d43a-54e7-4417-b19d-292afa15e88b-tls-cert\") pod \"authorino-84f585c5-n9z4n\" (UID: \"cce9d43a-54e7-4417-b19d-292afa15e88b\") " pod="kuadrant-system/authorino-84f585c5-n9z4n"
Oct 30 19:37:33.838032 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:33.837937    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vms5n\" (UniqueName: \"kubernetes.io/projected/cce9d43a-54e7-4417-b19d-292afa15e88b-kube-api-access-vms5n\") pod \"authorino-84f585c5-n9z4n\" (UID: \"cce9d43a-54e7-4417-b19d-292afa15e88b\") " pod="kuadrant-system/authorino-84f585c5-n9z4n"
Oct 30 19:37:33.840440 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:33.840415    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/cce9d43a-54e7-4417-b19d-292afa15e88b-tls-cert\") pod \"authorino-84f585c5-n9z4n\" (UID: \"cce9d43a-54e7-4417-b19d-292afa15e88b\") " pod="kuadrant-system/authorino-84f585c5-n9z4n"
Oct 30 19:37:33.850560 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:33.850537    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vms5n\" (UniqueName: \"kubernetes.io/projected/cce9d43a-54e7-4417-b19d-292afa15e88b-kube-api-access-vms5n\") pod \"authorino-84f585c5-n9z4n\" (UID: \"cce9d43a-54e7-4417-b19d-292afa15e88b\") " pod="kuadrant-system/authorino-84f585c5-n9z4n"
Oct 30 19:37:33.969712 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:33.969673    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-84f585c5-n9z4n"
Oct 30 19:37:34.050822 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:34.050705    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5cc5c6d754-r8xbw"]
Oct 30 19:37:34.053545 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:37:34.053521    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9df8a8aa_54f9_43a0_b047_4b6e121259ed.slice/crio-2b3991c50c47d9fa5e10a5cc0084c9671d480d305fea68315bbfadb82bd23e8e WatchSource:0}: Error finding container 2b3991c50c47d9fa5e10a5cc0084c9671d480d305fea68315bbfadb82bd23e8e: Status 404 returned error can't find the container with id 2b3991c50c47d9fa5e10a5cc0084c9671d480d305fea68315bbfadb82bd23e8e
Oct 30 19:37:34.243410 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:34.243387    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-84f585c5-n9z4n"]
Oct 30 19:37:34.245974 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:37:34.245953    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcce9d43a_54e7_4417_b19d_292afa15e88b.slice/crio-a5157f6607f9e79fe2631c19c71ff4a6fe09c073aafddf0b7c53f02b3f2b0eac WatchSource:0}: Error finding container a5157f6607f9e79fe2631c19c71ff4a6fe09c073aafddf0b7c53f02b3f2b0eac: Status 404 returned error can't find the container with id a5157f6607f9e79fe2631c19c71ff4a6fe09c073aafddf0b7c53f02b3f2b0eac
Oct 30 19:37:34.558308 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:34.558261    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5cc5c6d754-r8xbw" event={"ID":"9df8a8aa-54f9-43a0-b047-4b6e121259ed","Type":"ContainerStarted","Data":"2b3991c50c47d9fa5e10a5cc0084c9671d480d305fea68315bbfadb82bd23e8e"}
Oct 30 19:37:34.560177 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:34.560153    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6dd76875b5-2ws5q"
Oct 30 19:37:34.560557 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:34.560526    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-84f585c5-n9z4n" event={"ID":"cce9d43a-54e7-4417-b19d-292afa15e88b","Type":"ContainerStarted","Data":"a5157f6607f9e79fe2631c19c71ff4a6fe09c073aafddf0b7c53f02b3f2b0eac"}
Oct 30 19:37:34.600808 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:34.600774    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6dd76875b5-2ws5q"]
Oct 30 19:37:34.606755 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:34.606700    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6dd76875b5-2ws5q"]
Oct 30 19:37:34.792142 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:34.792103    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="17c39354-e52c-400e-99dd-d2ba0b2a73a3" path="/var/lib/kubelet/pods/17c39354-e52c-400e-99dd-d2ba0b2a73a3/volumes"
Oct 30 19:37:35.568616 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:35.568041    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-84f585c5-n9z4n" event={"ID":"cce9d43a-54e7-4417-b19d-292afa15e88b","Type":"ContainerStarted","Data":"382c1911026b81a4c942a058f294dd868fe1037073050b9d22093a0cf98f6eb9"}
Oct 30 19:37:35.587258 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:35.587204    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-84f585c5-n9z4n" podStartSLOduration=1.727138643 podStartE2EDuration="2.587185975s" totalImagesPullingTime="860.047332ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:37:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:37:35.584402237 +0000 UTC m=+330.912520391" watchObservedRunningTime="2026-10-30 19:37:35.587185975 +0000 UTC m=+330.915304127"
Oct 30 19:37:35.627843 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:35.626218    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-59fbh"]
Oct 30 19:37:35.628094 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:35.628064    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-56c6c9ddd5-59fbh" podUID="ae663b82-a97f-4d25-abfb-81c987110994" containerName="authorino" containerID="cri-o://199ac20eb67543424adcdd8c338d5472ea9da5c5888ecca4321c08bc01dcbff1" gracePeriod=30
Oct 30 19:37:35.912969 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:35.912940    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-59fbh"
Oct 30 19:37:36.063254 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:36.063215    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ae663b82-a97f-4d25-abfb-81c987110994-kube-api-access-5jq8n\" (UniqueName: \"kubernetes.io/projected/ae663b82-a97f-4d25-abfb-81c987110994-kube-api-access-5jq8n\") pod \"ae663b82-a97f-4d25-abfb-81c987110994\" (UID: \"ae663b82-a97f-4d25-abfb-81c987110994\") "
Oct 30 19:37:36.065194 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:36.065167    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae663b82-a97f-4d25-abfb-81c987110994-kube-api-access-5jq8n" pod "ae663b82-a97f-4d25-abfb-81c987110994" (UID: "ae663b82-a97f-4d25-abfb-81c987110994"). InnerVolumeSpecName "kube-api-access-5jq8n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:37:36.164637 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:36.164602    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5jq8n\" (UniqueName: \"kubernetes.io/projected/ae663b82-a97f-4d25-abfb-81c987110994-kube-api-access-5jq8n\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:37:36.574110 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:36.574081    6996 generic.go:356] "Generic (PLEG): container finished" podID="ae663b82-a97f-4d25-abfb-81c987110994" containerID="199ac20eb67543424adcdd8c338d5472ea9da5c5888ecca4321c08bc01dcbff1" exitCode=0
Oct 30 19:37:36.574454 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:36.574128    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-59fbh"
Oct 30 19:37:36.574454 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:36.574165    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-59fbh" event={"ID":"ae663b82-a97f-4d25-abfb-81c987110994","Type":"ContainerDied","Data":"199ac20eb67543424adcdd8c338d5472ea9da5c5888ecca4321c08bc01dcbff1"}
Oct 30 19:37:36.574454 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:36.574212    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-59fbh" event={"ID":"ae663b82-a97f-4d25-abfb-81c987110994","Type":"ContainerDied","Data":"0bbb9d8af2e751f4ee9c15f7b37abc0f0fa6c0b5d45832c33c9cca5608473e16"}
Oct 30 19:37:36.574454 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:36.574229    6996 scope.go:122] "RemoveContainer" containerID="199ac20eb67543424adcdd8c338d5472ea9da5c5888ecca4321c08bc01dcbff1"
Oct 30 19:37:36.582535 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:36.582516    6996 scope.go:122] "RemoveContainer" containerID="199ac20eb67543424adcdd8c338d5472ea9da5c5888ecca4321c08bc01dcbff1"
Oct 30 19:37:36.582778 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:37:36.582760    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"199ac20eb67543424adcdd8c338d5472ea9da5c5888ecca4321c08bc01dcbff1\": container with ID starting with 199ac20eb67543424adcdd8c338d5472ea9da5c5888ecca4321c08bc01dcbff1 not found: ID does not exist" containerID="199ac20eb67543424adcdd8c338d5472ea9da5c5888ecca4321c08bc01dcbff1"
Oct 30 19:37:36.582854 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:36.582788    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"199ac20eb67543424adcdd8c338d5472ea9da5c5888ecca4321c08bc01dcbff1"} err="failed to get container status \"199ac20eb67543424adcdd8c338d5472ea9da5c5888ecca4321c08bc01dcbff1\": rpc error: code = NotFound desc = could not find container \"199ac20eb67543424adcdd8c338d5472ea9da5c5888ecca4321c08bc01dcbff1\": container with ID starting with 199ac20eb67543424adcdd8c338d5472ea9da5c5888ecca4321c08bc01dcbff1 not found: ID does not exist"
Oct 30 19:37:36.597725 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:36.597704    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-59fbh"]
Oct 30 19:37:36.604588 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:36.604561    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-59fbh"]
Oct 30 19:37:36.789810 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:36.789734    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ae663b82-a97f-4d25-abfb-81c987110994" path="/var/lib/kubelet/pods/ae663b82-a97f-4d25-abfb-81c987110994/volumes"
Oct 30 19:37:37.580987 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:37.580950    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5cc5c6d754-r8xbw" event={"ID":"9df8a8aa-54f9-43a0-b047-4b6e121259ed","Type":"ContainerStarted","Data":"90cadec0dc1939ebea08a4773526152135f3f14e3129a1b0c39936ada3c331f7"}
Oct 30 19:37:37.581387 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:37.581078    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5cc5c6d754-r8xbw"
Oct 30 19:37:37.601858 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:37.601807    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5cc5c6d754-r8xbw" podStartSLOduration=2.140403682 podStartE2EDuration="4.601793175s" totalImagesPullingTime="2.461389493s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:37:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:37:37.600853691 +0000 UTC m=+332.928971843" watchObservedRunningTime="2026-10-30 19:37:37.601793175 +0000 UTC m=+332.929911328"
Oct 30 19:37:43.591707 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:37:43.591678    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5cc5c6d754-r8xbw"
Oct 30 19:38:01.188814 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:38:01.188771    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 19:38:31.189169 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:38:31.189128    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 19:38:32.894768 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:32.894740    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-7d85675f98-bw5xh"]
Oct 30 19:38:32.895182 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:32.895154    6996 cpu_manager.go:414] "containerMap: removing container" podUID="ae663b82-a97f-4d25-abfb-81c987110994" containerName="authorino"
Oct 30 19:38:32.895182 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:32.895166    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ae663b82-a97f-4d25-abfb-81c987110994" containerName="authorino"
Oct 30 19:38:32.895270 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:32.895229    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="ae663b82-a97f-4d25-abfb-81c987110994" containerName="authorino"
Oct 30 19:38:32.898188 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:32.898171    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-7d85675f98-bw5xh"
Oct 30 19:38:32.904373 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:32.904338    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"ai-gateway-controller-parameters\""
Oct 30 19:38:32.907889 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:32.907868    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-metrics-tls\""
Oct 30 19:38:32.908017 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:32.907871    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-dockercfg-jhjqp\""
Oct 30 19:38:32.909628 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:32.909605    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-7d85675f98-bw5xh"]
Oct 30 19:38:33.049854 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:33.049815    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9lxg\" (UniqueName: \"kubernetes.io/projected/01d3af60-d671-464f-ba54-b82fb0b92af3-kube-api-access-d9lxg\") pod \"ai-gateway-controller-7d85675f98-bw5xh\" (UID: \"01d3af60-d671-464f-ba54-b82fb0b92af3\") " pod="opendatahub/ai-gateway-controller-7d85675f98-bw5xh"
Oct 30 19:38:33.050023 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:33.049951    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/01d3af60-d671-464f-ba54-b82fb0b92af3-metrics-certs\") pod \"ai-gateway-controller-7d85675f98-bw5xh\" (UID: \"01d3af60-d671-464f-ba54-b82fb0b92af3\") " pod="opendatahub/ai-gateway-controller-7d85675f98-bw5xh"
Oct 30 19:38:33.150890 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:33.150799    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9lxg\" (UniqueName: \"kubernetes.io/projected/01d3af60-d671-464f-ba54-b82fb0b92af3-kube-api-access-d9lxg\") pod \"ai-gateway-controller-7d85675f98-bw5xh\" (UID: \"01d3af60-d671-464f-ba54-b82fb0b92af3\") " pod="opendatahub/ai-gateway-controller-7d85675f98-bw5xh"
Oct 30 19:38:33.151088 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:33.150900    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/01d3af60-d671-464f-ba54-b82fb0b92af3-metrics-certs\") pod \"ai-gateway-controller-7d85675f98-bw5xh\" (UID: \"01d3af60-d671-464f-ba54-b82fb0b92af3\") " pod="opendatahub/ai-gateway-controller-7d85675f98-bw5xh"
Oct 30 19:38:33.153416 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:33.153390    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/01d3af60-d671-464f-ba54-b82fb0b92af3-metrics-certs\") pod \"ai-gateway-controller-7d85675f98-bw5xh\" (UID: \"01d3af60-d671-464f-ba54-b82fb0b92af3\") " pod="opendatahub/ai-gateway-controller-7d85675f98-bw5xh"
Oct 30 19:38:33.163394 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:33.163368    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9lxg\" (UniqueName: \"kubernetes.io/projected/01d3af60-d671-464f-ba54-b82fb0b92af3-kube-api-access-d9lxg\") pod \"ai-gateway-controller-7d85675f98-bw5xh\" (UID: \"01d3af60-d671-464f-ba54-b82fb0b92af3\") " pod="opendatahub/ai-gateway-controller-7d85675f98-bw5xh"
Oct 30 19:38:33.211320 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:33.211295    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-7d85675f98-bw5xh"
Oct 30 19:38:33.490505 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:33.490453    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-7d85675f98-bw5xh"]
Oct 30 19:38:33.493293 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:38:33.493270    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01d3af60_d671_464f_ba54_b82fb0b92af3.slice/crio-46ad40e7395c595b50fda295cf2524f96bbba9d195c98e1173b4353e7a4c5293 WatchSource:0}: Error finding container 46ad40e7395c595b50fda295cf2524f96bbba9d195c98e1173b4353e7a4c5293: Status 404 returned error can't find the container with id 46ad40e7395c595b50fda295cf2524f96bbba9d195c98e1173b4353e7a4c5293
Oct 30 19:38:33.844678 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:33.844637    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-7d85675f98-bw5xh" event={"ID":"01d3af60-d671-464f-ba54-b82fb0b92af3","Type":"ContainerStarted","Data":"46ad40e7395c595b50fda295cf2524f96bbba9d195c98e1173b4353e7a4c5293"}
Oct 30 19:38:36.860644 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:36.860604    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-7d85675f98-bw5xh" event={"ID":"01d3af60-d671-464f-ba54-b82fb0b92af3","Type":"ContainerStarted","Data":"bcdc37a7e2776555cdb87d067ced07af57dd576c704e5790f092f3a422034fda"}
Oct 30 19:38:36.861049 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:36.860728    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-7d85675f98-bw5xh"
Oct 30 19:38:36.888171 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:36.888125    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-7d85675f98-bw5xh" podStartSLOduration=2.426679019 podStartE2EDuration="4.888113131s" totalImagesPullingTime="2.461434112s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:38:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:38:36.885886872 +0000 UTC m=+392.214005024" watchObservedRunningTime="2026-10-30 19:38:36.888113131 +0000 UTC m=+392.216231330"
Oct 30 19:38:37.825360 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:37.825328    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-2j4r6"]
Oct 30 19:38:37.828842 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:37.828826    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7b98bb678c-2j4r6"
Oct 30 19:38:37.838664 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:37.838639    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-7gxgn\""
Oct 30 19:38:37.838781 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:37.838693    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Oct 30 19:38:37.839843 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:37.839821    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-2j4r6"]
Oct 30 19:38:37.845859 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:37.845835    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5f78687d5b-dkcrs"]
Oct 30 19:38:37.849209 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:37.849186    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5f78687d5b-dkcrs"
Oct 30 19:38:37.868774 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:37.868749    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5f78687d5b-dkcrs"]
Oct 30 19:38:37.994199 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:37.994165    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwn7h\" (UniqueName: \"kubernetes.io/projected/c72164f7-cc46-4f11-b1f0-f50adf9c4c0b-kube-api-access-kwn7h\") pod \"payload-processing-5f78687d5b-dkcrs\" (UID: \"c72164f7-cc46-4f11-b1f0-f50adf9c4c0b\") " pod="openshift-ingress/payload-processing-5f78687d5b-dkcrs"
Oct 30 19:38:37.994347 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:37.994217    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2s6wd\" (UniqueName: \"kubernetes.io/projected/8879e7d6-4b91-4abf-9952-feb51aea83e6-kube-api-access-2s6wd\") pod \"payload-pre-processing-7b98bb678c-2j4r6\" (UID: \"8879e7d6-4b91-4abf-9952-feb51aea83e6\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-2j4r6"
Oct 30 19:38:37.994347 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:37.994324    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8879e7d6-4b91-4abf-9952-feb51aea83e6-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-2j4r6\" (UID: \"8879e7d6-4b91-4abf-9952-feb51aea83e6\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-2j4r6"
Oct 30 19:38:37.994419 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:37.994389    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c72164f7-cc46-4f11-b1f0-f50adf9c4c0b-plugins-config-volume\") pod \"payload-processing-5f78687d5b-dkcrs\" (UID: \"c72164f7-cc46-4f11-b1f0-f50adf9c4c0b\") " pod="openshift-ingress/payload-processing-5f78687d5b-dkcrs"
Oct 30 19:38:38.095162 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:38.095091    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2s6wd\" (UniqueName: \"kubernetes.io/projected/8879e7d6-4b91-4abf-9952-feb51aea83e6-kube-api-access-2s6wd\") pod \"payload-pre-processing-7b98bb678c-2j4r6\" (UID: \"8879e7d6-4b91-4abf-9952-feb51aea83e6\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-2j4r6"
Oct 30 19:38:38.095162 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:38.095132    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8879e7d6-4b91-4abf-9952-feb51aea83e6-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-2j4r6\" (UID: \"8879e7d6-4b91-4abf-9952-feb51aea83e6\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-2j4r6"
Oct 30 19:38:38.095162 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:38.095155    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c72164f7-cc46-4f11-b1f0-f50adf9c4c0b-plugins-config-volume\") pod \"payload-processing-5f78687d5b-dkcrs\" (UID: \"c72164f7-cc46-4f11-b1f0-f50adf9c4c0b\") " pod="openshift-ingress/payload-processing-5f78687d5b-dkcrs"
Oct 30 19:38:38.095372 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:38.095217    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwn7h\" (UniqueName: \"kubernetes.io/projected/c72164f7-cc46-4f11-b1f0-f50adf9c4c0b-kube-api-access-kwn7h\") pod \"payload-processing-5f78687d5b-dkcrs\" (UID: \"c72164f7-cc46-4f11-b1f0-f50adf9c4c0b\") " pod="openshift-ingress/payload-processing-5f78687d5b-dkcrs"
Oct 30 19:38:38.095828 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:38.095808    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8879e7d6-4b91-4abf-9952-feb51aea83e6-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-2j4r6\" (UID: \"8879e7d6-4b91-4abf-9952-feb51aea83e6\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-2j4r6"
Oct 30 19:38:38.095945 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:38.095907    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c72164f7-cc46-4f11-b1f0-f50adf9c4c0b-plugins-config-volume\") pod \"payload-processing-5f78687d5b-dkcrs\" (UID: \"c72164f7-cc46-4f11-b1f0-f50adf9c4c0b\") " pod="openshift-ingress/payload-processing-5f78687d5b-dkcrs"
Oct 30 19:38:38.110281 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:38.110253    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwn7h\" (UniqueName: \"kubernetes.io/projected/c72164f7-cc46-4f11-b1f0-f50adf9c4c0b-kube-api-access-kwn7h\") pod \"payload-processing-5f78687d5b-dkcrs\" (UID: \"c72164f7-cc46-4f11-b1f0-f50adf9c4c0b\") " pod="openshift-ingress/payload-processing-5f78687d5b-dkcrs"
Oct 30 19:38:38.110415 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:38.110393    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2s6wd\" (UniqueName: \"kubernetes.io/projected/8879e7d6-4b91-4abf-9952-feb51aea83e6-kube-api-access-2s6wd\") pod \"payload-pre-processing-7b98bb678c-2j4r6\" (UID: \"8879e7d6-4b91-4abf-9952-feb51aea83e6\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-2j4r6"
Oct 30 19:38:38.141199 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:38.141159    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7b98bb678c-2j4r6"
Oct 30 19:38:38.161352 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:38.161328    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5f78687d5b-dkcrs"
Oct 30 19:38:38.417152 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:38.417123    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-2j4r6"]
Oct 30 19:38:38.419691 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:38:38.419659    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8879e7d6_4b91_4abf_9952_feb51aea83e6.slice/crio-e470709b35ac6e16b993d21eb650e98e9fd07a8d0cbefe88a80fff65716343d8 WatchSource:0}: Error finding container e470709b35ac6e16b993d21eb650e98e9fd07a8d0cbefe88a80fff65716343d8: Status 404 returned error can't find the container with id e470709b35ac6e16b993d21eb650e98e9fd07a8d0cbefe88a80fff65716343d8
Oct 30 19:38:38.446845 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:38.446821    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5f78687d5b-dkcrs"]
Oct 30 19:38:38.449801 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:38:38.449778    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc72164f7_cc46_4f11_b1f0_f50adf9c4c0b.slice/crio-97e1b25703e57b8e575b934c94aec93ae1303f399b6703cf2cb35a0906418b3d WatchSource:0}: Error finding container 97e1b25703e57b8e575b934c94aec93ae1303f399b6703cf2cb35a0906418b3d: Status 404 returned error can't find the container with id 97e1b25703e57b8e575b934c94aec93ae1303f399b6703cf2cb35a0906418b3d
Oct 30 19:38:38.871432 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:38.871396    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7b98bb678c-2j4r6" event={"ID":"8879e7d6-4b91-4abf-9952-feb51aea83e6","Type":"ContainerStarted","Data":"e470709b35ac6e16b993d21eb650e98e9fd07a8d0cbefe88a80fff65716343d8"}
Oct 30 19:38:38.872651 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:38.872625    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5f78687d5b-dkcrs" event={"ID":"c72164f7-cc46-4f11-b1f0-f50adf9c4c0b","Type":"ContainerStarted","Data":"97e1b25703e57b8e575b934c94aec93ae1303f399b6703cf2cb35a0906418b3d"}
Oct 30 19:38:40.887084 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:40.887000    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5f78687d5b-dkcrs" event={"ID":"c72164f7-cc46-4f11-b1f0-f50adf9c4c0b","Type":"ContainerStarted","Data":"63b5d068bb1ff4c62c4e47ae7b2783476897739f3ea3eae0ea6965de332b1a20"}
Oct 30 19:38:40.887084 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:40.887067    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5f78687d5b-dkcrs"
Oct 30 19:38:40.888621 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:40.888599    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7b98bb678c-2j4r6" event={"ID":"8879e7d6-4b91-4abf-9952-feb51aea83e6","Type":"ContainerStarted","Data":"dc7e000a6660bccfeeb18305e9b922e09d40be527c936c3c30c4842bb62df968"}
Oct 30 19:38:40.888732 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:40.888649    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7b98bb678c-2j4r6"
Oct 30 19:38:40.906073 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:40.906031    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5f78687d5b-dkcrs" podStartSLOduration=1.7084725189999999 podStartE2EDuration="3.90601789s" totalImagesPullingTime="2.197545371s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:38:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:38:40.90530331 +0000 UTC m=+396.233421463" watchObservedRunningTime="2026-10-30 19:38:40.90601789 +0000 UTC m=+396.234136042"
Oct 30 19:38:40.928027 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:40.927984    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7b98bb678c-2j4r6" podStartSLOduration=1.7332932049999998 podStartE2EDuration="3.927973905s" totalImagesPullingTime="2.1946807s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:38:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:38:40.926915638 +0000 UTC m=+396.255033790" watchObservedRunningTime="2026-10-30 19:38:40.927973905 +0000 UTC m=+396.256092051"
Oct 30 19:38:47.868202 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:47.868161    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-7d85675f98-bw5xh"
Oct 30 19:38:48.149496 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:48.149405    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-5c494664d6-7lvd2"]
Oct 30 19:38:48.152958 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:48.152943    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-5c494664d6-7lvd2"
Oct 30 19:38:48.164234 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:48.164210    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-5c494664d6-7lvd2"]
Oct 30 19:38:48.186444 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:48.186420    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50e907ce-5794-4533-9bda-d0e3abaf4e21-metrics-certs\") pod \"ai-gateway-controller-5c494664d6-7lvd2\" (UID: \"50e907ce-5794-4533-9bda-d0e3abaf4e21\") " pod="opendatahub/ai-gateway-controller-5c494664d6-7lvd2"
Oct 30 19:38:48.186599 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:48.186455    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvpls\" (UniqueName: \"kubernetes.io/projected/50e907ce-5794-4533-9bda-d0e3abaf4e21-kube-api-access-qvpls\") pod \"ai-gateway-controller-5c494664d6-7lvd2\" (UID: \"50e907ce-5794-4533-9bda-d0e3abaf4e21\") " pod="opendatahub/ai-gateway-controller-5c494664d6-7lvd2"
Oct 30 19:38:48.286951 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:48.286925    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvpls\" (UniqueName: \"kubernetes.io/projected/50e907ce-5794-4533-9bda-d0e3abaf4e21-kube-api-access-qvpls\") pod \"ai-gateway-controller-5c494664d6-7lvd2\" (UID: \"50e907ce-5794-4533-9bda-d0e3abaf4e21\") " pod="opendatahub/ai-gateway-controller-5c494664d6-7lvd2"
Oct 30 19:38:48.287076 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:48.287004    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50e907ce-5794-4533-9bda-d0e3abaf4e21-metrics-certs\") pod \"ai-gateway-controller-5c494664d6-7lvd2\" (UID: \"50e907ce-5794-4533-9bda-d0e3abaf4e21\") " pod="opendatahub/ai-gateway-controller-5c494664d6-7lvd2"
Oct 30 19:38:48.289225 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:48.289201    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50e907ce-5794-4533-9bda-d0e3abaf4e21-metrics-certs\") pod \"ai-gateway-controller-5c494664d6-7lvd2\" (UID: \"50e907ce-5794-4533-9bda-d0e3abaf4e21\") " pod="opendatahub/ai-gateway-controller-5c494664d6-7lvd2"
Oct 30 19:38:48.298933 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:48.298911    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvpls\" (UniqueName: \"kubernetes.io/projected/50e907ce-5794-4533-9bda-d0e3abaf4e21-kube-api-access-qvpls\") pod \"ai-gateway-controller-5c494664d6-7lvd2\" (UID: \"50e907ce-5794-4533-9bda-d0e3abaf4e21\") " pod="opendatahub/ai-gateway-controller-5c494664d6-7lvd2"
Oct 30 19:38:48.465509 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:48.465414    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-5c494664d6-7lvd2"
Oct 30 19:38:48.736795 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:48.736760    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-5c494664d6-7lvd2"]
Oct 30 19:38:48.739390 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:38:48.739359    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50e907ce_5794_4533_9bda_d0e3abaf4e21.slice/crio-2ae723487b15c9d06d9eaca8ff9d9365c6c49c66ebcf5b8b93812b8766a3b661 WatchSource:0}: Error finding container 2ae723487b15c9d06d9eaca8ff9d9365c6c49c66ebcf5b8b93812b8766a3b661: Status 404 returned error can't find the container with id 2ae723487b15c9d06d9eaca8ff9d9365c6c49c66ebcf5b8b93812b8766a3b661
Oct 30 19:38:48.927402 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:48.927361    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-5c494664d6-7lvd2" event={"ID":"50e907ce-5794-4533-9bda-d0e3abaf4e21","Type":"ContainerStarted","Data":"2ae723487b15c9d06d9eaca8ff9d9365c6c49c66ebcf5b8b93812b8766a3b661"}
Oct 30 19:38:49.934052 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:49.934006    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-5c494664d6-7lvd2" event={"ID":"50e907ce-5794-4533-9bda-d0e3abaf4e21","Type":"ContainerStarted","Data":"4b8223d20186b8f0e634d8ea7331a42699c9705572d17e414db2c6ba6f05f947"}
Oct 30 19:38:49.934446 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:49.934074    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-5c494664d6-7lvd2"
Oct 30 19:38:49.954159 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:49.954113    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-5c494664d6-7lvd2" podStartSLOduration=1.555225526 podStartE2EDuration="1.954100007s" totalImagesPullingTime="398.874481ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:38:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:38:49.952166761 +0000 UTC m=+405.280284916" watchObservedRunningTime="2026-10-30 19:38:49.954100007 +0000 UTC m=+405.282218157"
Oct 30 19:38:51.896380 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:51.896352    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5f78687d5b-dkcrs"
Oct 30 19:38:51.896751 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:38:51.896417    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7b98bb678c-2j4r6"
Oct 30 19:39:00.941790 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:00.941757    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-5c494664d6-7lvd2"
Oct 30 19:39:00.993333 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:00.993297    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-7d85675f98-bw5xh"]
Oct 30 19:39:00.993587 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:00.993547    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/ai-gateway-controller-7d85675f98-bw5xh" podUID="01d3af60-d671-464f-ba54-b82fb0b92af3" containerName="manager" containerID="cri-o://bcdc37a7e2776555cdb87d067ced07af57dd576c704e5790f092f3a422034fda" gracePeriod=30
Oct 30 19:39:01.193299 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:39:01.193199    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 19:39:01.237094 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:01.237071    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-7d85675f98-bw5xh"
Oct 30 19:39:01.301681 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:01.301647    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/01d3af60-d671-464f-ba54-b82fb0b92af3-metrics-certs\" (UniqueName: \"kubernetes.io/secret/01d3af60-d671-464f-ba54-b82fb0b92af3-metrics-certs\") pod \"01d3af60-d671-464f-ba54-b82fb0b92af3\" (UID: \"01d3af60-d671-464f-ba54-b82fb0b92af3\") "
Oct 30 19:39:01.301858 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:01.301698    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/01d3af60-d671-464f-ba54-b82fb0b92af3-kube-api-access-d9lxg\" (UniqueName: \"kubernetes.io/projected/01d3af60-d671-464f-ba54-b82fb0b92af3-kube-api-access-d9lxg\") pod \"01d3af60-d671-464f-ba54-b82fb0b92af3\" (UID: \"01d3af60-d671-464f-ba54-b82fb0b92af3\") "
Oct 30 19:39:01.303840 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:01.303803    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01d3af60-d671-464f-ba54-b82fb0b92af3-metrics-certs" pod "01d3af60-d671-464f-ba54-b82fb0b92af3" (UID: "01d3af60-d671-464f-ba54-b82fb0b92af3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:39:01.303965 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:01.303877    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01d3af60-d671-464f-ba54-b82fb0b92af3-kube-api-access-d9lxg" pod "01d3af60-d671-464f-ba54-b82fb0b92af3" (UID: "01d3af60-d671-464f-ba54-b82fb0b92af3"). InnerVolumeSpecName "kube-api-access-d9lxg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:39:01.402532 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:01.402472    6996 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/01d3af60-d671-464f-ba54-b82fb0b92af3-metrics-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:39:01.402532 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:01.402527    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d9lxg\" (UniqueName: \"kubernetes.io/projected/01d3af60-d671-464f-ba54-b82fb0b92af3-kube-api-access-d9lxg\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:39:01.995852 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:01.995814    6996 generic.go:356] "Generic (PLEG): container finished" podID="01d3af60-d671-464f-ba54-b82fb0b92af3" containerID="bcdc37a7e2776555cdb87d067ced07af57dd576c704e5790f092f3a422034fda" exitCode=0
Oct 30 19:39:01.996269 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:01.995856    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-7d85675f98-bw5xh"
Oct 30 19:39:01.996269 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:01.995905    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-7d85675f98-bw5xh" event={"ID":"01d3af60-d671-464f-ba54-b82fb0b92af3","Type":"ContainerDied","Data":"bcdc37a7e2776555cdb87d067ced07af57dd576c704e5790f092f3a422034fda"}
Oct 30 19:39:01.996269 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:01.995959    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-7d85675f98-bw5xh" event={"ID":"01d3af60-d671-464f-ba54-b82fb0b92af3","Type":"ContainerDied","Data":"46ad40e7395c595b50fda295cf2524f96bbba9d195c98e1173b4353e7a4c5293"}
Oct 30 19:39:01.996269 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:01.995980    6996 scope.go:122] "RemoveContainer" containerID="bcdc37a7e2776555cdb87d067ced07af57dd576c704e5790f092f3a422034fda"
Oct 30 19:39:02.005784 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:02.005769    6996 scope.go:122] "RemoveContainer" containerID="bcdc37a7e2776555cdb87d067ced07af57dd576c704e5790f092f3a422034fda"
Oct 30 19:39:02.006043 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:39:02.006028    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bcdc37a7e2776555cdb87d067ced07af57dd576c704e5790f092f3a422034fda\": container with ID starting with bcdc37a7e2776555cdb87d067ced07af57dd576c704e5790f092f3a422034fda not found: ID does not exist" containerID="bcdc37a7e2776555cdb87d067ced07af57dd576c704e5790f092f3a422034fda"
Oct 30 19:39:02.006078 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:02.006052    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bcdc37a7e2776555cdb87d067ced07af57dd576c704e5790f092f3a422034fda"} err="failed to get container status \"bcdc37a7e2776555cdb87d067ced07af57dd576c704e5790f092f3a422034fda\": rpc error: code = NotFound desc = could not find container \"bcdc37a7e2776555cdb87d067ced07af57dd576c704e5790f092f3a422034fda\": container with ID starting with bcdc37a7e2776555cdb87d067ced07af57dd576c704e5790f092f3a422034fda not found: ID does not exist"
Oct 30 19:39:02.018274 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:02.018250    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-7d85675f98-bw5xh"]
Oct 30 19:39:02.028575 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:02.028543    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/ai-gateway-controller-7d85675f98-bw5xh"]
Oct 30 19:39:02.790721 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:02.790683    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="01d3af60-d671-464f-ba54-b82fb0b92af3" path="/var/lib/kubelet/pods/01d3af60-d671-464f-ba54-b82fb0b92af3/volumes"
Oct 30 19:39:08.578081 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:08.578047    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-k5q55"]
Oct 30 19:39:08.578667 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:08.578649    6996 cpu_manager.go:414] "containerMap: removing container" podUID="01d3af60-d671-464f-ba54-b82fb0b92af3" containerName="manager"
Oct 30 19:39:08.578723 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:08.578670    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="01d3af60-d671-464f-ba54-b82fb0b92af3" containerName="manager"
Oct 30 19:39:08.578816 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:08.578805    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="01d3af60-d671-464f-ba54-b82fb0b92af3" containerName="manager"
Oct 30 19:39:08.586003 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:08.585983    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-k5q55"
Oct 30 19:39:08.593423 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:08.593395    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Oct 30 19:39:08.594203 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:08.593573    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Oct 30 19:39:08.594203 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:08.593582    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-qx8jh\""
Oct 30 19:39:08.594203 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:08.593640    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Oct 30 19:39:08.595511 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:08.595471    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-k5q55"]
Oct 30 19:39:08.669642 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:08.669601    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77c1aa9d-afba-4bb3-b583-d26d66592136-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-k5q55\" (UID: \"77c1aa9d-afba-4bb3-b583-d26d66592136\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-k5q55"
Oct 30 19:39:08.669825 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:08.669657    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77c1aa9d-afba-4bb3-b583-d26d66592136-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-k5q55\" (UID: \"77c1aa9d-afba-4bb3-b583-d26d66592136\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-k5q55"
Oct 30 19:39:08.669825 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:08.669772    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fn7nl\" (UniqueName: \"kubernetes.io/projected/77c1aa9d-afba-4bb3-b583-d26d66592136-kube-api-access-fn7nl\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-k5q55\" (UID: \"77c1aa9d-afba-4bb3-b583-d26d66592136\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-k5q55"
Oct 30 19:39:08.669825 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:08.669810    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77c1aa9d-afba-4bb3-b583-d26d66592136-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-k5q55\" (UID: \"77c1aa9d-afba-4bb3-b583-d26d66592136\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-k5q55"
Oct 30 19:39:08.669946 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:08.669834    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77c1aa9d-afba-4bb3-b583-d26d66592136-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-k5q55\" (UID: \"77c1aa9d-afba-4bb3-b583-d26d66592136\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-k5q55"
Oct 30 19:39:08.669981 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:08.669961    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77c1aa9d-afba-4bb3-b583-d26d66592136-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-k5q55\" (UID: \"77c1aa9d-afba-4bb3-b583-d26d66592136\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-k5q55"
Oct 30 19:39:08.771186 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:08.771149    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fn7nl\" (UniqueName: \"kubernetes.io/projected/77c1aa9d-afba-4bb3-b583-d26d66592136-kube-api-access-fn7nl\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-k5q55\" (UID: \"77c1aa9d-afba-4bb3-b583-d26d66592136\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-k5q55"
Oct 30 19:39:08.771186 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:08.771196    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77c1aa9d-afba-4bb3-b583-d26d66592136-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-k5q55\" (UID: \"77c1aa9d-afba-4bb3-b583-d26d66592136\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-k5q55"
Oct 30 19:39:08.771461 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:08.771221    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77c1aa9d-afba-4bb3-b583-d26d66592136-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-k5q55\" (UID: \"77c1aa9d-afba-4bb3-b583-d26d66592136\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-k5q55"
Oct 30 19:39:08.771461 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:08.771274    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77c1aa9d-afba-4bb3-b583-d26d66592136-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-k5q55\" (UID: \"77c1aa9d-afba-4bb3-b583-d26d66592136\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-k5q55"
Oct 30 19:39:08.771461 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:08.771299    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77c1aa9d-afba-4bb3-b583-d26d66592136-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-k5q55\" (UID: \"77c1aa9d-afba-4bb3-b583-d26d66592136\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-k5q55"
Oct 30 19:39:08.771461 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:08.771346    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77c1aa9d-afba-4bb3-b583-d26d66592136-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-k5q55\" (UID: \"77c1aa9d-afba-4bb3-b583-d26d66592136\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-k5q55"
Oct 30 19:39:08.771789 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:08.771735    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77c1aa9d-afba-4bb3-b583-d26d66592136-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-k5q55\" (UID: \"77c1aa9d-afba-4bb3-b583-d26d66592136\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-k5q55"
Oct 30 19:39:08.771789 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:08.771764    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77c1aa9d-afba-4bb3-b583-d26d66592136-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-k5q55\" (UID: \"77c1aa9d-afba-4bb3-b583-d26d66592136\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-k5q55"
Oct 30 19:39:08.771967 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:08.771831    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77c1aa9d-afba-4bb3-b583-d26d66592136-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-k5q55\" (UID: \"77c1aa9d-afba-4bb3-b583-d26d66592136\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-k5q55"
Oct 30 19:39:08.773675 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:08.773645    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77c1aa9d-afba-4bb3-b583-d26d66592136-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-k5q55\" (UID: \"77c1aa9d-afba-4bb3-b583-d26d66592136\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-k5q55"
Oct 30 19:39:08.773973 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:08.773950    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77c1aa9d-afba-4bb3-b583-d26d66592136-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-k5q55\" (UID: \"77c1aa9d-afba-4bb3-b583-d26d66592136\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-k5q55"
Oct 30 19:39:08.783757 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:08.783724    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fn7nl\" (UniqueName: \"kubernetes.io/projected/77c1aa9d-afba-4bb3-b583-d26d66592136-kube-api-access-fn7nl\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-k5q55\" (UID: \"77c1aa9d-afba-4bb3-b583-d26d66592136\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-k5q55"
Oct 30 19:39:08.901829 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:08.901725    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-k5q55"
Oct 30 19:39:09.189642 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:09.189563    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-k5q55"]
Oct 30 19:39:09.192744 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:39:09.192710    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77c1aa9d_afba_4bb3_b583_d26d66592136.slice/crio-220e5dc66b5e09f6a2ca0b0ce6ef6daea5417b8ce875f5af7a05a58ca331777a WatchSource:0}: Error finding container 220e5dc66b5e09f6a2ca0b0ce6ef6daea5417b8ce875f5af7a05a58ca331777a: Status 404 returned error can't find the container with id 220e5dc66b5e09f6a2ca0b0ce6ef6daea5417b8ce875f5af7a05a58ca331777a
Oct 30 19:39:09.284373 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:09.284333    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 19:39:10.039811 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:10.039775    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-k5q55" event={"ID":"77c1aa9d-afba-4bb3-b583-d26d66592136","Type":"ContainerStarted","Data":"220e5dc66b5e09f6a2ca0b0ce6ef6daea5417b8ce875f5af7a05a58ca331777a"}
Oct 30 19:39:11.045634 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:11.045599    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-k5q55" event={"ID":"77c1aa9d-afba-4bb3-b583-d26d66592136","Type":"ContainerStarted","Data":"9fd1169d3160fc75f5dc044900f9dd98d3adb2cca304200ce288c37fddc2aa1f"}
Oct 30 19:39:11.067521 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:11.067459    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-k5q55" podStartSLOduration=1.746813108 podStartE2EDuration="3.067445368s" totalImagesPullingTime="1.32063226s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:39:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:39:11.065973481 +0000 UTC m=+426.394091635" watchObservedRunningTime="2026-10-30 19:39:11.067445368 +0000 UTC m=+426.395563521"
Oct 30 19:39:11.902232 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:11.902201    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-k5q55"
Oct 30 19:39:12.000299 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:12.000253    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 19:39:12.932445 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:12.932413    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-6jh27"]
Oct 30 19:39:12.940392 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:12.940373    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-6jh27"
Oct 30 19:39:12.951465 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:12.951442    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Oct 30 19:39:12.961431 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:12.961405    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-6jh27"]
Oct 30 19:39:13.016534 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:13.016501    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41ed96e2-0939-4c7b-9c32-ab8d4025c577-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-6jh27\" (UID: \"41ed96e2-0939-4c7b-9c32-ab8d4025c577\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-6jh27"
Oct 30 19:39:13.016717 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:13.016557    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41ed96e2-0939-4c7b-9c32-ab8d4025c577-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-6jh27\" (UID: \"41ed96e2-0939-4c7b-9c32-ab8d4025c577\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-6jh27"
Oct 30 19:39:13.016717 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:13.016588    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41ed96e2-0939-4c7b-9c32-ab8d4025c577-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-6jh27\" (UID: \"41ed96e2-0939-4c7b-9c32-ab8d4025c577\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-6jh27"
Oct 30 19:39:13.016818 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:13.016705    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41ed96e2-0939-4c7b-9c32-ab8d4025c577-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-6jh27\" (UID: \"41ed96e2-0939-4c7b-9c32-ab8d4025c577\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-6jh27"
Oct 30 19:39:13.016818 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:13.016751    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41ed96e2-0939-4c7b-9c32-ab8d4025c577-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-6jh27\" (UID: \"41ed96e2-0939-4c7b-9c32-ab8d4025c577\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-6jh27"
Oct 30 19:39:13.016879 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:13.016834    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5cljm\" (UniqueName: \"kubernetes.io/projected/41ed96e2-0939-4c7b-9c32-ab8d4025c577-kube-api-access-5cljm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-6jh27\" (UID: \"41ed96e2-0939-4c7b-9c32-ab8d4025c577\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-6jh27"
Oct 30 19:39:13.117640 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:13.117603    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41ed96e2-0939-4c7b-9c32-ab8d4025c577-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-6jh27\" (UID: \"41ed96e2-0939-4c7b-9c32-ab8d4025c577\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-6jh27"
Oct 30 19:39:13.117640 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:13.117642    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41ed96e2-0939-4c7b-9c32-ab8d4025c577-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-6jh27\" (UID: \"41ed96e2-0939-4c7b-9c32-ab8d4025c577\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-6jh27"
Oct 30 19:39:13.117892 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:13.117665    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cljm\" (UniqueName: \"kubernetes.io/projected/41ed96e2-0939-4c7b-9c32-ab8d4025c577-kube-api-access-5cljm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-6jh27\" (UID: \"41ed96e2-0939-4c7b-9c32-ab8d4025c577\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-6jh27"
Oct 30 19:39:13.117892 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:13.117706    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41ed96e2-0939-4c7b-9c32-ab8d4025c577-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-6jh27\" (UID: \"41ed96e2-0939-4c7b-9c32-ab8d4025c577\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-6jh27"
Oct 30 19:39:13.117892 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:13.117731    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41ed96e2-0939-4c7b-9c32-ab8d4025c577-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-6jh27\" (UID: \"41ed96e2-0939-4c7b-9c32-ab8d4025c577\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-6jh27"
Oct 30 19:39:13.117892 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:13.117765    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41ed96e2-0939-4c7b-9c32-ab8d4025c577-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-6jh27\" (UID: \"41ed96e2-0939-4c7b-9c32-ab8d4025c577\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-6jh27"
Oct 30 19:39:13.118165 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:13.118145    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41ed96e2-0939-4c7b-9c32-ab8d4025c577-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-6jh27\" (UID: \"41ed96e2-0939-4c7b-9c32-ab8d4025c577\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-6jh27"
Oct 30 19:39:13.118226 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:13.118182    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41ed96e2-0939-4c7b-9c32-ab8d4025c577-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-6jh27\" (UID: \"41ed96e2-0939-4c7b-9c32-ab8d4025c577\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-6jh27"
Oct 30 19:39:13.118273 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:13.118249    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41ed96e2-0939-4c7b-9c32-ab8d4025c577-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-6jh27\" (UID: \"41ed96e2-0939-4c7b-9c32-ab8d4025c577\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-6jh27"
Oct 30 19:39:13.119953 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:13.119928    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41ed96e2-0939-4c7b-9c32-ab8d4025c577-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-6jh27\" (UID: \"41ed96e2-0939-4c7b-9c32-ab8d4025c577\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-6jh27"
Oct 30 19:39:13.120117 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:13.120100    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41ed96e2-0939-4c7b-9c32-ab8d4025c577-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-6jh27\" (UID: \"41ed96e2-0939-4c7b-9c32-ab8d4025c577\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-6jh27"
Oct 30 19:39:13.130985 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:13.130955    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5cljm\" (UniqueName: \"kubernetes.io/projected/41ed96e2-0939-4c7b-9c32-ab8d4025c577-kube-api-access-5cljm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-6jh27\" (UID: \"41ed96e2-0939-4c7b-9c32-ab8d4025c577\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-6jh27"
Oct 30 19:39:13.253265 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:13.253183    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-6jh27"
Oct 30 19:39:13.550422 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:13.550394    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-6jh27"]
Oct 30 19:39:13.551968 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:39:13.551941    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41ed96e2_0939_4c7b_9c32_ab8d4025c577.slice/crio-f297a058bcc75777d4996ddb195d1e63dc59cb0cef2c982d01242c5d7c80b65f WatchSource:0}: Error finding container f297a058bcc75777d4996ddb195d1e63dc59cb0cef2c982d01242c5d7c80b65f: Status 404 returned error can't find the container with id f297a058bcc75777d4996ddb195d1e63dc59cb0cef2c982d01242c5d7c80b65f
Oct 30 19:39:13.763462 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:13.763420    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 19:39:14.066291 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:14.066255    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-6jh27" event={"ID":"41ed96e2-0939-4c7b-9c32-ab8d4025c577","Type":"ContainerStarted","Data":"1e566cb69b6746c8c9b3069e9cf580462d084eeba55e8a7bb8bcbc0b8a3ef84c"}
Oct 30 19:39:14.066705 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:14.066312    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-6jh27" event={"ID":"41ed96e2-0939-4c7b-9c32-ab8d4025c577","Type":"ContainerStarted","Data":"f297a058bcc75777d4996ddb195d1e63dc59cb0cef2c982d01242c5d7c80b65f"}
Oct 30 19:39:14.097925 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:14.097770    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-6jh27" podStartSLOduration=1.8249240260000001 podStartE2EDuration="2.097749736s" totalImagesPullingTime="272.82571ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:39:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:39:14.096772622 +0000 UTC m=+429.424890784" watchObservedRunningTime="2026-10-30 19:39:14.097749736 +0000 UTC m=+429.425867890"
Oct 30 19:39:14.253532 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:14.253487    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-6jh27"
Oct 30 19:39:16.247064 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:16.247032    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-jt9lh"]
Oct 30 19:39:16.250654 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:16.250634    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-jt9lh"
Oct 30 19:39:16.257243 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:16.257223    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Oct 30 19:39:16.270921 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:16.270898    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-jt9lh"]
Oct 30 19:39:16.348780 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:16.348750    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b701a7d1-2f99-40d8-bc31-6f120f695f6b-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-jt9lh\" (UID: \"b701a7d1-2f99-40d8-bc31-6f120f695f6b\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-jt9lh"
Oct 30 19:39:16.348935 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:16.348794    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b701a7d1-2f99-40d8-bc31-6f120f695f6b-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-jt9lh\" (UID: \"b701a7d1-2f99-40d8-bc31-6f120f695f6b\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-jt9lh"
Oct 30 19:39:16.348935 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:16.348878    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b701a7d1-2f99-40d8-bc31-6f120f695f6b-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-jt9lh\" (UID: \"b701a7d1-2f99-40d8-bc31-6f120f695f6b\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-jt9lh"
Oct 30 19:39:16.348935 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:16.348915    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b701a7d1-2f99-40d8-bc31-6f120f695f6b-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-jt9lh\" (UID: \"b701a7d1-2f99-40d8-bc31-6f120f695f6b\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-jt9lh"
Oct 30 19:39:16.349121 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:16.349014    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b701a7d1-2f99-40d8-bc31-6f120f695f6b-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-jt9lh\" (UID: \"b701a7d1-2f99-40d8-bc31-6f120f695f6b\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-jt9lh"
Oct 30 19:39:16.349171 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:16.349134    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmjf2\" (UniqueName: \"kubernetes.io/projected/b701a7d1-2f99-40d8-bc31-6f120f695f6b-kube-api-access-qmjf2\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-jt9lh\" (UID: \"b701a7d1-2f99-40d8-bc31-6f120f695f6b\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-jt9lh"
Oct 30 19:39:16.450352 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:16.450322    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmjf2\" (UniqueName: \"kubernetes.io/projected/b701a7d1-2f99-40d8-bc31-6f120f695f6b-kube-api-access-qmjf2\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-jt9lh\" (UID: \"b701a7d1-2f99-40d8-bc31-6f120f695f6b\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-jt9lh"
Oct 30 19:39:16.450576 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:16.450360    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b701a7d1-2f99-40d8-bc31-6f120f695f6b-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-jt9lh\" (UID: \"b701a7d1-2f99-40d8-bc31-6f120f695f6b\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-jt9lh"
Oct 30 19:39:16.450576 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:16.450391    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b701a7d1-2f99-40d8-bc31-6f120f695f6b-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-jt9lh\" (UID: \"b701a7d1-2f99-40d8-bc31-6f120f695f6b\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-jt9lh"
Oct 30 19:39:16.450576 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:16.450412    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b701a7d1-2f99-40d8-bc31-6f120f695f6b-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-jt9lh\" (UID: \"b701a7d1-2f99-40d8-bc31-6f120f695f6b\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-jt9lh"
Oct 30 19:39:16.450576 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:16.450428    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b701a7d1-2f99-40d8-bc31-6f120f695f6b-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-jt9lh\" (UID: \"b701a7d1-2f99-40d8-bc31-6f120f695f6b\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-jt9lh"
Oct 30 19:39:16.450576 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:16.450460    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b701a7d1-2f99-40d8-bc31-6f120f695f6b-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-jt9lh\" (UID: \"b701a7d1-2f99-40d8-bc31-6f120f695f6b\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-jt9lh"
Oct 30 19:39:16.450864 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:16.450840    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b701a7d1-2f99-40d8-bc31-6f120f695f6b-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-jt9lh\" (UID: \"b701a7d1-2f99-40d8-bc31-6f120f695f6b\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-jt9lh"
Oct 30 19:39:16.450904 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:16.450839    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b701a7d1-2f99-40d8-bc31-6f120f695f6b-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-jt9lh\" (UID: \"b701a7d1-2f99-40d8-bc31-6f120f695f6b\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-jt9lh"
Oct 30 19:39:16.450904 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:16.450884    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b701a7d1-2f99-40d8-bc31-6f120f695f6b-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-jt9lh\" (UID: \"b701a7d1-2f99-40d8-bc31-6f120f695f6b\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-jt9lh"
Oct 30 19:39:16.452663 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:16.452637    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b701a7d1-2f99-40d8-bc31-6f120f695f6b-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-jt9lh\" (UID: \"b701a7d1-2f99-40d8-bc31-6f120f695f6b\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-jt9lh"
Oct 30 19:39:16.452963 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:16.452946    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b701a7d1-2f99-40d8-bc31-6f120f695f6b-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-jt9lh\" (UID: \"b701a7d1-2f99-40d8-bc31-6f120f695f6b\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-jt9lh"
Oct 30 19:39:16.463052 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:16.463028    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmjf2\" (UniqueName: \"kubernetes.io/projected/b701a7d1-2f99-40d8-bc31-6f120f695f6b-kube-api-access-qmjf2\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-jt9lh\" (UID: \"b701a7d1-2f99-40d8-bc31-6f120f695f6b\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-jt9lh"
Oct 30 19:39:16.564189 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:16.564108    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-jt9lh"
Oct 30 19:39:16.844746 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:16.844711    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-jt9lh"]
Oct 30 19:39:16.847327 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:39:16.847301    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb701a7d1_2f99_40d8_bc31_6f120f695f6b.slice/crio-d502a28513a97107500abc070a89639b205469dcc48e024c9a8f2e690be14134 WatchSource:0}: Error finding container d502a28513a97107500abc070a89639b205469dcc48e024c9a8f2e690be14134: Status 404 returned error can't find the container with id d502a28513a97107500abc070a89639b205469dcc48e024c9a8f2e690be14134
Oct 30 19:39:17.083818 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:17.083781    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-jt9lh" event={"ID":"b701a7d1-2f99-40d8-bc31-6f120f695f6b","Type":"ContainerStarted","Data":"d502a28513a97107500abc070a89639b205469dcc48e024c9a8f2e690be14134"}
Oct 30 19:39:17.316503 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:17.302609    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 19:39:17.700775 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:17.700742    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zcm5m"]
Oct 30 19:39:17.704728 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:17.704705    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zcm5m"
Oct 30 19:39:17.721309 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:17.721283    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Oct 30 19:39:17.723453 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:17.723425    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zcm5m"]
Oct 30 19:39:17.865074 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:17.865031    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skxsw\" (UniqueName: \"kubernetes.io/projected/bbb6c18e-0752-4ad7-a42a-ee5ffb73d58b-kube-api-access-skxsw\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zcm5m\" (UID: \"bbb6c18e-0752-4ad7-a42a-ee5ffb73d58b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zcm5m"
Oct 30 19:39:17.865271 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:17.865081    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bbb6c18e-0752-4ad7-a42a-ee5ffb73d58b-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zcm5m\" (UID: \"bbb6c18e-0752-4ad7-a42a-ee5ffb73d58b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zcm5m"
Oct 30 19:39:17.865271 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:17.865226    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bbb6c18e-0752-4ad7-a42a-ee5ffb73d58b-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zcm5m\" (UID: \"bbb6c18e-0752-4ad7-a42a-ee5ffb73d58b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zcm5m"
Oct 30 19:39:17.865378 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:17.865273    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bbb6c18e-0752-4ad7-a42a-ee5ffb73d58b-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zcm5m\" (UID: \"bbb6c18e-0752-4ad7-a42a-ee5ffb73d58b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zcm5m"
Oct 30 19:39:17.865583 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:17.865548    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bbb6c18e-0752-4ad7-a42a-ee5ffb73d58b-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zcm5m\" (UID: \"bbb6c18e-0752-4ad7-a42a-ee5ffb73d58b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zcm5m"
Oct 30 19:39:17.865719 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:17.865683    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bbb6c18e-0752-4ad7-a42a-ee5ffb73d58b-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zcm5m\" (UID: \"bbb6c18e-0752-4ad7-a42a-ee5ffb73d58b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zcm5m"
Oct 30 19:39:17.966645 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:17.966551    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bbb6c18e-0752-4ad7-a42a-ee5ffb73d58b-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zcm5m\" (UID: \"bbb6c18e-0752-4ad7-a42a-ee5ffb73d58b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zcm5m"
Oct 30 19:39:17.966645 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:17.966610    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skxsw\" (UniqueName: \"kubernetes.io/projected/bbb6c18e-0752-4ad7-a42a-ee5ffb73d58b-kube-api-access-skxsw\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zcm5m\" (UID: \"bbb6c18e-0752-4ad7-a42a-ee5ffb73d58b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zcm5m"
Oct 30 19:39:17.966848 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:17.966653    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bbb6c18e-0752-4ad7-a42a-ee5ffb73d58b-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zcm5m\" (UID: \"bbb6c18e-0752-4ad7-a42a-ee5ffb73d58b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zcm5m"
Oct 30 19:39:17.966848 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:17.966710    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bbb6c18e-0752-4ad7-a42a-ee5ffb73d58b-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zcm5m\" (UID: \"bbb6c18e-0752-4ad7-a42a-ee5ffb73d58b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zcm5m"
Oct 30 19:39:17.966848 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:17.966750    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bbb6c18e-0752-4ad7-a42a-ee5ffb73d58b-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zcm5m\" (UID: \"bbb6c18e-0752-4ad7-a42a-ee5ffb73d58b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zcm5m"
Oct 30 19:39:17.966848 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:17.966821    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bbb6c18e-0752-4ad7-a42a-ee5ffb73d58b-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zcm5m\" (UID: \"bbb6c18e-0752-4ad7-a42a-ee5ffb73d58b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zcm5m"
Oct 30 19:39:17.967070 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:17.967036    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bbb6c18e-0752-4ad7-a42a-ee5ffb73d58b-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zcm5m\" (UID: \"bbb6c18e-0752-4ad7-a42a-ee5ffb73d58b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zcm5m"
Oct 30 19:39:17.967344 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:17.967322    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bbb6c18e-0752-4ad7-a42a-ee5ffb73d58b-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zcm5m\" (UID: \"bbb6c18e-0752-4ad7-a42a-ee5ffb73d58b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zcm5m"
Oct 30 19:39:17.967466 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:17.967445    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bbb6c18e-0752-4ad7-a42a-ee5ffb73d58b-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zcm5m\" (UID: \"bbb6c18e-0752-4ad7-a42a-ee5ffb73d58b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zcm5m"
Oct 30 19:39:17.970428 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:17.970402    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bbb6c18e-0752-4ad7-a42a-ee5ffb73d58b-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zcm5m\" (UID: \"bbb6c18e-0752-4ad7-a42a-ee5ffb73d58b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zcm5m"
Oct 30 19:39:17.970795 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:17.970772    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bbb6c18e-0752-4ad7-a42a-ee5ffb73d58b-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zcm5m\" (UID: \"bbb6c18e-0752-4ad7-a42a-ee5ffb73d58b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zcm5m"
Oct 30 19:39:17.981558 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:17.981526    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-skxsw\" (UniqueName: \"kubernetes.io/projected/bbb6c18e-0752-4ad7-a42a-ee5ffb73d58b-kube-api-access-skxsw\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zcm5m\" (UID: \"bbb6c18e-0752-4ad7-a42a-ee5ffb73d58b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zcm5m"
Oct 30 19:39:18.022687 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:18.022646    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zcm5m"
Oct 30 19:39:18.113040 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:18.110027    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-jt9lh" event={"ID":"b701a7d1-2f99-40d8-bc31-6f120f695f6b","Type":"ContainerStarted","Data":"92823d559916b2f4eaaf58ad8400d6b91624b5ec1c9e593eecada9bfa619da23"}
Oct 30 19:39:18.335067 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:18.335011    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-jt9lh" podStartSLOduration=2.050262585 podStartE2EDuration="2.334994931s" totalImagesPullingTime="284.732346ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:39:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:39:18.14255202 +0000 UTC m=+433.470670174" watchObservedRunningTime="2026-10-30 19:39:18.334994931 +0000 UTC m=+433.663113082"
Oct 30 19:39:18.336499 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:18.336460    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zcm5m"]
Oct 30 19:39:18.339451 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:39:18.339424    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbb6c18e_0752_4ad7_a42a_ee5ffb73d58b.slice/crio-23b0f040eb12520c6859a6c0ea056a8ed1167bfb7195a69fb0e925ad88b4a3d4 WatchSource:0}: Error finding container 23b0f040eb12520c6859a6c0ea056a8ed1167bfb7195a69fb0e925ad88b4a3d4: Status 404 returned error can't find the container with id 23b0f040eb12520c6859a6c0ea056a8ed1167bfb7195a69fb0e925ad88b4a3d4
Oct 30 19:39:18.565232 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:18.565192    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-jt9lh"
Oct 30 19:39:18.901959 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:18.901870    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-k5q55"
Oct 30 19:39:18.917454 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:18.917427    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-k5q55"
Oct 30 19:39:19.119045 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:19.119008    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zcm5m" event={"ID":"bbb6c18e-0752-4ad7-a42a-ee5ffb73d58b","Type":"ContainerStarted","Data":"93eeb4f9390d1c6e4367b3850409b6a35bc6894ead35622ab5b9c10f8c1fc65c"}
Oct 30 19:39:19.119231 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:19.119065    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zcm5m" event={"ID":"bbb6c18e-0752-4ad7-a42a-ee5ffb73d58b","Type":"ContainerStarted","Data":"23b0f040eb12520c6859a6c0ea056a8ed1167bfb7195a69fb0e925ad88b4a3d4"}
Oct 30 19:39:19.130767 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:19.130740    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-k5q55"
Oct 30 19:39:19.162664 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:19.162567    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zcm5m" podStartSLOduration=1.877076399 podStartE2EDuration="2.162554347s" totalImagesPullingTime="285.477948ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:39:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:39:19.1586637 +0000 UTC m=+434.486781854" watchObservedRunningTime="2026-10-30 19:39:19.162554347 +0000 UTC m=+434.490672501"
Oct 30 19:39:20.023440 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:20.023408    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zcm5m"
Oct 30 19:39:20.349502 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:20.349456    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 19:39:23.254360 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:23.254319    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-6jh27"
Oct 30 19:39:23.269698 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:23.269673    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-6jh27"
Oct 30 19:39:24.157651 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:24.157617    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-6jh27"
Oct 30 19:39:25.953037 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:25.953000    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 19:39:26.565170 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:26.565135    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-jt9lh"
Oct 30 19:39:26.577898 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:26.577875    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-jt9lh"
Oct 30 19:39:27.172073 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:27.172047    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-jt9lh"
Oct 30 19:39:28.023101 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:28.023069    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zcm5m"
Oct 30 19:39:28.035693 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:28.035655    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zcm5m"
Oct 30 19:39:28.177712 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:28.177684    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zcm5m"
Oct 30 19:39:31.189848 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:39:31.189808    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 19:39:38.070823 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:38.070788    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 19:39:38.563353 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:38.563271    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5677757b6d-m57tz"]
Oct 30 19:39:38.568249 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:38.568231    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5677757b6d-m57tz"
Oct 30 19:39:38.577504 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:38.577461    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5677757b6d-m57tz"]
Oct 30 19:39:38.691046 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:38.691014    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a8eded1a-5c8c-4740-8f7a-dfc5c05c0bcd-tls-cert\") pod \"authorino-5677757b6d-m57tz\" (UID: \"a8eded1a-5c8c-4740-8f7a-dfc5c05c0bcd\") " pod="kuadrant-system/authorino-5677757b6d-m57tz"
Oct 30 19:39:38.691212 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:38.691157    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kn9r4\" (UniqueName: \"kubernetes.io/projected/a8eded1a-5c8c-4740-8f7a-dfc5c05c0bcd-kube-api-access-kn9r4\") pod \"authorino-5677757b6d-m57tz\" (UID: \"a8eded1a-5c8c-4740-8f7a-dfc5c05c0bcd\") " pod="kuadrant-system/authorino-5677757b6d-m57tz"
Oct 30 19:39:38.792294 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:38.792262    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kn9r4\" (UniqueName: \"kubernetes.io/projected/a8eded1a-5c8c-4740-8f7a-dfc5c05c0bcd-kube-api-access-kn9r4\") pod \"authorino-5677757b6d-m57tz\" (UID: \"a8eded1a-5c8c-4740-8f7a-dfc5c05c0bcd\") " pod="kuadrant-system/authorino-5677757b6d-m57tz"
Oct 30 19:39:38.792451 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:38.792319    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a8eded1a-5c8c-4740-8f7a-dfc5c05c0bcd-tls-cert\") pod \"authorino-5677757b6d-m57tz\" (UID: \"a8eded1a-5c8c-4740-8f7a-dfc5c05c0bcd\") " pod="kuadrant-system/authorino-5677757b6d-m57tz"
Oct 30 19:39:38.794832 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:38.794802    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a8eded1a-5c8c-4740-8f7a-dfc5c05c0bcd-tls-cert\") pod \"authorino-5677757b6d-m57tz\" (UID: \"a8eded1a-5c8c-4740-8f7a-dfc5c05c0bcd\") " pod="kuadrant-system/authorino-5677757b6d-m57tz"
Oct 30 19:39:38.804524 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:38.804501    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kn9r4\" (UniqueName: \"kubernetes.io/projected/a8eded1a-5c8c-4740-8f7a-dfc5c05c0bcd-kube-api-access-kn9r4\") pod \"authorino-5677757b6d-m57tz\" (UID: \"a8eded1a-5c8c-4740-8f7a-dfc5c05c0bcd\") " pod="kuadrant-system/authorino-5677757b6d-m57tz"
Oct 30 19:39:38.881023 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:38.880994    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5677757b6d-m57tz"
Oct 30 19:39:39.156570 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:39.156542    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5677757b6d-m57tz"]
Oct 30 19:39:39.158683 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:39:39.158649    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda8eded1a_5c8c_4740_8f7a_dfc5c05c0bcd.slice/crio-5f3b038ea38f25536baba91649581f40da5725f4eb499588fddaee65ea46a33b WatchSource:0}: Error finding container 5f3b038ea38f25536baba91649581f40da5725f4eb499588fddaee65ea46a33b: Status 404 returned error can't find the container with id 5f3b038ea38f25536baba91649581f40da5725f4eb499588fddaee65ea46a33b
Oct 30 19:39:39.222458 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:39.222424    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5677757b6d-m57tz" event={"ID":"a8eded1a-5c8c-4740-8f7a-dfc5c05c0bcd","Type":"ContainerStarted","Data":"5f3b038ea38f25536baba91649581f40da5725f4eb499588fddaee65ea46a33b"}
Oct 30 19:39:40.228924 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:40.228879    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5677757b6d-m57tz" event={"ID":"a8eded1a-5c8c-4740-8f7a-dfc5c05c0bcd","Type":"ContainerStarted","Data":"da3c5b157da47343e4fddcf7ca4d520c55291539d3c0154149ef6e993e4fd3b1"}
Oct 30 19:39:40.249850 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:40.249804    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-5677757b6d-m57tz" podStartSLOduration=1.278702791 podStartE2EDuration="2.249788908s" totalImagesPullingTime="971.086117ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:39:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:39:40.248027515 +0000 UTC m=+455.576145669" watchObservedRunningTime="2026-10-30 19:39:40.249788908 +0000 UTC m=+455.577907061"
Oct 30 19:39:40.277132 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:40.277089    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-84f585c5-n9z4n"]
Oct 30 19:39:40.277674 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:40.277640    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-84f585c5-n9z4n" podUID="cce9d43a-54e7-4417-b19d-292afa15e88b" containerName="authorino" containerID="cri-o://382c1911026b81a4c942a058f294dd868fe1037073050b9d22093a0cf98f6eb9" gracePeriod=30
Oct 30 19:39:40.540909 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:40.539564    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-84f585c5-n9z4n"
Oct 30 19:39:40.611085 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:40.611043    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cce9d43a-54e7-4417-b19d-292afa15e88b-tls-cert\" (UniqueName: \"kubernetes.io/secret/cce9d43a-54e7-4417-b19d-292afa15e88b-tls-cert\") pod \"cce9d43a-54e7-4417-b19d-292afa15e88b\" (UID: \"cce9d43a-54e7-4417-b19d-292afa15e88b\") "
Oct 30 19:39:40.611282 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:40.611164    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cce9d43a-54e7-4417-b19d-292afa15e88b-kube-api-access-vms5n\" (UniqueName: \"kubernetes.io/projected/cce9d43a-54e7-4417-b19d-292afa15e88b-kube-api-access-vms5n\") pod \"cce9d43a-54e7-4417-b19d-292afa15e88b\" (UID: \"cce9d43a-54e7-4417-b19d-292afa15e88b\") "
Oct 30 19:39:40.613774 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:40.613737    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cce9d43a-54e7-4417-b19d-292afa15e88b-kube-api-access-vms5n" pod "cce9d43a-54e7-4417-b19d-292afa15e88b" (UID: "cce9d43a-54e7-4417-b19d-292afa15e88b"). InnerVolumeSpecName "kube-api-access-vms5n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:39:40.622810 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:40.622783    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cce9d43a-54e7-4417-b19d-292afa15e88b-tls-cert" pod "cce9d43a-54e7-4417-b19d-292afa15e88b" (UID: "cce9d43a-54e7-4417-b19d-292afa15e88b"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:39:40.712204 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:40.712115    6996 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/cce9d43a-54e7-4417-b19d-292afa15e88b-tls-cert\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:39:40.712204 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:40.712148    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vms5n\" (UniqueName: \"kubernetes.io/projected/cce9d43a-54e7-4417-b19d-292afa15e88b-kube-api-access-vms5n\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:39:41.236790 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:41.236751    6996 generic.go:356] "Generic (PLEG): container finished" podID="cce9d43a-54e7-4417-b19d-292afa15e88b" containerID="382c1911026b81a4c942a058f294dd868fe1037073050b9d22093a0cf98f6eb9" exitCode=0
Oct 30 19:39:41.237270 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:41.236815    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-84f585c5-n9z4n"
Oct 30 19:39:41.237270 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:41.236840    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-84f585c5-n9z4n" event={"ID":"cce9d43a-54e7-4417-b19d-292afa15e88b","Type":"ContainerDied","Data":"382c1911026b81a4c942a058f294dd868fe1037073050b9d22093a0cf98f6eb9"}
Oct 30 19:39:41.237270 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:41.236902    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-84f585c5-n9z4n" event={"ID":"cce9d43a-54e7-4417-b19d-292afa15e88b","Type":"ContainerDied","Data":"a5157f6607f9e79fe2631c19c71ff4a6fe09c073aafddf0b7c53f02b3f2b0eac"}
Oct 30 19:39:41.237270 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:41.236928    6996 scope.go:122] "RemoveContainer" containerID="382c1911026b81a4c942a058f294dd868fe1037073050b9d22093a0cf98f6eb9"
Oct 30 19:39:41.247704 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:41.247670    6996 scope.go:122] "RemoveContainer" containerID="382c1911026b81a4c942a058f294dd868fe1037073050b9d22093a0cf98f6eb9"
Oct 30 19:39:41.248292 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:39:41.248264    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"382c1911026b81a4c942a058f294dd868fe1037073050b9d22093a0cf98f6eb9\": container with ID starting with 382c1911026b81a4c942a058f294dd868fe1037073050b9d22093a0cf98f6eb9 not found: ID does not exist" containerID="382c1911026b81a4c942a058f294dd868fe1037073050b9d22093a0cf98f6eb9"
Oct 30 19:39:41.248396 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:41.248303    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"382c1911026b81a4c942a058f294dd868fe1037073050b9d22093a0cf98f6eb9"} err="failed to get container status \"382c1911026b81a4c942a058f294dd868fe1037073050b9d22093a0cf98f6eb9\": rpc error: code = NotFound desc = could not find container \"382c1911026b81a4c942a058f294dd868fe1037073050b9d22093a0cf98f6eb9\": container with ID starting with 382c1911026b81a4c942a058f294dd868fe1037073050b9d22093a0cf98f6eb9 not found: ID does not exist"
Oct 30 19:39:41.260084 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:41.260051    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-84f585c5-n9z4n"]
Oct 30 19:39:41.268846 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:41.268823    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-84f585c5-n9z4n"]
Oct 30 19:39:42.793028 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:42.792988    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cce9d43a-54e7-4417-b19d-292afa15e88b" path="/var/lib/kubelet/pods/cce9d43a-54e7-4417-b19d-292afa15e88b/volumes"
Oct 30 19:39:46.987110 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:46.987026    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sjbzh"]
Oct 30 19:39:46.987502 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:46.987469    6996 cpu_manager.go:414] "containerMap: removing container" podUID="cce9d43a-54e7-4417-b19d-292afa15e88b" containerName="authorino"
Oct 30 19:39:46.987502 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:46.987499    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cce9d43a-54e7-4417-b19d-292afa15e88b" containerName="authorino"
Oct 30 19:39:46.987590 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:46.987578    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="cce9d43a-54e7-4417-b19d-292afa15e88b" containerName="authorino"
Oct 30 19:39:46.992851 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:46.992831    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sjbzh"
Oct 30 19:39:46.996685 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:46.996659    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-9slkz\""
Oct 30 19:39:47.008321 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:47.008296    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sjbzh"]
Oct 30 19:39:47.048614 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:47.048585    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Oct 30 19:39:47.084285 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:47.084253    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b79c69dc-9d5b-4b3b-802e-ac414149df33-credential-socket\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-sjbzh\" (UID: \"b79c69dc-9d5b-4b3b-802e-ac414149df33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sjbzh"
Oct 30 19:39:47.084285 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:47.084287    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b79c69dc-9d5b-4b3b-802e-ac414149df33-workload-certs\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-sjbzh\" (UID: \"b79c69dc-9d5b-4b3b-802e-ac414149df33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sjbzh"
Oct 30 19:39:47.084548 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:47.084306    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b79c69dc-9d5b-4b3b-802e-ac414149df33-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-sjbzh\" (UID: \"b79c69dc-9d5b-4b3b-802e-ac414149df33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sjbzh"
Oct 30 19:39:47.084548 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:47.084400    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b79c69dc-9d5b-4b3b-802e-ac414149df33-workload-socket\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-sjbzh\" (UID: \"b79c69dc-9d5b-4b3b-802e-ac414149df33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sjbzh"
Oct 30 19:39:47.084548 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:47.084467    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b79c69dc-9d5b-4b3b-802e-ac414149df33-istio-token\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-sjbzh\" (UID: \"b79c69dc-9d5b-4b3b-802e-ac414149df33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sjbzh"
Oct 30 19:39:47.084665 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:47.084569    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b79c69dc-9d5b-4b3b-802e-ac414149df33-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-sjbzh\" (UID: \"b79c69dc-9d5b-4b3b-802e-ac414149df33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sjbzh"
Oct 30 19:39:47.084665 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:47.084592    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b79c69dc-9d5b-4b3b-802e-ac414149df33-istio-data\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-sjbzh\" (UID: \"b79c69dc-9d5b-4b3b-802e-ac414149df33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sjbzh"
Oct 30 19:39:47.084761 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:47.084679    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b79c69dc-9d5b-4b3b-802e-ac414149df33-istio-envoy\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-sjbzh\" (UID: \"b79c69dc-9d5b-4b3b-802e-ac414149df33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sjbzh"
Oct 30 19:39:47.084761 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:47.084743    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bff66\" (UniqueName: \"kubernetes.io/projected/b79c69dc-9d5b-4b3b-802e-ac414149df33-kube-api-access-bff66\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-sjbzh\" (UID: \"b79c69dc-9d5b-4b3b-802e-ac414149df33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sjbzh"
Oct 30 19:39:47.186200 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:47.186171    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b79c69dc-9d5b-4b3b-802e-ac414149df33-istio-envoy\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-sjbzh\" (UID: \"b79c69dc-9d5b-4b3b-802e-ac414149df33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sjbzh"
Oct 30 19:39:47.186400 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:47.186213    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bff66\" (UniqueName: \"kubernetes.io/projected/b79c69dc-9d5b-4b3b-802e-ac414149df33-kube-api-access-bff66\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-sjbzh\" (UID: \"b79c69dc-9d5b-4b3b-802e-ac414149df33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sjbzh"
Oct 30 19:39:47.186400 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:47.186339    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b79c69dc-9d5b-4b3b-802e-ac414149df33-credential-socket\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-sjbzh\" (UID: \"b79c69dc-9d5b-4b3b-802e-ac414149df33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sjbzh"
Oct 30 19:39:47.186400 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:47.186381    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b79c69dc-9d5b-4b3b-802e-ac414149df33-workload-certs\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-sjbzh\" (UID: \"b79c69dc-9d5b-4b3b-802e-ac414149df33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sjbzh"
Oct 30 19:39:47.186613 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:47.186406    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b79c69dc-9d5b-4b3b-802e-ac414149df33-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-sjbzh\" (UID: \"b79c69dc-9d5b-4b3b-802e-ac414149df33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sjbzh"
Oct 30 19:39:47.186613 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:47.186449    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b79c69dc-9d5b-4b3b-802e-ac414149df33-workload-socket\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-sjbzh\" (UID: \"b79c69dc-9d5b-4b3b-802e-ac414149df33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sjbzh"
Oct 30 19:39:47.186613 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:47.186500    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b79c69dc-9d5b-4b3b-802e-ac414149df33-istio-token\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-sjbzh\" (UID: \"b79c69dc-9d5b-4b3b-802e-ac414149df33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sjbzh"
Oct 30 19:39:47.186613 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:47.186588    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b79c69dc-9d5b-4b3b-802e-ac414149df33-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-sjbzh\" (UID: \"b79c69dc-9d5b-4b3b-802e-ac414149df33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sjbzh"
Oct 30 19:39:47.186803 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:47.186616    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b79c69dc-9d5b-4b3b-802e-ac414149df33-istio-data\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-sjbzh\" (UID: \"b79c69dc-9d5b-4b3b-802e-ac414149df33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sjbzh"
Oct 30 19:39:47.186803 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:47.186728    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b79c69dc-9d5b-4b3b-802e-ac414149df33-credential-socket\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-sjbzh\" (UID: \"b79c69dc-9d5b-4b3b-802e-ac414149df33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sjbzh"
Oct 30 19:39:47.186803 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:47.186766    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b79c69dc-9d5b-4b3b-802e-ac414149df33-workload-certs\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-sjbzh\" (UID: \"b79c69dc-9d5b-4b3b-802e-ac414149df33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sjbzh"
Oct 30 19:39:47.186961 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:47.186833    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b79c69dc-9d5b-4b3b-802e-ac414149df33-workload-socket\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-sjbzh\" (UID: \"b79c69dc-9d5b-4b3b-802e-ac414149df33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sjbzh"
Oct 30 19:39:47.187056 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:47.187032    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b79c69dc-9d5b-4b3b-802e-ac414149df33-istio-data\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-sjbzh\" (UID: \"b79c69dc-9d5b-4b3b-802e-ac414149df33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sjbzh"
Oct 30 19:39:47.187110 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:47.187078    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b79c69dc-9d5b-4b3b-802e-ac414149df33-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-sjbzh\" (UID: \"b79c69dc-9d5b-4b3b-802e-ac414149df33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sjbzh"
Oct 30 19:39:47.188735 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:47.188705    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b79c69dc-9d5b-4b3b-802e-ac414149df33-istio-envoy\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-sjbzh\" (UID: \"b79c69dc-9d5b-4b3b-802e-ac414149df33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sjbzh"
Oct 30 19:39:47.188944 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:47.188925    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b79c69dc-9d5b-4b3b-802e-ac414149df33-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-sjbzh\" (UID: \"b79c69dc-9d5b-4b3b-802e-ac414149df33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sjbzh"
Oct 30 19:39:47.198507 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:47.198465    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bff66\" (UniqueName: \"kubernetes.io/projected/b79c69dc-9d5b-4b3b-802e-ac414149df33-kube-api-access-bff66\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-sjbzh\" (UID: \"b79c69dc-9d5b-4b3b-802e-ac414149df33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sjbzh"
Oct 30 19:39:47.199668 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:47.199647    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b79c69dc-9d5b-4b3b-802e-ac414149df33-istio-token\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-sjbzh\" (UID: \"b79c69dc-9d5b-4b3b-802e-ac414149df33\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sjbzh"
Oct 30 19:39:47.307349 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:47.307258    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sjbzh"
Oct 30 19:39:47.591226 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:47.591191    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sjbzh"]
Oct 30 19:39:47.594335 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:39:47.594313    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb79c69dc_9d5b_4b3b_802e_ac414149df33.slice/crio-0e7f4c9315bf8c577e58aa1dad3f2bdfa27553f5f8753337beb24b331a1e6870 WatchSource:0}: Error finding container 0e7f4c9315bf8c577e58aa1dad3f2bdfa27553f5f8753337beb24b331a1e6870: Status 404 returned error can't find the container with id 0e7f4c9315bf8c577e58aa1dad3f2bdfa27553f5f8753337beb24b331a1e6870
Oct 30 19:39:47.596271 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:47.596251    6996 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 19:39:48.273796 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:48.273756    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sjbzh" event={"ID":"b79c69dc-9d5b-4b3b-802e-ac414149df33","Type":"ContainerStarted","Data":"0e7f4c9315bf8c577e58aa1dad3f2bdfa27553f5f8753337beb24b331a1e6870"}
Oct 30 19:39:51.174700 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:51.174665    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:39:51.175105 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:51.174749    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:39:51.175105 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:51.174798    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:39:51.297706 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:51.297665    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sjbzh" event={"ID":"b79c69dc-9d5b-4b3b-802e-ac414149df33","Type":"ContainerStarted","Data":"338b722aac784cf1a6f4d68877e9588fe54dc349397a8e14f81a90d14dc15bf2"}
Oct 30 19:39:51.308132 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:51.308102    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sjbzh"
Oct 30 19:39:51.322916 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:51.322842    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sjbzh" podStartSLOduration=1.745476586 podStartE2EDuration="5.322822208s" totalImagesPullingTime="3.577345622s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:39:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:39:51.31998843 +0000 UTC m=+466.648106609" watchObservedRunningTime="2026-10-30 19:39:51.322822208 +0000 UTC m=+466.650940368"
Oct 30 19:39:52.310012 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:52.309978    6996 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5db77fbd9f-sjbzh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Oct 30 19:39:52.310426 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:52.310032    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sjbzh" podUID="b79c69dc-9d5b-4b3b-802e-ac414149df33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Oct 30 19:39:53.308147 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:53.308115    6996 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5db77fbd9f-sjbzh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Oct 30 19:39:53.308340 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:53.308187    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sjbzh" podUID="b79c69dc-9d5b-4b3b-802e-ac414149df33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Oct 30 19:39:54.308355 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:54.308314    6996 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5db77fbd9f-sjbzh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Oct 30 19:39:54.308878 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:54.308377    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sjbzh" podUID="b79c69dc-9d5b-4b3b-802e-ac414149df33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Oct 30 19:39:55.308352 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:55.308311    6996 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5db77fbd9f-sjbzh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Oct 30 19:39:55.308564 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:55.308386    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sjbzh" podUID="b79c69dc-9d5b-4b3b-802e-ac414149df33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Oct 30 19:39:56.307857 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:56.307816    6996 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5db77fbd9f-sjbzh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Oct 30 19:39:56.308045 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:56.307870    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sjbzh" podUID="b79c69dc-9d5b-4b3b-802e-ac414149df33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Oct 30 19:39:57.308247 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:57.308201    6996 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5db77fbd9f-sjbzh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Oct 30 19:39:57.308668 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:57.308267    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sjbzh" podUID="b79c69dc-9d5b-4b3b-802e-ac414149df33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Oct 30 19:39:58.307793 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:58.307751    6996 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5db77fbd9f-sjbzh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Oct 30 19:39:58.307974 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:58.307812    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sjbzh" podUID="b79c69dc-9d5b-4b3b-802e-ac414149df33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Oct 30 19:39:59.438846 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:59.438814    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sjbzh"
Oct 30 19:39:59.439336 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:59.439261    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sjbzh"
Oct 30 19:39:59.440093 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:39:59.440072    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sjbzh"
Oct 30 19:40:01.190779 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:40:01.190741    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 19:40:02.079298 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.079264    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-2rj4k"]
Oct 30 19:40:02.082996 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.082973    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-2rj4k"
Oct 30 19:40:02.161328 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.161294    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sbbjg"]
Oct 30 19:40:02.161904 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.161870    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8d0ef765-f9f8-4f8a-82ad-0a766a663782-istio-envoy\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-2rj4k\" (UID: \"8d0ef765-f9f8-4f8a-82ad-0a766a663782\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-2rj4k"
Oct 30 19:40:02.162055 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.161950    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d0ef765-f9f8-4f8a-82ad-0a766a663782-workload-certs\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-2rj4k\" (UID: \"8d0ef765-f9f8-4f8a-82ad-0a766a663782\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-2rj4k"
Oct 30 19:40:02.162055 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.162013    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8d0ef765-f9f8-4f8a-82ad-0a766a663782-istio-data\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-2rj4k\" (UID: \"8d0ef765-f9f8-4f8a-82ad-0a766a663782\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-2rj4k"
Oct 30 19:40:02.162055 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.162041    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8d0ef765-f9f8-4f8a-82ad-0a766a663782-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-2rj4k\" (UID: \"8d0ef765-f9f8-4f8a-82ad-0a766a663782\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-2rj4k"
Oct 30 19:40:02.162215 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.162186    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d0ef765-f9f8-4f8a-82ad-0a766a663782-istio-token\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-2rj4k\" (UID: \"8d0ef765-f9f8-4f8a-82ad-0a766a663782\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-2rj4k"
Oct 30 19:40:02.162280 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.162229    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvvnf\" (UniqueName: \"kubernetes.io/projected/8d0ef765-f9f8-4f8a-82ad-0a766a663782-kube-api-access-cvvnf\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-2rj4k\" (UID: \"8d0ef765-f9f8-4f8a-82ad-0a766a663782\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-2rj4k"
Oct 30 19:40:02.162337 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.162280    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8d0ef765-f9f8-4f8a-82ad-0a766a663782-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-2rj4k\" (UID: \"8d0ef765-f9f8-4f8a-82ad-0a766a663782\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-2rj4k"
Oct 30 19:40:02.162337 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.162318    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d0ef765-f9f8-4f8a-82ad-0a766a663782-workload-socket\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-2rj4k\" (UID: \"8d0ef765-f9f8-4f8a-82ad-0a766a663782\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-2rj4k"
Oct 30 19:40:02.162440 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.162343    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d0ef765-f9f8-4f8a-82ad-0a766a663782-credential-socket\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-2rj4k\" (UID: \"8d0ef765-f9f8-4f8a-82ad-0a766a663782\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-2rj4k"
Oct 30 19:40:02.166079 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.166060    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sbbjg"
Oct 30 19:40:02.191188 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.191158    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-2rj4k"]
Oct 30 19:40:02.250733 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.250689    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sbbjg"]
Oct 30 19:40:02.262924 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.262894    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d0ef765-f9f8-4f8a-82ad-0a766a663782-workload-socket\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-2rj4k\" (UID: \"8d0ef765-f9f8-4f8a-82ad-0a766a663782\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-2rj4k"
Oct 30 19:40:02.262924 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.262926    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d0ef765-f9f8-4f8a-82ad-0a766a663782-credential-socket\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-2rj4k\" (UID: \"8d0ef765-f9f8-4f8a-82ad-0a766a663782\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-2rj4k"
Oct 30 19:40:02.263166 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.262950    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be-workload-socket\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-sbbjg\" (UID: \"81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sbbjg"
Oct 30 19:40:02.263166 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.262969    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvp96\" (UniqueName: \"kubernetes.io/projected/81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be-kube-api-access-pvp96\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-sbbjg\" (UID: \"81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sbbjg"
Oct 30 19:40:02.263166 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.263039    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8d0ef765-f9f8-4f8a-82ad-0a766a663782-istio-envoy\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-2rj4k\" (UID: \"8d0ef765-f9f8-4f8a-82ad-0a766a663782\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-2rj4k"
Oct 30 19:40:02.263166 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.263124    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be-istio-envoy\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-sbbjg\" (UID: \"81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sbbjg"
Oct 30 19:40:02.263166 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.263155    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be-istio-data\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-sbbjg\" (UID: \"81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sbbjg"
Oct 30 19:40:02.263422 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.263272    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d0ef765-f9f8-4f8a-82ad-0a766a663782-workload-certs\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-2rj4k\" (UID: \"8d0ef765-f9f8-4f8a-82ad-0a766a663782\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-2rj4k"
Oct 30 19:40:02.263422 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.263344    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8d0ef765-f9f8-4f8a-82ad-0a766a663782-istio-data\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-2rj4k\" (UID: \"8d0ef765-f9f8-4f8a-82ad-0a766a663782\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-2rj4k"
Oct 30 19:40:02.263422 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.263377    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8d0ef765-f9f8-4f8a-82ad-0a766a663782-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-2rj4k\" (UID: \"8d0ef765-f9f8-4f8a-82ad-0a766a663782\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-2rj4k"
Oct 30 19:40:02.263422 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.263369    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d0ef765-f9f8-4f8a-82ad-0a766a663782-credential-socket\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-2rj4k\" (UID: \"8d0ef765-f9f8-4f8a-82ad-0a766a663782\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-2rj4k"
Oct 30 19:40:02.263422 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.263386    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d0ef765-f9f8-4f8a-82ad-0a766a663782-workload-socket\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-2rj4k\" (UID: \"8d0ef765-f9f8-4f8a-82ad-0a766a663782\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-2rj4k"
Oct 30 19:40:02.263699 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.263468    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-sbbjg\" (UID: \"81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sbbjg"
Oct 30 19:40:02.263699 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.263563    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be-workload-certs\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-sbbjg\" (UID: \"81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sbbjg"
Oct 30 19:40:02.263699 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.263592    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be-credential-socket\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-sbbjg\" (UID: \"81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sbbjg"
Oct 30 19:40:02.263699 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.263624    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d0ef765-f9f8-4f8a-82ad-0a766a663782-istio-token\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-2rj4k\" (UID: \"8d0ef765-f9f8-4f8a-82ad-0a766a663782\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-2rj4k"
Oct 30 19:40:02.263699 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.263655    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvvnf\" (UniqueName: \"kubernetes.io/projected/8d0ef765-f9f8-4f8a-82ad-0a766a663782-kube-api-access-cvvnf\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-2rj4k\" (UID: \"8d0ef765-f9f8-4f8a-82ad-0a766a663782\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-2rj4k"
Oct 30 19:40:02.263699 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.263679    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-sbbjg\" (UID: \"81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sbbjg"
Oct 30 19:40:02.263699 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.263686    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d0ef765-f9f8-4f8a-82ad-0a766a663782-workload-certs\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-2rj4k\" (UID: \"8d0ef765-f9f8-4f8a-82ad-0a766a663782\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-2rj4k"
Oct 30 19:40:02.263699 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.263698    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8d0ef765-f9f8-4f8a-82ad-0a766a663782-istio-data\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-2rj4k\" (UID: \"8d0ef765-f9f8-4f8a-82ad-0a766a663782\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-2rj4k"
Oct 30 19:40:02.264007 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.263792    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be-istio-token\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-sbbjg\" (UID: \"81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sbbjg"
Oct 30 19:40:02.264007 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.263828    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8d0ef765-f9f8-4f8a-82ad-0a766a663782-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-2rj4k\" (UID: \"8d0ef765-f9f8-4f8a-82ad-0a766a663782\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-2rj4k"
Oct 30 19:40:02.264616 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.264594    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8d0ef765-f9f8-4f8a-82ad-0a766a663782-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-2rj4k\" (UID: \"8d0ef765-f9f8-4f8a-82ad-0a766a663782\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-2rj4k"
Oct 30 19:40:02.265389 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.265366    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8d0ef765-f9f8-4f8a-82ad-0a766a663782-istio-envoy\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-2rj4k\" (UID: \"8d0ef765-f9f8-4f8a-82ad-0a766a663782\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-2rj4k"
Oct 30 19:40:02.266021 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.265991    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8d0ef765-f9f8-4f8a-82ad-0a766a663782-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-2rj4k\" (UID: \"8d0ef765-f9f8-4f8a-82ad-0a766a663782\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-2rj4k"
Oct 30 19:40:02.276706 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.276681    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d0ef765-f9f8-4f8a-82ad-0a766a663782-istio-token\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-2rj4k\" (UID: \"8d0ef765-f9f8-4f8a-82ad-0a766a663782\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-2rj4k"
Oct 30 19:40:02.276879 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.276864    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvvnf\" (UniqueName: \"kubernetes.io/projected/8d0ef765-f9f8-4f8a-82ad-0a766a663782-kube-api-access-cvvnf\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-2rj4k\" (UID: \"8d0ef765-f9f8-4f8a-82ad-0a766a663782\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-2rj4k"
Oct 30 19:40:02.364551 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.364425    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-sbbjg\" (UID: \"81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sbbjg"
Oct 30 19:40:02.364551 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.364528    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be-workload-certs\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-sbbjg\" (UID: \"81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sbbjg"
Oct 30 19:40:02.364786 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.364561    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be-credential-socket\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-sbbjg\" (UID: \"81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sbbjg"
Oct 30 19:40:02.364786 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.364595    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-sbbjg\" (UID: \"81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sbbjg"
Oct 30 19:40:02.364786 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.364638    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be-istio-token\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-sbbjg\" (UID: \"81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sbbjg"
Oct 30 19:40:02.364786 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.364684    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be-workload-socket\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-sbbjg\" (UID: \"81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sbbjg"
Oct 30 19:40:02.364786 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.364709    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvp96\" (UniqueName: \"kubernetes.io/projected/81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be-kube-api-access-pvp96\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-sbbjg\" (UID: \"81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sbbjg"
Oct 30 19:40:02.364786 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.364756    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be-istio-envoy\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-sbbjg\" (UID: \"81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sbbjg"
Oct 30 19:40:02.365091 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.364786    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be-istio-data\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-sbbjg\" (UID: \"81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sbbjg"
Oct 30 19:40:02.365091 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.365003    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be-credential-socket\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-sbbjg\" (UID: \"81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sbbjg"
Oct 30 19:40:02.365205 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.365142    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be-workload-socket\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-sbbjg\" (UID: \"81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sbbjg"
Oct 30 19:40:02.365265 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.365213    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be-istio-data\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-sbbjg\" (UID: \"81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sbbjg"
Oct 30 19:40:02.365324 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.365262    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be-workload-certs\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-sbbjg\" (UID: \"81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sbbjg"
Oct 30 19:40:02.365669 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.365647    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-sbbjg\" (UID: \"81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sbbjg"
Oct 30 19:40:02.367079 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.367053    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-sbbjg\" (UID: \"81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sbbjg"
Oct 30 19:40:02.367159 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.367113    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be-istio-envoy\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-sbbjg\" (UID: \"81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sbbjg"
Oct 30 19:40:02.376773 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.376753    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be-istio-token\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-sbbjg\" (UID: \"81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sbbjg"
Oct 30 19:40:02.376924 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.376907    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvp96\" (UniqueName: \"kubernetes.io/projected/81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be-kube-api-access-pvp96\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-sbbjg\" (UID: \"81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sbbjg"
Oct 30 19:40:02.395779 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.395757    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-2rj4k"
Oct 30 19:40:02.481488 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.481456    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sbbjg"
Oct 30 19:40:02.679456 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.679425    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-2rj4k"]
Oct 30 19:40:02.681025 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:40:02.681001    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d0ef765_f9f8_4f8a_82ad_0a766a663782.slice/crio-fad37487d4da79cb51c85f81b6b674c33dad4ddef481d90caef9a772cd2388db WatchSource:0}: Error finding container fad37487d4da79cb51c85f81b6b674c33dad4ddef481d90caef9a772cd2388db: Status 404 returned error can't find the container with id fad37487d4da79cb51c85f81b6b674c33dad4ddef481d90caef9a772cd2388db
Oct 30 19:40:02.683982 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.683953    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:40:02.684060 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.684021    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:40:02.684060 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.684053    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:40:02.762493 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.762459    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sbbjg"]
Oct 30 19:40:02.764935 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:40:02.764911    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81b16f1b_7e58_4ec5_a7d2_d2ed7e43e5be.slice/crio-c69c5d4cf2d2aeb52fd786928a7168a7622ddc43f9ebcb2c7591abda4f95bd8e WatchSource:0}: Error finding container c69c5d4cf2d2aeb52fd786928a7168a7622ddc43f9ebcb2c7591abda4f95bd8e: Status 404 returned error can't find the container with id c69c5d4cf2d2aeb52fd786928a7168a7622ddc43f9ebcb2c7591abda4f95bd8e
Oct 30 19:40:02.768578 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.768531    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:40:02.768657 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.768633    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:40:02.768715 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:02.768682    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:40:03.389782 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:03.389741    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-2rj4k" event={"ID":"8d0ef765-f9f8-4f8a-82ad-0a766a663782","Type":"ContainerStarted","Data":"0e4b1db12b6da99fb8258d5b38eb0c7b49c53024421b0c92a310d962a32bf424"}
Oct 30 19:40:03.390428 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:03.390397    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-2rj4k" event={"ID":"8d0ef765-f9f8-4f8a-82ad-0a766a663782","Type":"ContainerStarted","Data":"fad37487d4da79cb51c85f81b6b674c33dad4ddef481d90caef9a772cd2388db"}
Oct 30 19:40:03.392471 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:03.392092    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sbbjg" event={"ID":"81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be","Type":"ContainerStarted","Data":"cb4f913e0a9b69bddb3f83fc3d35534e8147f297257953c60d3b837e919b30e4"}
Oct 30 19:40:03.392471 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:03.392155    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sbbjg" event={"ID":"81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be","Type":"ContainerStarted","Data":"c69c5d4cf2d2aeb52fd786928a7168a7622ddc43f9ebcb2c7591abda4f95bd8e"}
Oct 30 19:40:03.396960 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:03.396938    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-2rj4k"
Oct 30 19:40:03.399690 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:03.399663    6996 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5db77fbd9f-2rj4k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Oct 30 19:40:03.399770 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:03.399738    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-2rj4k" podUID="8d0ef765-f9f8-4f8a-82ad-0a766a663782" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Oct 30 19:40:03.415414 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:03.415375    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-2rj4k" podStartSLOduration=1.415363723 podStartE2EDuration="1.415363723s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:40:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:40:03.413600949 +0000 UTC m=+478.741719104" watchObservedRunningTime="2026-10-30 19:40:03.415363723 +0000 UTC m=+478.743481876"
Oct 30 19:40:03.486984 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:03.486575    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sbbjg"
Oct 30 19:40:03.487593 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:03.487571    6996 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5db77fbd9f-sbbjg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Oct 30 19:40:03.487762 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:03.487742    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sbbjg" podUID="81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Oct 30 19:40:04.397062 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:04.397026    6996 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5db77fbd9f-2rj4k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Oct 30 19:40:04.397465 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:04.397088    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-2rj4k" podUID="8d0ef765-f9f8-4f8a-82ad-0a766a663782" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Oct 30 19:40:04.485507 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:04.483876    6996 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5db77fbd9f-sbbjg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Oct 30 19:40:04.485507 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:04.483937    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sbbjg" podUID="81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Oct 30 19:40:05.396533 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:05.396473    6996 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5db77fbd9f-2rj4k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Oct 30 19:40:05.396717 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:05.396558    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-2rj4k" podUID="8d0ef765-f9f8-4f8a-82ad-0a766a663782" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Oct 30 19:40:05.483504 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:05.482943    6996 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5db77fbd9f-sbbjg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Oct 30 19:40:05.483504 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:05.483016    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sbbjg" podUID="81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Oct 30 19:40:06.397039 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:06.396995    6996 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5db77fbd9f-2rj4k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Oct 30 19:40:06.397228 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:06.397052    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-2rj4k" podUID="8d0ef765-f9f8-4f8a-82ad-0a766a663782" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Oct 30 19:40:06.482503 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:06.481924    6996 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5db77fbd9f-sbbjg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Oct 30 19:40:06.482503 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:06.481988    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sbbjg" podUID="81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Oct 30 19:40:07.396201 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:07.396161    6996 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5db77fbd9f-2rj4k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Oct 30 19:40:07.396659 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:07.396217    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-2rj4k" podUID="8d0ef765-f9f8-4f8a-82ad-0a766a663782" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Oct 30 19:40:07.481952 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:07.481910    6996 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5db77fbd9f-sbbjg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Oct 30 19:40:07.482248 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:07.482211    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sbbjg" podUID="81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Oct 30 19:40:08.397095 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:08.397055    6996 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5db77fbd9f-2rj4k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Oct 30 19:40:08.397521 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:08.397118    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-2rj4k" podUID="8d0ef765-f9f8-4f8a-82ad-0a766a663782" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Oct 30 19:40:08.487388 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:08.487341    6996 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5db77fbd9f-sbbjg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Oct 30 19:40:08.487627 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:08.487398    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sbbjg" podUID="81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Oct 30 19:40:09.396492 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:09.396453    6996 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5db77fbd9f-2rj4k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Oct 30 19:40:09.396750 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:09.396524    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-2rj4k" podUID="8d0ef765-f9f8-4f8a-82ad-0a766a663782" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Oct 30 19:40:09.482293 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:09.482250    6996 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5db77fbd9f-sbbjg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Oct 30 19:40:09.482864 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:09.482311    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sbbjg" podUID="81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Oct 30 19:40:11.092104 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:11.092065    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-2rj4k"
Oct 30 19:40:11.093376 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:11.093356    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-2rj4k"
Oct 30 19:40:11.094048 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:11.094025    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-2rj4k"
Oct 30 19:40:11.118457 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:11.118391    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sbbjg" podStartSLOduration=9.118369698 podStartE2EDuration="9.118369698s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:40:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:40:03.455848996 +0000 UTC m=+478.783967179" watchObservedRunningTime="2026-10-30 19:40:11.118369698 +0000 UTC m=+486.446487856"
Oct 30 19:40:11.215683 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:11.215645    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sbbjg"
Oct 30 19:40:11.216452 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:11.216433    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sbbjg"
Oct 30 19:40:11.217356 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:11.217338    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sbbjg"
Oct 30 19:40:14.110822 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:14.110789    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw"]
Oct 30 19:40:14.116284 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:14.116263    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw"
Oct 30 19:40:14.128098 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:14.128077    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-b33c04be-openshift-default-dockercfg-lk9ws\""
Oct 30 19:40:14.213061 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:14.213021    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3a80a33-44df-464f-bdf1-5f9804cb29c5-credential-socket\") pod \"e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw\" (UID: \"b3a80a33-44df-464f-bdf1-5f9804cb29c5\") " pod="openshift-ingress/e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw"
Oct 30 19:40:14.213241 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:14.213146    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b3a80a33-44df-464f-bdf1-5f9804cb29c5-istio-podinfo\") pod \"e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw\" (UID: \"b3a80a33-44df-464f-bdf1-5f9804cb29c5\") " pod="openshift-ingress/e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw"
Oct 30 19:40:14.213241 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:14.213199    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b3a80a33-44df-464f-bdf1-5f9804cb29c5-istio-data\") pod \"e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw\" (UID: \"b3a80a33-44df-464f-bdf1-5f9804cb29c5\") " pod="openshift-ingress/e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw"
Oct 30 19:40:14.213241 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:14.213228    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b3a80a33-44df-464f-bdf1-5f9804cb29c5-istio-envoy\") pod \"e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw\" (UID: \"b3a80a33-44df-464f-bdf1-5f9804cb29c5\") " pod="openshift-ingress/e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw"
Oct 30 19:40:14.213406 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:14.213287    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3a80a33-44df-464f-bdf1-5f9804cb29c5-istio-token\") pod \"e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw\" (UID: \"b3a80a33-44df-464f-bdf1-5f9804cb29c5\") " pod="openshift-ingress/e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw"
Oct 30 19:40:14.213462 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:14.213396    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3a80a33-44df-464f-bdf1-5f9804cb29c5-workload-socket\") pod \"e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw\" (UID: \"b3a80a33-44df-464f-bdf1-5f9804cb29c5\") " pod="openshift-ingress/e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw"
Oct 30 19:40:14.213543 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:14.213475    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t74qd\" (UniqueName: \"kubernetes.io/projected/b3a80a33-44df-464f-bdf1-5f9804cb29c5-kube-api-access-t74qd\") pod \"e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw\" (UID: \"b3a80a33-44df-464f-bdf1-5f9804cb29c5\") " pod="openshift-ingress/e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw"
Oct 30 19:40:14.213612 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:14.213593    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3a80a33-44df-464f-bdf1-5f9804cb29c5-workload-certs\") pod \"e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw\" (UID: \"b3a80a33-44df-464f-bdf1-5f9804cb29c5\") " pod="openshift-ingress/e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw"
Oct 30 19:40:14.213667 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:14.213635    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b3a80a33-44df-464f-bdf1-5f9804cb29c5-istiod-ca-cert\") pod \"e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw\" (UID: \"b3a80a33-44df-464f-bdf1-5f9804cb29c5\") " pod="openshift-ingress/e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw"
Oct 30 19:40:14.244621 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:14.244580    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw"]
Oct 30 19:40:14.314360 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:14.314324    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t74qd\" (UniqueName: \"kubernetes.io/projected/b3a80a33-44df-464f-bdf1-5f9804cb29c5-kube-api-access-t74qd\") pod \"e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw\" (UID: \"b3a80a33-44df-464f-bdf1-5f9804cb29c5\") " pod="openshift-ingress/e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw"
Oct 30 19:40:14.314594 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:14.314375    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3a80a33-44df-464f-bdf1-5f9804cb29c5-workload-certs\") pod \"e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw\" (UID: \"b3a80a33-44df-464f-bdf1-5f9804cb29c5\") " pod="openshift-ingress/e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw"
Oct 30 19:40:14.314594 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:14.314396    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b3a80a33-44df-464f-bdf1-5f9804cb29c5-istiod-ca-cert\") pod \"e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw\" (UID: \"b3a80a33-44df-464f-bdf1-5f9804cb29c5\") " pod="openshift-ingress/e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw"
Oct 30 19:40:14.314594 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:14.314420    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3a80a33-44df-464f-bdf1-5f9804cb29c5-credential-socket\") pod \"e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw\" (UID: \"b3a80a33-44df-464f-bdf1-5f9804cb29c5\") " pod="openshift-ingress/e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw"
Oct 30 19:40:14.314594 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:14.314457    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b3a80a33-44df-464f-bdf1-5f9804cb29c5-istio-podinfo\") pod \"e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw\" (UID: \"b3a80a33-44df-464f-bdf1-5f9804cb29c5\") " pod="openshift-ingress/e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw"
Oct 30 19:40:14.314594 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:14.314524    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b3a80a33-44df-464f-bdf1-5f9804cb29c5-istio-data\") pod \"e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw\" (UID: \"b3a80a33-44df-464f-bdf1-5f9804cb29c5\") " pod="openshift-ingress/e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw"
Oct 30 19:40:14.314594 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:14.314548    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b3a80a33-44df-464f-bdf1-5f9804cb29c5-istio-envoy\") pod \"e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw\" (UID: \"b3a80a33-44df-464f-bdf1-5f9804cb29c5\") " pod="openshift-ingress/e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw"
Oct 30 19:40:14.314594 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:14.314584    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3a80a33-44df-464f-bdf1-5f9804cb29c5-istio-token\") pod \"e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw\" (UID: \"b3a80a33-44df-464f-bdf1-5f9804cb29c5\") " pod="openshift-ingress/e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw"
Oct 30 19:40:14.314957 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:14.314619    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3a80a33-44df-464f-bdf1-5f9804cb29c5-workload-socket\") pod \"e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw\" (UID: \"b3a80a33-44df-464f-bdf1-5f9804cb29c5\") " pod="openshift-ingress/e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw"
Oct 30 19:40:14.315023 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:14.314956    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3a80a33-44df-464f-bdf1-5f9804cb29c5-workload-certs\") pod \"e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw\" (UID: \"b3a80a33-44df-464f-bdf1-5f9804cb29c5\") " pod="openshift-ingress/e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw"
Oct 30 19:40:14.315023 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:14.315011    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3a80a33-44df-464f-bdf1-5f9804cb29c5-credential-socket\") pod \"e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw\" (UID: \"b3a80a33-44df-464f-bdf1-5f9804cb29c5\") " pod="openshift-ingress/e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw"
Oct 30 19:40:14.315129 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:14.315099    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3a80a33-44df-464f-bdf1-5f9804cb29c5-workload-socket\") pod \"e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw\" (UID: \"b3a80a33-44df-464f-bdf1-5f9804cb29c5\") " pod="openshift-ingress/e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw"
Oct 30 19:40:14.315244 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:14.315215    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b3a80a33-44df-464f-bdf1-5f9804cb29c5-istio-data\") pod \"e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw\" (UID: \"b3a80a33-44df-464f-bdf1-5f9804cb29c5\") " pod="openshift-ingress/e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw"
Oct 30 19:40:14.315376 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:14.315356    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b3a80a33-44df-464f-bdf1-5f9804cb29c5-istiod-ca-cert\") pod \"e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw\" (UID: \"b3a80a33-44df-464f-bdf1-5f9804cb29c5\") " pod="openshift-ingress/e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw"
Oct 30 19:40:14.317415 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:14.317393    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b3a80a33-44df-464f-bdf1-5f9804cb29c5-istio-envoy\") pod \"e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw\" (UID: \"b3a80a33-44df-464f-bdf1-5f9804cb29c5\") " pod="openshift-ingress/e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw"
Oct 30 19:40:14.317633 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:14.317618    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b3a80a33-44df-464f-bdf1-5f9804cb29c5-istio-podinfo\") pod \"e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw\" (UID: \"b3a80a33-44df-464f-bdf1-5f9804cb29c5\") " pod="openshift-ingress/e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw"
Oct 30 19:40:14.339222 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:14.339190    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t74qd\" (UniqueName: \"kubernetes.io/projected/b3a80a33-44df-464f-bdf1-5f9804cb29c5-kube-api-access-t74qd\") pod \"e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw\" (UID: \"b3a80a33-44df-464f-bdf1-5f9804cb29c5\") " pod="openshift-ingress/e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw"
Oct 30 19:40:14.344111 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:14.344086    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3a80a33-44df-464f-bdf1-5f9804cb29c5-istio-token\") pod \"e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw\" (UID: \"b3a80a33-44df-464f-bdf1-5f9804cb29c5\") " pod="openshift-ingress/e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw"
Oct 30 19:40:14.429779 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:14.429693    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw"
Oct 30 19:40:14.617792 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:14.617757    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f"]
Oct 30 19:40:14.623244 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:14.623224    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f"
Oct 30 19:40:14.635901 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:14.635876    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-1d482894-gw-openshift-default-dockercfg-pfnpx\""
Oct 30 19:40:14.643052 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:14.642101    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f"]
Oct 30 19:40:14.719230 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:14.719147    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8529bd14-0cbc-4e29-a8e6-8d46acb50268-workload-socket\") pod \"e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f\" (UID: \"8529bd14-0cbc-4e29-a8e6-8d46acb50268\") " pod="openshift-ingress/e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f"
Oct 30 19:40:14.719230 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:14.719185    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8529bd14-0cbc-4e29-a8e6-8d46acb50268-istio-envoy\") pod \"e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f\" (UID: \"8529bd14-0cbc-4e29-a8e6-8d46acb50268\") " pod="openshift-ingress/e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f"
Oct 30 19:40:14.719415 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:14.719331    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8529bd14-0cbc-4e29-a8e6-8d46acb50268-workload-certs\") pod \"e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f\" (UID: \"8529bd14-0cbc-4e29-a8e6-8d46acb50268\") " pod="openshift-ingress/e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f"
Oct 30 19:40:14.719415 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:14.719361    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8529bd14-0cbc-4e29-a8e6-8d46acb50268-istiod-ca-cert\") pod \"e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f\" (UID: \"8529bd14-0cbc-4e29-a8e6-8d46acb50268\") " pod="openshift-ingress/e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f"
Oct 30 19:40:14.719510 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:14.719460    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2k59\" (UniqueName: \"kubernetes.io/projected/8529bd14-0cbc-4e29-a8e6-8d46acb50268-kube-api-access-s2k59\") pod \"e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f\" (UID: \"8529bd14-0cbc-4e29-a8e6-8d46acb50268\") " pod="openshift-ingress/e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f"
Oct 30 19:40:14.719510 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:14.719503    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8529bd14-0cbc-4e29-a8e6-8d46acb50268-istio-token\") pod \"e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f\" (UID: \"8529bd14-0cbc-4e29-a8e6-8d46acb50268\") " pod="openshift-ingress/e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f"
Oct 30 19:40:14.719580 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:14.719567    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8529bd14-0cbc-4e29-a8e6-8d46acb50268-credential-socket\") pod \"e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f\" (UID: \"8529bd14-0cbc-4e29-a8e6-8d46acb50268\") " pod="openshift-ingress/e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f"
Oct 30 19:40:14.719623 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:14.719606    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8529bd14-0cbc-4e29-a8e6-8d46acb50268-istio-data\") pod \"e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f\" (UID: \"8529bd14-0cbc-4e29-a8e6-8d46acb50268\") " pod="openshift-ingress/e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f"
Oct 30 19:40:14.719713 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:14.719694    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8529bd14-0cbc-4e29-a8e6-8d46acb50268-istio-podinfo\") pod \"e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f\" (UID: \"8529bd14-0cbc-4e29-a8e6-8d46acb50268\") " pod="openshift-ingress/e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f"
Oct 30 19:40:14.754843 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:14.754814    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw"]
Oct 30 19:40:14.756512 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:40:14.756466    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3a80a33_44df_464f_bdf1_5f9804cb29c5.slice/crio-4825c7c95e566c0f52a2d978ac810bbbdfcf5ec4e6b7e1db7ed8e0f1cc611df8 WatchSource:0}: Error finding container 4825c7c95e566c0f52a2d978ac810bbbdfcf5ec4e6b7e1db7ed8e0f1cc611df8: Status 404 returned error can't find the container with id 4825c7c95e566c0f52a2d978ac810bbbdfcf5ec4e6b7e1db7ed8e0f1cc611df8
Oct 30 19:40:14.758952 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:14.758920    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:40:14.759040 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:14.758996    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:40:14.759040 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:14.759026    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:40:14.820590 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:14.820561    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2k59\" (UniqueName: \"kubernetes.io/projected/8529bd14-0cbc-4e29-a8e6-8d46acb50268-kube-api-access-s2k59\") pod \"e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f\" (UID: \"8529bd14-0cbc-4e29-a8e6-8d46acb50268\") " pod="openshift-ingress/e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f"
Oct 30 19:40:14.820726 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:14.820610    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8529bd14-0cbc-4e29-a8e6-8d46acb50268-istio-token\") pod \"e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f\" (UID: \"8529bd14-0cbc-4e29-a8e6-8d46acb50268\") " pod="openshift-ingress/e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f"
Oct 30 19:40:14.820726 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:14.820643    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8529bd14-0cbc-4e29-a8e6-8d46acb50268-credential-socket\") pod \"e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f\" (UID: \"8529bd14-0cbc-4e29-a8e6-8d46acb50268\") " pod="openshift-ingress/e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f"
Oct 30 19:40:14.820726 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:14.820683    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8529bd14-0cbc-4e29-a8e6-8d46acb50268-istio-data\") pod \"e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f\" (UID: \"8529bd14-0cbc-4e29-a8e6-8d46acb50268\") " pod="openshift-ingress/e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f"
Oct 30 19:40:14.820897 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:14.820759    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8529bd14-0cbc-4e29-a8e6-8d46acb50268-istio-podinfo\") pod \"e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f\" (UID: \"8529bd14-0cbc-4e29-a8e6-8d46acb50268\") " pod="openshift-ingress/e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f"
Oct 30 19:40:14.820897 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:14.820805    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8529bd14-0cbc-4e29-a8e6-8d46acb50268-workload-socket\") pod \"e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f\" (UID: \"8529bd14-0cbc-4e29-a8e6-8d46acb50268\") " pod="openshift-ingress/e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f"
Oct 30 19:40:14.820897 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:14.820835    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8529bd14-0cbc-4e29-a8e6-8d46acb50268-istio-envoy\") pod \"e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f\" (UID: \"8529bd14-0cbc-4e29-a8e6-8d46acb50268\") " pod="openshift-ingress/e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f"
Oct 30 19:40:14.821047 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:14.820946    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8529bd14-0cbc-4e29-a8e6-8d46acb50268-workload-certs\") pod \"e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f\" (UID: \"8529bd14-0cbc-4e29-a8e6-8d46acb50268\") " pod="openshift-ingress/e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f"
Oct 30 19:40:14.821047 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:14.820977    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8529bd14-0cbc-4e29-a8e6-8d46acb50268-istiod-ca-cert\") pod \"e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f\" (UID: \"8529bd14-0cbc-4e29-a8e6-8d46acb50268\") " pod="openshift-ingress/e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f"
Oct 30 19:40:14.821261 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:14.821236    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8529bd14-0cbc-4e29-a8e6-8d46acb50268-istio-data\") pod \"e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f\" (UID: \"8529bd14-0cbc-4e29-a8e6-8d46acb50268\") " pod="openshift-ingress/e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f"
Oct 30 19:40:14.821394 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:14.821260    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8529bd14-0cbc-4e29-a8e6-8d46acb50268-credential-socket\") pod \"e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f\" (UID: \"8529bd14-0cbc-4e29-a8e6-8d46acb50268\") " pod="openshift-ingress/e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f"
Oct 30 19:40:14.821994 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:14.821714    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8529bd14-0cbc-4e29-a8e6-8d46acb50268-workload-certs\") pod \"e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f\" (UID: \"8529bd14-0cbc-4e29-a8e6-8d46acb50268\") " pod="openshift-ingress/e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f"
Oct 30 19:40:14.821994 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:14.821793    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8529bd14-0cbc-4e29-a8e6-8d46acb50268-workload-socket\") pod \"e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f\" (UID: \"8529bd14-0cbc-4e29-a8e6-8d46acb50268\") " pod="openshift-ingress/e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f"
Oct 30 19:40:14.821994 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:14.821935    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8529bd14-0cbc-4e29-a8e6-8d46acb50268-istiod-ca-cert\") pod \"e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f\" (UID: \"8529bd14-0cbc-4e29-a8e6-8d46acb50268\") " pod="openshift-ingress/e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f"
Oct 30 19:40:14.823892 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:14.823869    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8529bd14-0cbc-4e29-a8e6-8d46acb50268-istio-podinfo\") pod \"e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f\" (UID: \"8529bd14-0cbc-4e29-a8e6-8d46acb50268\") " pod="openshift-ingress/e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f"
Oct 30 19:40:14.824061 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:14.824040    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8529bd14-0cbc-4e29-a8e6-8d46acb50268-istio-envoy\") pod \"e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f\" (UID: \"8529bd14-0cbc-4e29-a8e6-8d46acb50268\") " pod="openshift-ingress/e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f"
Oct 30 19:40:14.833387 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:14.833363    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8529bd14-0cbc-4e29-a8e6-8d46acb50268-istio-token\") pod \"e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f\" (UID: \"8529bd14-0cbc-4e29-a8e6-8d46acb50268\") " pod="openshift-ingress/e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f"
Oct 30 19:40:14.845054 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:14.845028    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2k59\" (UniqueName: \"kubernetes.io/projected/8529bd14-0cbc-4e29-a8e6-8d46acb50268-kube-api-access-s2k59\") pod \"e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f\" (UID: \"8529bd14-0cbc-4e29-a8e6-8d46acb50268\") " pod="openshift-ingress/e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f"
Oct 30 19:40:14.937544 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:14.937511    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f"
Oct 30 19:40:15.224698 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:15.224673    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f"]
Oct 30 19:40:15.226762 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:40:15.226736    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8529bd14_0cbc_4e29_a8e6_8d46acb50268.slice/crio-50baf6252fa23d60e296c1783e3ca4488317f7042570869f86149ea5d960bb17 WatchSource:0}: Error finding container 50baf6252fa23d60e296c1783e3ca4488317f7042570869f86149ea5d960bb17: Status 404 returned error can't find the container with id 50baf6252fa23d60e296c1783e3ca4488317f7042570869f86149ea5d960bb17
Oct 30 19:40:15.229072 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:15.229037    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:40:15.229151 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:15.229102    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:40:15.229151 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:15.229138    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:40:15.464980 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:15.464934    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw" event={"ID":"b3a80a33-44df-464f-bdf1-5f9804cb29c5","Type":"ContainerStarted","Data":"fab12fe15a32f1bf9e890dc0309116c75de5dbeae9a82076cb670207e5afab16"}
Oct 30 19:40:15.465177 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:15.464998    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw" event={"ID":"b3a80a33-44df-464f-bdf1-5f9804cb29c5","Type":"ContainerStarted","Data":"4825c7c95e566c0f52a2d978ac810bbbdfcf5ec4e6b7e1db7ed8e0f1cc611df8"}
Oct 30 19:40:15.466664 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:15.466631    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f" event={"ID":"8529bd14-0cbc-4e29-a8e6-8d46acb50268","Type":"ContainerStarted","Data":"ab97147e44f11a3cb1cffa87c40ea670f07d3d7232a80aca20df994b0b91a2b0"}
Oct 30 19:40:15.466796 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:15.466678    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f" event={"ID":"8529bd14-0cbc-4e29-a8e6-8d46acb50268","Type":"ContainerStarted","Data":"50baf6252fa23d60e296c1783e3ca4488317f7042570869f86149ea5d960bb17"}
Oct 30 19:40:15.493124 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:15.493023    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw" podStartSLOduration=1.493007032 podStartE2EDuration="1.493007032s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:40:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:40:15.490666168 +0000 UTC m=+490.818784333" watchObservedRunningTime="2026-10-30 19:40:15.493007032 +0000 UTC m=+490.821125266"
Oct 30 19:40:15.515207 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:15.515158    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f" podStartSLOduration=1.515139454 podStartE2EDuration="1.515139454s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:40:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:40:15.513559777 +0000 UTC m=+490.841677930" watchObservedRunningTime="2026-10-30 19:40:15.515139454 +0000 UTC m=+490.843257658"
Oct 30 19:40:15.938438 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:15.938405    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f"
Oct 30 19:40:16.430879 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:16.430846    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw"
Oct 30 19:40:16.436128 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:16.436100    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw"
Oct 30 19:40:16.472301 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:16.472274    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw"
Oct 30 19:40:16.473360 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:16.473335    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw"
Oct 30 19:40:16.942969 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:16.942937    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f"
Oct 30 19:40:17.104468 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:17.104428    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-5jtlw"]
Oct 30 19:40:17.108557 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:17.108532    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-5jtlw"
Oct 30 19:40:17.134166 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:17.134130    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-5jtlw"]
Oct 30 19:40:17.247965 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:17.247878    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/abe1772a-c94e-4dd9-b023-176bab5081d3-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-5jtlw\" (UID: \"abe1772a-c94e-4dd9-b023-176bab5081d3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-5jtlw"
Oct 30 19:40:17.247965 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:17.247934    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/abe1772a-c94e-4dd9-b023-176bab5081d3-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-5jtlw\" (UID: \"abe1772a-c94e-4dd9-b023-176bab5081d3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-5jtlw"
Oct 30 19:40:17.248149 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:17.248038    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/abe1772a-c94e-4dd9-b023-176bab5081d3-workload-socket\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-5jtlw\" (UID: \"abe1772a-c94e-4dd9-b023-176bab5081d3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-5jtlw"
Oct 30 19:40:17.248149 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:17.248098    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/abe1772a-c94e-4dd9-b023-176bab5081d3-istio-data\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-5jtlw\" (UID: \"abe1772a-c94e-4dd9-b023-176bab5081d3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-5jtlw"
Oct 30 19:40:17.248214 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:17.248175    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/abe1772a-c94e-4dd9-b023-176bab5081d3-credential-socket\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-5jtlw\" (UID: \"abe1772a-c94e-4dd9-b023-176bab5081d3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-5jtlw"
Oct 30 19:40:17.248248 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:17.248229    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/abe1772a-c94e-4dd9-b023-176bab5081d3-istio-token\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-5jtlw\" (UID: \"abe1772a-c94e-4dd9-b023-176bab5081d3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-5jtlw"
Oct 30 19:40:17.248295 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:17.248278    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7z7tc\" (UniqueName: \"kubernetes.io/projected/abe1772a-c94e-4dd9-b023-176bab5081d3-kube-api-access-7z7tc\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-5jtlw\" (UID: \"abe1772a-c94e-4dd9-b023-176bab5081d3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-5jtlw"
Oct 30 19:40:17.248343 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:17.248332    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/abe1772a-c94e-4dd9-b023-176bab5081d3-workload-certs\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-5jtlw\" (UID: \"abe1772a-c94e-4dd9-b023-176bab5081d3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-5jtlw"
Oct 30 19:40:17.248378 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:17.248354    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/abe1772a-c94e-4dd9-b023-176bab5081d3-istio-envoy\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-5jtlw\" (UID: \"abe1772a-c94e-4dd9-b023-176bab5081d3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-5jtlw"
Oct 30 19:40:17.349087 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:17.349046    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/abe1772a-c94e-4dd9-b023-176bab5081d3-workload-socket\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-5jtlw\" (UID: \"abe1772a-c94e-4dd9-b023-176bab5081d3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-5jtlw"
Oct 30 19:40:17.349283 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:17.349111    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/abe1772a-c94e-4dd9-b023-176bab5081d3-istio-data\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-5jtlw\" (UID: \"abe1772a-c94e-4dd9-b023-176bab5081d3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-5jtlw"
Oct 30 19:40:17.349283 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:17.349144    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/abe1772a-c94e-4dd9-b023-176bab5081d3-credential-socket\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-5jtlw\" (UID: \"abe1772a-c94e-4dd9-b023-176bab5081d3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-5jtlw"
Oct 30 19:40:17.349283 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:17.349195    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/abe1772a-c94e-4dd9-b023-176bab5081d3-istio-token\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-5jtlw\" (UID: \"abe1772a-c94e-4dd9-b023-176bab5081d3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-5jtlw"
Oct 30 19:40:17.349513 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:17.349454    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7z7tc\" (UniqueName: \"kubernetes.io/projected/abe1772a-c94e-4dd9-b023-176bab5081d3-kube-api-access-7z7tc\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-5jtlw\" (UID: \"abe1772a-c94e-4dd9-b023-176bab5081d3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-5jtlw"
Oct 30 19:40:17.349987 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:17.349559    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/abe1772a-c94e-4dd9-b023-176bab5081d3-workload-socket\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-5jtlw\" (UID: \"abe1772a-c94e-4dd9-b023-176bab5081d3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-5jtlw"
Oct 30 19:40:17.349987 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:17.349571    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/abe1772a-c94e-4dd9-b023-176bab5081d3-workload-certs\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-5jtlw\" (UID: \"abe1772a-c94e-4dd9-b023-176bab5081d3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-5jtlw"
Oct 30 19:40:17.349987 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:17.349591    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/abe1772a-c94e-4dd9-b023-176bab5081d3-istio-data\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-5jtlw\" (UID: \"abe1772a-c94e-4dd9-b023-176bab5081d3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-5jtlw"
Oct 30 19:40:17.349987 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:17.349628    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/abe1772a-c94e-4dd9-b023-176bab5081d3-istio-envoy\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-5jtlw\" (UID: \"abe1772a-c94e-4dd9-b023-176bab5081d3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-5jtlw"
Oct 30 19:40:17.349987 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:17.349728    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/abe1772a-c94e-4dd9-b023-176bab5081d3-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-5jtlw\" (UID: \"abe1772a-c94e-4dd9-b023-176bab5081d3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-5jtlw"
Oct 30 19:40:17.349987 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:17.349762    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/abe1772a-c94e-4dd9-b023-176bab5081d3-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-5jtlw\" (UID: \"abe1772a-c94e-4dd9-b023-176bab5081d3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-5jtlw"
Oct 30 19:40:17.350509 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:17.350052    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/abe1772a-c94e-4dd9-b023-176bab5081d3-workload-certs\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-5jtlw\" (UID: \"abe1772a-c94e-4dd9-b023-176bab5081d3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-5jtlw"
Oct 30 19:40:17.350779 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:17.350754    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/abe1772a-c94e-4dd9-b023-176bab5081d3-credential-socket\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-5jtlw\" (UID: \"abe1772a-c94e-4dd9-b023-176bab5081d3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-5jtlw"
Oct 30 19:40:17.351017 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:17.350999    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/abe1772a-c94e-4dd9-b023-176bab5081d3-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-5jtlw\" (UID: \"abe1772a-c94e-4dd9-b023-176bab5081d3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-5jtlw"
Oct 30 19:40:17.352999 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:17.352974    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/abe1772a-c94e-4dd9-b023-176bab5081d3-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-5jtlw\" (UID: \"abe1772a-c94e-4dd9-b023-176bab5081d3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-5jtlw"
Oct 30 19:40:17.353105 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:17.353049    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/abe1772a-c94e-4dd9-b023-176bab5081d3-istio-envoy\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-5jtlw\" (UID: \"abe1772a-c94e-4dd9-b023-176bab5081d3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-5jtlw"
Oct 30 19:40:17.362655 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:17.362630    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/abe1772a-c94e-4dd9-b023-176bab5081d3-istio-token\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-5jtlw\" (UID: \"abe1772a-c94e-4dd9-b023-176bab5081d3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-5jtlw"
Oct 30 19:40:17.362829 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:17.362812    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7z7tc\" (UniqueName: \"kubernetes.io/projected/abe1772a-c94e-4dd9-b023-176bab5081d3-kube-api-access-7z7tc\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-5jtlw\" (UID: \"abe1772a-c94e-4dd9-b023-176bab5081d3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-5jtlw"
Oct 30 19:40:17.426733 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:17.426688    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-5jtlw"
Oct 30 19:40:17.480317 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:17.480231    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f"
Oct 30 19:40:17.482090 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:17.482007    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f"
Oct 30 19:40:17.717340 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:17.717281    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-5jtlw"]
Oct 30 19:40:17.720455 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:40:17.720430    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabe1772a_c94e_4dd9_b023_176bab5081d3.slice/crio-b07634d1a46807bdfd512009ed98fb60ef229b99b3c45e41f431633566d11d49 WatchSource:0}: Error finding container b07634d1a46807bdfd512009ed98fb60ef229b99b3c45e41f431633566d11d49: Status 404 returned error can't find the container with id b07634d1a46807bdfd512009ed98fb60ef229b99b3c45e41f431633566d11d49
Oct 30 19:40:17.722910 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:17.722874    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:40:17.723006 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:17.722962    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:40:17.723006 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:17.722999    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:40:18.038927 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:18.038832    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-1d482894-6fc5774b99-msqpt"]
Oct 30 19:40:18.042653 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:18.042630    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-1d482894-6fc5774b99-msqpt"
Oct 30 19:40:18.052421 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:18.052398    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-1d482894-dockercfg-4z4w2\""
Oct 30 19:40:18.052421 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:18.052398    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-1d482894\""
Oct 30 19:40:18.066795 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:18.066749    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-1d482894-6fc5774b99-msqpt"]
Oct 30 19:40:18.122436 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:18.122394    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-1d482894-56c95f9f44-m528s"]
Oct 30 19:40:18.126694 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:18.126666    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-1d482894-56c95f9f44-m528s"
Oct 30 19:40:18.145395 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:18.145355    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-1d482894-56c95f9f44-m528s"]
Oct 30 19:40:18.163905 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:18.163862    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15e26590-2ab2-4b01-aa34-fbedcc89e919-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-1d482894-6fc5774b99-msqpt\" (UID: \"15e26590-2ab2-4b01-aa34-fbedcc89e919\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-1d482894-6fc5774b99-msqpt"
Oct 30 19:40:18.164337 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:18.164305    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qsbqs\" (UniqueName: \"kubernetes.io/projected/15e26590-2ab2-4b01-aa34-fbedcc89e919-kube-api-access-qsbqs\") pod \"payload-pre-processing-e2e-migrate-1d482894-6fc5774b99-msqpt\" (UID: \"15e26590-2ab2-4b01-aa34-fbedcc89e919\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-1d482894-6fc5774b99-msqpt"
Oct 30 19:40:18.201888 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:18.201850    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-1d482894-bbfc8cdb9-dpnsm"]
Oct 30 19:40:18.206215 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:18.206182    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1d482894-bbfc8cdb9-dpnsm"
Oct 30 19:40:18.211422 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:18.211392    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-1d482894-serving-cert\""
Oct 30 19:40:18.213402 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:18.213376    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-1d482894-bbfc8cdb9-dpnsm"]
Oct 30 19:40:18.265155 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:18.265099    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15e26590-2ab2-4b01-aa34-fbedcc89e919-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-1d482894-6fc5774b99-msqpt\" (UID: \"15e26590-2ab2-4b01-aa34-fbedcc89e919\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-1d482894-6fc5774b99-msqpt"
Oct 30 19:40:18.265342 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:18.265191    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26mq2\" (UniqueName: \"kubernetes.io/projected/36ff5561-cb31-4a4c-9702-3749feae0ffc-kube-api-access-26mq2\") pod \"payload-processing-e2e-migrate-1d482894-56c95f9f44-m528s\" (UID: \"36ff5561-cb31-4a4c-9702-3749feae0ffc\") " pod="openshift-ingress/payload-processing-e2e-migrate-1d482894-56c95f9f44-m528s"
Oct 30 19:40:18.265342 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:18.265236    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36ff5561-cb31-4a4c-9702-3749feae0ffc-plugins-config-volume\") pod \"payload-processing-e2e-migrate-1d482894-56c95f9f44-m528s\" (UID: \"36ff5561-cb31-4a4c-9702-3749feae0ffc\") " pod="openshift-ingress/payload-processing-e2e-migrate-1d482894-56c95f9f44-m528s"
Oct 30 19:40:18.265342 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:18.265280    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsbqs\" (UniqueName: \"kubernetes.io/projected/15e26590-2ab2-4b01-aa34-fbedcc89e919-kube-api-access-qsbqs\") pod \"payload-pre-processing-e2e-migrate-1d482894-6fc5774b99-msqpt\" (UID: \"15e26590-2ab2-4b01-aa34-fbedcc89e919\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-1d482894-6fc5774b99-msqpt"
Oct 30 19:40:18.266443 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:18.266414    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15e26590-2ab2-4b01-aa34-fbedcc89e919-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-1d482894-6fc5774b99-msqpt\" (UID: \"15e26590-2ab2-4b01-aa34-fbedcc89e919\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-1d482894-6fc5774b99-msqpt"
Oct 30 19:40:18.278097 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:18.278070    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qsbqs\" (UniqueName: \"kubernetes.io/projected/15e26590-2ab2-4b01-aa34-fbedcc89e919-kube-api-access-qsbqs\") pod \"payload-pre-processing-e2e-migrate-1d482894-6fc5774b99-msqpt\" (UID: \"15e26590-2ab2-4b01-aa34-fbedcc89e919\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-1d482894-6fc5774b99-msqpt"
Oct 30 19:40:18.361519 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:18.360704    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-1d482894-6fc5774b99-msqpt"
Oct 30 19:40:18.366438 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:18.366408    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ba269cd-36fe-4a68-ae4e-1c7e7c7076a1-metrics-certs\") pod \"maas-api-e2e-migrate-1d482894-bbfc8cdb9-dpnsm\" (UID: \"5ba269cd-36fe-4a68-ae4e-1c7e7c7076a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1d482894-bbfc8cdb9-dpnsm"
Oct 30 19:40:18.366585 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:18.366557    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lb2rc\" (UniqueName: \"kubernetes.io/projected/5ba269cd-36fe-4a68-ae4e-1c7e7c7076a1-kube-api-access-lb2rc\") pod \"maas-api-e2e-migrate-1d482894-bbfc8cdb9-dpnsm\" (UID: \"5ba269cd-36fe-4a68-ae4e-1c7e7c7076a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1d482894-bbfc8cdb9-dpnsm"
Oct 30 19:40:18.366645 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:18.366600    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ba269cd-36fe-4a68-ae4e-1c7e7c7076a1-openshift-service-ca\") pod \"maas-api-e2e-migrate-1d482894-bbfc8cdb9-dpnsm\" (UID: \"5ba269cd-36fe-4a68-ae4e-1c7e7c7076a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1d482894-bbfc8cdb9-dpnsm"
Oct 30 19:40:18.366702 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:18.366683    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5ba269cd-36fe-4a68-ae4e-1c7e7c7076a1-maas-api-tls\") pod \"maas-api-e2e-migrate-1d482894-bbfc8cdb9-dpnsm\" (UID: \"5ba269cd-36fe-4a68-ae4e-1c7e7c7076a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1d482894-bbfc8cdb9-dpnsm"
Oct 30 19:40:18.367831 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:18.366722    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26mq2\" (UniqueName: \"kubernetes.io/projected/36ff5561-cb31-4a4c-9702-3749feae0ffc-kube-api-access-26mq2\") pod \"payload-processing-e2e-migrate-1d482894-56c95f9f44-m528s\" (UID: \"36ff5561-cb31-4a4c-9702-3749feae0ffc\") " pod="openshift-ingress/payload-processing-e2e-migrate-1d482894-56c95f9f44-m528s"
Oct 30 19:40:18.367831 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:18.366784    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36ff5561-cb31-4a4c-9702-3749feae0ffc-plugins-config-volume\") pod \"payload-processing-e2e-migrate-1d482894-56c95f9f44-m528s\" (UID: \"36ff5561-cb31-4a4c-9702-3749feae0ffc\") " pod="openshift-ingress/payload-processing-e2e-migrate-1d482894-56c95f9f44-m528s"
Oct 30 19:40:18.367831 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:18.367766    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36ff5561-cb31-4a4c-9702-3749feae0ffc-plugins-config-volume\") pod \"payload-processing-e2e-migrate-1d482894-56c95f9f44-m528s\" (UID: \"36ff5561-cb31-4a4c-9702-3749feae0ffc\") " pod="openshift-ingress/payload-processing-e2e-migrate-1d482894-56c95f9f44-m528s"
Oct 30 19:40:18.381444 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:18.381415    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-26mq2\" (UniqueName: \"kubernetes.io/projected/36ff5561-cb31-4a4c-9702-3749feae0ffc-kube-api-access-26mq2\") pod \"payload-processing-e2e-migrate-1d482894-56c95f9f44-m528s\" (UID: \"36ff5561-cb31-4a4c-9702-3749feae0ffc\") " pod="openshift-ingress/payload-processing-e2e-migrate-1d482894-56c95f9f44-m528s"
Oct 30 19:40:18.475503 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:18.451915    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-1d482894-56c95f9f44-m528s"
Oct 30 19:40:18.481541 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:18.476449    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ba269cd-36fe-4a68-ae4e-1c7e7c7076a1-metrics-certs\") pod \"maas-api-e2e-migrate-1d482894-bbfc8cdb9-dpnsm\" (UID: \"5ba269cd-36fe-4a68-ae4e-1c7e7c7076a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1d482894-bbfc8cdb9-dpnsm"
Oct 30 19:40:18.481541 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:18.476539    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lb2rc\" (UniqueName: \"kubernetes.io/projected/5ba269cd-36fe-4a68-ae4e-1c7e7c7076a1-kube-api-access-lb2rc\") pod \"maas-api-e2e-migrate-1d482894-bbfc8cdb9-dpnsm\" (UID: \"5ba269cd-36fe-4a68-ae4e-1c7e7c7076a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1d482894-bbfc8cdb9-dpnsm"
Oct 30 19:40:18.481541 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:18.476576    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ba269cd-36fe-4a68-ae4e-1c7e7c7076a1-openshift-service-ca\") pod \"maas-api-e2e-migrate-1d482894-bbfc8cdb9-dpnsm\" (UID: \"5ba269cd-36fe-4a68-ae4e-1c7e7c7076a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1d482894-bbfc8cdb9-dpnsm"
Oct 30 19:40:18.481541 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:18.476653    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5ba269cd-36fe-4a68-ae4e-1c7e7c7076a1-maas-api-tls\") pod \"maas-api-e2e-migrate-1d482894-bbfc8cdb9-dpnsm\" (UID: \"5ba269cd-36fe-4a68-ae4e-1c7e7c7076a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1d482894-bbfc8cdb9-dpnsm"
Oct 30 19:40:18.484066 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:18.484030    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5ba269cd-36fe-4a68-ae4e-1c7e7c7076a1-maas-api-tls\") pod \"maas-api-e2e-migrate-1d482894-bbfc8cdb9-dpnsm\" (UID: \"5ba269cd-36fe-4a68-ae4e-1c7e7c7076a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1d482894-bbfc8cdb9-dpnsm"
Oct 30 19:40:18.484552 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:18.484469    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ba269cd-36fe-4a68-ae4e-1c7e7c7076a1-openshift-service-ca\") pod \"maas-api-e2e-migrate-1d482894-bbfc8cdb9-dpnsm\" (UID: \"5ba269cd-36fe-4a68-ae4e-1c7e7c7076a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1d482894-bbfc8cdb9-dpnsm"
Oct 30 19:40:18.496304 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:18.490076    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ba269cd-36fe-4a68-ae4e-1c7e7c7076a1-metrics-certs\") pod \"maas-api-e2e-migrate-1d482894-bbfc8cdb9-dpnsm\" (UID: \"5ba269cd-36fe-4a68-ae4e-1c7e7c7076a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1d482894-bbfc8cdb9-dpnsm"
Oct 30 19:40:18.500672 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:18.499543    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lb2rc\" (UniqueName: \"kubernetes.io/projected/5ba269cd-36fe-4a68-ae4e-1c7e7c7076a1-kube-api-access-lb2rc\") pod \"maas-api-e2e-migrate-1d482894-bbfc8cdb9-dpnsm\" (UID: \"5ba269cd-36fe-4a68-ae4e-1c7e7c7076a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1d482894-bbfc8cdb9-dpnsm"
Oct 30 19:40:18.505277 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:18.505250    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-5jtlw" event={"ID":"abe1772a-c94e-4dd9-b023-176bab5081d3","Type":"ContainerStarted","Data":"8e7ebb1ebab993527d3113ca0c69ae01fa3bd47da507481ea9012243fdc43dc8"}
Oct 30 19:40:18.505359 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:18.505307    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-5jtlw" event={"ID":"abe1772a-c94e-4dd9-b023-176bab5081d3","Type":"ContainerStarted","Data":"b07634d1a46807bdfd512009ed98fb60ef229b99b3c45e41f431633566d11d49"}
Oct 30 19:40:18.522173 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:18.522138    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1d482894-bbfc8cdb9-dpnsm"
Oct 30 19:40:18.531518 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:18.531097    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-5jtlw" podStartSLOduration=1.531082126 podStartE2EDuration="1.531082126s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:40:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:40:18.529864302 +0000 UTC m=+493.857982497" watchObservedRunningTime="2026-10-30 19:40:18.531082126 +0000 UTC m=+493.859200280"
Oct 30 19:40:18.669651 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:18.669621    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-1d482894-6fc5774b99-msqpt"]
Oct 30 19:40:18.674940 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:40:18.674901    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15e26590_2ab2_4b01_aa34_fbedcc89e919.slice/crio-c81ddb1dcbd708e7fbc1f6fa41c0e6972091c625dacef5b9e9566935d9f3730a WatchSource:0}: Error finding container c81ddb1dcbd708e7fbc1f6fa41c0e6972091c625dacef5b9e9566935d9f3730a: Status 404 returned error can't find the container with id c81ddb1dcbd708e7fbc1f6fa41c0e6972091c625dacef5b9e9566935d9f3730a
Oct 30 19:40:18.797096 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:18.797051    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-1d482894-56c95f9f44-m528s"]
Oct 30 19:40:18.799756 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:40:18.799727    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36ff5561_cb31_4a4c_9702_3749feae0ffc.slice/crio-bf615a7b6e740d3f9919f9d2a6522374947bac72cda2ab8e225903ac2f15fa83 WatchSource:0}: Error finding container bf615a7b6e740d3f9919f9d2a6522374947bac72cda2ab8e225903ac2f15fa83: Status 404 returned error can't find the container with id bf615a7b6e740d3f9919f9d2a6522374947bac72cda2ab8e225903ac2f15fa83
Oct 30 19:40:18.871681 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:18.871650    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-1d482894-bbfc8cdb9-dpnsm"]
Oct 30 19:40:18.875620 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:40:18.874536    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ba269cd_36fe_4a68_ae4e_1c7e7c7076a1.slice/crio-d37b5a23d958211710acf9197add407e7af10b5895d520eb9d7c8501372e9d03 WatchSource:0}: Error finding container d37b5a23d958211710acf9197add407e7af10b5895d520eb9d7c8501372e9d03: Status 404 returned error can't find the container with id d37b5a23d958211710acf9197add407e7af10b5895d520eb9d7c8501372e9d03
Oct 30 19:40:19.427869 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:19.427838    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-5jtlw"
Oct 30 19:40:19.429684 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:19.429662    6996 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5db77fbd9f-5jtlw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Oct 30 19:40:19.429791 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:19.429711    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-5jtlw" podUID="abe1772a-c94e-4dd9-b023-176bab5081d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Oct 30 19:40:19.518823 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:19.517944    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-1d482894-56c95f9f44-m528s" event={"ID":"36ff5561-cb31-4a4c-9702-3749feae0ffc","Type":"ContainerStarted","Data":"28e7d89462f22570ead08a3d00249390d31bc2966bc79c0433b9141ce6845e04"}
Oct 30 19:40:19.519335 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:19.519292    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-1d482894-56c95f9f44-m528s"
Oct 30 19:40:19.519417 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:19.519342    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-1d482894-56c95f9f44-m528s" event={"ID":"36ff5561-cb31-4a4c-9702-3749feae0ffc","Type":"ContainerStarted","Data":"bf615a7b6e740d3f9919f9d2a6522374947bac72cda2ab8e225903ac2f15fa83"}
Oct 30 19:40:19.521549 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:19.521519    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1d482894-bbfc8cdb9-dpnsm" event={"ID":"5ba269cd-36fe-4a68-ae4e-1c7e7c7076a1","Type":"ContainerStarted","Data":"3ae72d15bd2446bdbe0f60c5a5340f18abb5f3a844179f102a1059002f7695d4"}
Oct 30 19:40:19.521701 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:19.521571    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1d482894-bbfc8cdb9-dpnsm" event={"ID":"5ba269cd-36fe-4a68-ae4e-1c7e7c7076a1","Type":"ContainerStarted","Data":"d37b5a23d958211710acf9197add407e7af10b5895d520eb9d7c8501372e9d03"}
Oct 30 19:40:19.521701 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:19.521594    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1d482894-bbfc8cdb9-dpnsm"
Oct 30 19:40:19.524069 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:19.524036    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-1d482894-6fc5774b99-msqpt" event={"ID":"15e26590-2ab2-4b01-aa34-fbedcc89e919","Type":"ContainerStarted","Data":"8dcb960a751b2c68dace04750af1a88ee78da715cfdef7593d9b1debe527e235"}
Oct 30 19:40:19.524204 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:19.524113    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-1d482894-6fc5774b99-msqpt" event={"ID":"15e26590-2ab2-4b01-aa34-fbedcc89e919","Type":"ContainerStarted","Data":"c81ddb1dcbd708e7fbc1f6fa41c0e6972091c625dacef5b9e9566935d9f3730a"}
Oct 30 19:40:19.524204 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:19.524184    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-1d482894-6fc5774b99-msqpt"
Oct 30 19:40:19.592800 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:19.592725    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1d482894-bbfc8cdb9-dpnsm" podStartSLOduration=1.072116848 podStartE2EDuration="1.592706323s" totalImagesPullingTime="520.589475ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:40:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:40:19.590023121 +0000 UTC m=+494.918141277" watchObservedRunningTime="2026-10-30 19:40:19.592706323 +0000 UTC m=+494.920824480"
Oct 30 19:40:19.594040 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:19.593989    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-1d482894-56c95f9f44-m528s" podStartSLOduration=1.593976346 podStartE2EDuration="1.593976346s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:40:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:40:19.568846594 +0000 UTC m=+494.896964749" watchObservedRunningTime="2026-10-30 19:40:19.593976346 +0000 UTC m=+494.922094502"
Oct 30 19:40:19.609035 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:19.608989    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-1d482894-6fc5774b99-msqpt" podStartSLOduration=1.608975815 podStartE2EDuration="1.608975815s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:40:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:40:19.60755986 +0000 UTC m=+494.935678013" watchObservedRunningTime="2026-10-30 19:40:19.608975815 +0000 UTC m=+494.937094029"
Oct 30 19:40:20.428002 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:20.427965    6996 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5db77fbd9f-5jtlw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Oct 30 19:40:20.428163 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:20.428020    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-5jtlw" podUID="abe1772a-c94e-4dd9-b023-176bab5081d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Oct 30 19:40:21.164395 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:21.164359    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 19:40:21.427657 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:21.427578    6996 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5db77fbd9f-5jtlw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Oct 30 19:40:21.427657 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:21.427633    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-5jtlw" podUID="abe1772a-c94e-4dd9-b023-176bab5081d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Oct 30 19:40:21.514816 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:21.514783    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-3a55af38-564c4bcc96-cdl5f"]
Oct 30 19:40:21.521307 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:21.521283    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3a55af38-564c4bcc96-cdl5f"
Oct 30 19:40:21.529205 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:21.529176    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-3a55af38-serving-cert\""
Oct 30 19:40:21.565711 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:21.565676    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-3a55af38-564c4bcc96-cdl5f"]
Oct 30 19:40:21.625840 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:21.625682    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d-metrics-certs\") pod \"maas-api-e2e-ait-3a55af38-564c4bcc96-cdl5f\" (UID: \"8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3a55af38-564c4bcc96-cdl5f"
Oct 30 19:40:21.626309 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:21.626216    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d-openshift-service-ca\") pod \"maas-api-e2e-ait-3a55af38-564c4bcc96-cdl5f\" (UID: \"8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3a55af38-564c4bcc96-cdl5f"
Oct 30 19:40:21.626842 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:21.626816    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdgbd\" (UniqueName: \"kubernetes.io/projected/8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d-kube-api-access-qdgbd\") pod \"maas-api-e2e-ait-3a55af38-564c4bcc96-cdl5f\" (UID: \"8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3a55af38-564c4bcc96-cdl5f"
Oct 30 19:40:21.626951 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:21.626888    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d-maas-api-tls\") pod \"maas-api-e2e-ait-3a55af38-564c4bcc96-cdl5f\" (UID: \"8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3a55af38-564c4bcc96-cdl5f"
Oct 30 19:40:21.727666 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:21.727575    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d-openshift-service-ca\") pod \"maas-api-e2e-ait-3a55af38-564c4bcc96-cdl5f\" (UID: \"8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3a55af38-564c4bcc96-cdl5f"
Oct 30 19:40:21.727836 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:21.727681    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdgbd\" (UniqueName: \"kubernetes.io/projected/8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d-kube-api-access-qdgbd\") pod \"maas-api-e2e-ait-3a55af38-564c4bcc96-cdl5f\" (UID: \"8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3a55af38-564c4bcc96-cdl5f"
Oct 30 19:40:21.727836 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:21.727724    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d-maas-api-tls\") pod \"maas-api-e2e-ait-3a55af38-564c4bcc96-cdl5f\" (UID: \"8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3a55af38-564c4bcc96-cdl5f"
Oct 30 19:40:21.727836 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:21.727753    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d-metrics-certs\") pod \"maas-api-e2e-ait-3a55af38-564c4bcc96-cdl5f\" (UID: \"8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3a55af38-564c4bcc96-cdl5f"
Oct 30 19:40:21.727970 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:40:21.727887    6996 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-3a55af38-serving-cert: secret "maas-api-e2e-ait-3a55af38-serving-cert" not found
Oct 30 19:40:21.727970 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:40:21.727962    6996 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d-maas-api-tls podName:8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d nodeName:}" failed. No retries permitted until 2026-10-30 19:40:22.227945139 +0000 UTC m=+497.556063283 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d-maas-api-tls") pod "maas-api-e2e-ait-3a55af38-564c4bcc96-cdl5f" (UID: "8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d") : secret "maas-api-e2e-ait-3a55af38-serving-cert" not found
Oct 30 19:40:21.728560 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:21.728533    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d-openshift-service-ca\") pod \"maas-api-e2e-ait-3a55af38-564c4bcc96-cdl5f\" (UID: \"8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3a55af38-564c4bcc96-cdl5f"
Oct 30 19:40:21.730187 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:21.730165    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d-metrics-certs\") pod \"maas-api-e2e-ait-3a55af38-564c4bcc96-cdl5f\" (UID: \"8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3a55af38-564c4bcc96-cdl5f"
Oct 30 19:40:21.767340 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:21.767302    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdgbd\" (UniqueName: \"kubernetes.io/projected/8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d-kube-api-access-qdgbd\") pod \"maas-api-e2e-ait-3a55af38-564c4bcc96-cdl5f\" (UID: \"8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3a55af38-564c4bcc96-cdl5f"
Oct 30 19:40:22.234055 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:22.234013    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d-maas-api-tls\") pod \"maas-api-e2e-ait-3a55af38-564c4bcc96-cdl5f\" (UID: \"8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3a55af38-564c4bcc96-cdl5f"
Oct 30 19:40:22.265504 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:22.265076    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d-maas-api-tls\") pod \"maas-api-e2e-ait-3a55af38-564c4bcc96-cdl5f\" (UID: \"8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3a55af38-564c4bcc96-cdl5f"
Oct 30 19:40:22.427636 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:22.427598    6996 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5db77fbd9f-5jtlw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Oct 30 19:40:22.427804 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:22.427656    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-5jtlw" podUID="abe1772a-c94e-4dd9-b023-176bab5081d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Oct 30 19:40:22.434922 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:22.434897    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3a55af38-564c4bcc96-cdl5f"
Oct 30 19:40:22.891571 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:22.891458    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-3a55af38-564c4bcc96-cdl5f"]
Oct 30 19:40:23.428592 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:23.428570    6996 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5db77fbd9f-5jtlw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Oct 30 19:40:23.428880 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:23.428618    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-5jtlw" podUID="abe1772a-c94e-4dd9-b023-176bab5081d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Oct 30 19:40:23.715176 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:23.715090    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3a55af38-564c4bcc96-cdl5f" event={"ID":"8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d","Type":"ContainerStarted","Data":"13a5203dbeb03bdf2a6df2e17224bc563b45b7fc6be6aa7f1ce1fcb532bdf19b"}
Oct 30 19:40:23.715176 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:23.715153    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3a55af38-564c4bcc96-cdl5f" event={"ID":"8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d","Type":"ContainerStarted","Data":"0d0341b9a53aba7e41f02e95d112299368f7b56921f4808ba3a74a0a9c90b41f"}
Oct 30 19:40:23.715366 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:23.715183    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3a55af38-564c4bcc96-cdl5f"
Oct 30 19:40:23.735958 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:23.735916    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3a55af38-564c4bcc96-cdl5f" podStartSLOduration=2.196814516 podStartE2EDuration="2.735904209s" totalImagesPullingTime="539.089693ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:40:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:40:23.735680207 +0000 UTC m=+499.063798360" watchObservedRunningTime="2026-10-30 19:40:23.735904209 +0000 UTC m=+499.064022362"
Oct 30 19:40:24.427130 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:24.427092    6996 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5db77fbd9f-5jtlw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Oct 30 19:40:24.427331 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:24.427170    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-5jtlw" podUID="abe1772a-c94e-4dd9-b023-176bab5081d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Oct 30 19:40:25.427695 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:25.427652    6996 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5db77fbd9f-5jtlw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Oct 30 19:40:25.428169 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:25.427731    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-5jtlw" podUID="abe1772a-c94e-4dd9-b023-176bab5081d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Oct 30 19:40:25.571504 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:25.570671    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1d482894-bbfc8cdb9-dpnsm"
Oct 30 19:40:27.029314 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:27.029277    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-5jtlw"
Oct 30 19:40:27.030172 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:27.030149    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-5jtlw"
Oct 30 19:40:27.031329 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:27.031305    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-5jtlw"
Oct 30 19:40:27.226253 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:27.226225    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-d05eeacd-6bb9664cb7-lmrkw"]
Oct 30 19:40:27.235059 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:27.235027    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-d05eeacd-6bb9664cb7-lmrkw"
Oct 30 19:40:27.260512 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:27.258345    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-d05eeacd\""
Oct 30 19:40:27.280654 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:27.280567    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-d05eeacd-6bb9664cb7-lmrkw"]
Oct 30 19:40:27.300579 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:27.300548    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-d05eeacd-596f4f9cb6-tkmg2"]
Oct 30 19:40:27.313404 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:27.313358    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-d05eeacd-596f4f9cb6-tkmg2"]
Oct 30 19:40:27.313570 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:27.313541    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-d05eeacd-596f4f9cb6-tkmg2"
Oct 30 19:40:27.319197 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:27.319167    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-d05eeacd-dockercfg-62mlc\""
Oct 30 19:40:27.416607 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:27.416559    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f0dbb94-ef0a-4531-9b45-2bb2dff8d130-plugins-config-volume\") pod \"payload-processing-e2e-derive-d05eeacd-596f4f9cb6-tkmg2\" (UID: \"0f0dbb94-ef0a-4531-9b45-2bb2dff8d130\") " pod="openshift-ingress/payload-processing-e2e-derive-d05eeacd-596f4f9cb6-tkmg2"
Oct 30 19:40:27.416811 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:27.416655    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/468d7149-0982-44ce-a793-4d3e4cf04719-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-d05eeacd-6bb9664cb7-lmrkw\" (UID: \"468d7149-0982-44ce-a793-4d3e4cf04719\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-d05eeacd-6bb9664cb7-lmrkw"
Oct 30 19:40:27.416811 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:27.416713    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvzp5\" (UniqueName: \"kubernetes.io/projected/468d7149-0982-44ce-a793-4d3e4cf04719-kube-api-access-zvzp5\") pod \"payload-pre-processing-e2e-derive-d05eeacd-6bb9664cb7-lmrkw\" (UID: \"468d7149-0982-44ce-a793-4d3e4cf04719\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-d05eeacd-6bb9664cb7-lmrkw"
Oct 30 19:40:27.416937 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:27.416845    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5lt2\" (UniqueName: \"kubernetes.io/projected/0f0dbb94-ef0a-4531-9b45-2bb2dff8d130-kube-api-access-d5lt2\") pod \"payload-processing-e2e-derive-d05eeacd-596f4f9cb6-tkmg2\" (UID: \"0f0dbb94-ef0a-4531-9b45-2bb2dff8d130\") " pod="openshift-ingress/payload-processing-e2e-derive-d05eeacd-596f4f9cb6-tkmg2"
Oct 30 19:40:27.518470 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:27.518433    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvzp5\" (UniqueName: \"kubernetes.io/projected/468d7149-0982-44ce-a793-4d3e4cf04719-kube-api-access-zvzp5\") pod \"payload-pre-processing-e2e-derive-d05eeacd-6bb9664cb7-lmrkw\" (UID: \"468d7149-0982-44ce-a793-4d3e4cf04719\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-d05eeacd-6bb9664cb7-lmrkw"
Oct 30 19:40:27.518681 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:27.518576    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5lt2\" (UniqueName: \"kubernetes.io/projected/0f0dbb94-ef0a-4531-9b45-2bb2dff8d130-kube-api-access-d5lt2\") pod \"payload-processing-e2e-derive-d05eeacd-596f4f9cb6-tkmg2\" (UID: \"0f0dbb94-ef0a-4531-9b45-2bb2dff8d130\") " pod="openshift-ingress/payload-processing-e2e-derive-d05eeacd-596f4f9cb6-tkmg2"
Oct 30 19:40:27.518681 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:27.518650    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f0dbb94-ef0a-4531-9b45-2bb2dff8d130-plugins-config-volume\") pod \"payload-processing-e2e-derive-d05eeacd-596f4f9cb6-tkmg2\" (UID: \"0f0dbb94-ef0a-4531-9b45-2bb2dff8d130\") " pod="openshift-ingress/payload-processing-e2e-derive-d05eeacd-596f4f9cb6-tkmg2"
Oct 30 19:40:27.518766 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:27.518711    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/468d7149-0982-44ce-a793-4d3e4cf04719-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-d05eeacd-6bb9664cb7-lmrkw\" (UID: \"468d7149-0982-44ce-a793-4d3e4cf04719\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-d05eeacd-6bb9664cb7-lmrkw"
Oct 30 19:40:27.519403 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:27.519384    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/468d7149-0982-44ce-a793-4d3e4cf04719-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-d05eeacd-6bb9664cb7-lmrkw\" (UID: \"468d7149-0982-44ce-a793-4d3e4cf04719\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-d05eeacd-6bb9664cb7-lmrkw"
Oct 30 19:40:27.519447 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:27.519386    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f0dbb94-ef0a-4531-9b45-2bb2dff8d130-plugins-config-volume\") pod \"payload-processing-e2e-derive-d05eeacd-596f4f9cb6-tkmg2\" (UID: \"0f0dbb94-ef0a-4531-9b45-2bb2dff8d130\") " pod="openshift-ingress/payload-processing-e2e-derive-d05eeacd-596f4f9cb6-tkmg2"
Oct 30 19:40:27.531416 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:27.531361    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvzp5\" (UniqueName: \"kubernetes.io/projected/468d7149-0982-44ce-a793-4d3e4cf04719-kube-api-access-zvzp5\") pod \"payload-pre-processing-e2e-derive-d05eeacd-6bb9664cb7-lmrkw\" (UID: \"468d7149-0982-44ce-a793-4d3e4cf04719\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-d05eeacd-6bb9664cb7-lmrkw"
Oct 30 19:40:27.531546 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:27.531522    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5lt2\" (UniqueName: \"kubernetes.io/projected/0f0dbb94-ef0a-4531-9b45-2bb2dff8d130-kube-api-access-d5lt2\") pod \"payload-processing-e2e-derive-d05eeacd-596f4f9cb6-tkmg2\" (UID: \"0f0dbb94-ef0a-4531-9b45-2bb2dff8d130\") " pod="openshift-ingress/payload-processing-e2e-derive-d05eeacd-596f4f9cb6-tkmg2"
Oct 30 19:40:27.615278 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:27.615242    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-d05eeacd-6bb9664cb7-lmrkw"
Oct 30 19:40:27.630393 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:27.630368    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-d05eeacd-596f4f9cb6-tkmg2"
Oct 30 19:40:27.962505 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:27.962012    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-d05eeacd-6bb9664cb7-lmrkw"]
Oct 30 19:40:27.967361 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:40:27.967323    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod468d7149_0982_44ce_a793_4d3e4cf04719.slice/crio-a0a3426c3c8782a98906ff0aed334091e21102b06c45e3302a7d4fb2f03f00fc WatchSource:0}: Error finding container a0a3426c3c8782a98906ff0aed334091e21102b06c45e3302a7d4fb2f03f00fc: Status 404 returned error can't find the container with id a0a3426c3c8782a98906ff0aed334091e21102b06c45e3302a7d4fb2f03f00fc
Oct 30 19:40:28.013527 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:28.013469    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-d05eeacd-596f4f9cb6-tkmg2"]
Oct 30 19:40:28.014274 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:40:28.014246    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f0dbb94_ef0a_4531_9b45_2bb2dff8d130.slice/crio-bce8a157eff3bf25e7cf4478275bb6a46706fb8d2b02eb36ff4eec8aec7e9211 WatchSource:0}: Error finding container bce8a157eff3bf25e7cf4478275bb6a46706fb8d2b02eb36ff4eec8aec7e9211: Status 404 returned error can't find the container with id bce8a157eff3bf25e7cf4478275bb6a46706fb8d2b02eb36ff4eec8aec7e9211
Oct 30 19:40:28.781843 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:28.781801    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-d05eeacd-6bb9664cb7-lmrkw" event={"ID":"468d7149-0982-44ce-a793-4d3e4cf04719","Type":"ContainerStarted","Data":"f0a333f03e12c50bb413b98cf43a9f579db5873f214dc20e273933b7ff5dbe08"}
Oct 30 19:40:28.782286 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:28.781861    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-d05eeacd-6bb9664cb7-lmrkw" event={"ID":"468d7149-0982-44ce-a793-4d3e4cf04719","Type":"ContainerStarted","Data":"a0a3426c3c8782a98906ff0aed334091e21102b06c45e3302a7d4fb2f03f00fc"}
Oct 30 19:40:28.782286 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:28.781888    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-d05eeacd-6bb9664cb7-lmrkw"
Oct 30 19:40:28.791779 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:28.791751    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-d05eeacd-596f4f9cb6-tkmg2"
Oct 30 19:40:28.791913 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:28.791803    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-d05eeacd-596f4f9cb6-tkmg2" event={"ID":"0f0dbb94-ef0a-4531-9b45-2bb2dff8d130","Type":"ContainerStarted","Data":"73471f672fe6466ac73ee2014ff8d8e737249ad9a59c60f3559ed9471dfa4547"}
Oct 30 19:40:28.791913 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:28.791829    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-d05eeacd-596f4f9cb6-tkmg2" event={"ID":"0f0dbb94-ef0a-4531-9b45-2bb2dff8d130","Type":"ContainerStarted","Data":"bce8a157eff3bf25e7cf4478275bb6a46706fb8d2b02eb36ff4eec8aec7e9211"}
Oct 30 19:40:28.802587 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:28.802542    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-d05eeacd-6bb9664cb7-lmrkw" podStartSLOduration=1.802530172 podStartE2EDuration="1.802530172s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:40:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:40:28.799550063 +0000 UTC m=+504.127668219" watchObservedRunningTime="2026-10-30 19:40:28.802530172 +0000 UTC m=+504.130648350"
Oct 30 19:40:28.818768 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:28.818719    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-d05eeacd-596f4f9cb6-tkmg2" podStartSLOduration=1.818705371 podStartE2EDuration="1.818705371s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:40:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:40:28.816623871 +0000 UTC m=+504.144742030" watchObservedRunningTime="2026-10-30 19:40:28.818705371 +0000 UTC m=+504.146823525"
Oct 30 19:40:29.333883 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:29.333846    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 19:40:29.761556 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:29.761421    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3a55af38-564c4bcc96-cdl5f"
Oct 30 19:40:30.533723 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:30.533693    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-1d482894-56c95f9f44-m528s"
Oct 30 19:40:30.535304 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:30.535277    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-1d482894-6fc5774b99-msqpt"
Oct 30 19:40:31.192732 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:40:31.192699    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 19:40:32.482272 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:32.482239    6996 patch_prober.go:29] interesting pod/e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Oct 30 19:40:32.482790 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:32.482296    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f" podUID="8529bd14-0cbc-4e29-a8e6-8d46acb50268" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Oct 30 19:40:32.832987 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:32.832950    6996 generic.go:356] "Generic (PLEG): container finished" podID="8529bd14-0cbc-4e29-a8e6-8d46acb50268" containerID="ab97147e44f11a3cb1cffa87c40ea670f07d3d7232a80aca20df994b0b91a2b0" exitCode=0
Oct 30 19:40:32.833189 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:32.833023    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f" event={"ID":"8529bd14-0cbc-4e29-a8e6-8d46acb50268","Type":"ContainerDied","Data":"ab97147e44f11a3cb1cffa87c40ea670f07d3d7232a80aca20df994b0b91a2b0"}
Oct 30 19:40:32.833460 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:32.833435    6996 scope.go:122] "RemoveContainer" containerID="ab97147e44f11a3cb1cffa87c40ea670f07d3d7232a80aca20df994b0b91a2b0"
Oct 30 19:40:32.834913 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:32.834878    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:40:32.835033 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:32.834965    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:40:32.835033 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:32.835014    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:40:33.880976 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:33.880940    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f" event={"ID":"8529bd14-0cbc-4e29-a8e6-8d46acb50268","Type":"ContainerStarted","Data":"09890536dfb76cf057f03f6bea205f116c4ba9f6d8f7f3c74112571e5ddaf011"}
Oct 30 19:40:33.949934 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:33.949900    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f"
Oct 30 19:40:33.950321 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:33.950234    6996 patch_prober.go:29] interesting pod/e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Oct 30 19:40:33.950531 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:33.950468    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f" podUID="8529bd14-0cbc-4e29-a8e6-8d46acb50268" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Oct 30 19:40:34.959639 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:34.959603    6996 patch_prober.go:29] interesting pod/e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Oct 30 19:40:34.960658 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:34.960221    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f" podUID="8529bd14-0cbc-4e29-a8e6-8d46acb50268" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Oct 30 19:40:35.894542 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:35.894509    6996 generic.go:356] "Generic (PLEG): container finished" podID="b3a80a33-44df-464f-bdf1-5f9804cb29c5" containerID="fab12fe15a32f1bf9e890dc0309116c75de5dbeae9a82076cb670207e5afab16" exitCode=0
Oct 30 19:40:35.894768 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:35.894588    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw" event={"ID":"b3a80a33-44df-464f-bdf1-5f9804cb29c5","Type":"ContainerDied","Data":"fab12fe15a32f1bf9e890dc0309116c75de5dbeae9a82076cb670207e5afab16"}
Oct 30 19:40:35.895030 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:35.895016    6996 scope.go:122] "RemoveContainer" containerID="fab12fe15a32f1bf9e890dc0309116c75de5dbeae9a82076cb670207e5afab16"
Oct 30 19:40:35.896433 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:35.896404    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:40:35.896510 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:35.896490    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:40:35.896550 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:35.896527    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:40:35.943348 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:35.943319    6996 patch_prober.go:29] interesting pod/e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Oct 30 19:40:35.943717 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:35.943684    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f" podUID="8529bd14-0cbc-4e29-a8e6-8d46acb50268" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Oct 30 19:40:36.901609 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:36.901566    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw" event={"ID":"b3a80a33-44df-464f-bdf1-5f9804cb29c5","Type":"ContainerStarted","Data":"4189e74b6b2a3e4b7583b1dfaf4dce14ea23b260a5d376ab9bc58fc08bd1a2ff"}
Oct 30 19:40:36.949397 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:36.948927    6996 patch_prober.go:29] interesting pod/e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Oct 30 19:40:36.949397 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:36.949006    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f" podUID="8529bd14-0cbc-4e29-a8e6-8d46acb50268" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Oct 30 19:40:37.430107 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:37.430076    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw"
Oct 30 19:40:37.430392 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:37.430371    6996 patch_prober.go:29] interesting pod/e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Oct 30 19:40:37.430457 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:37.430436    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw" podUID="b3a80a33-44df-464f-bdf1-5f9804cb29c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Oct 30 19:40:37.945952 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:37.945912    6996 patch_prober.go:29] interesting pod/e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Oct 30 19:40:37.950595 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:37.950551    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f" podUID="8529bd14-0cbc-4e29-a8e6-8d46acb50268" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Oct 30 19:40:38.430222 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:38.430182    6996 patch_prober.go:29] interesting pod/e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Oct 30 19:40:38.430414 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:38.430236    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw" podUID="b3a80a33-44df-464f-bdf1-5f9804cb29c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Oct 30 19:40:38.801310 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:38.801227    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 19:40:38.963000 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:38.962950    6996 patch_prober.go:29] interesting pod/e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Oct 30 19:40:38.963625 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:38.963592    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f" podUID="8529bd14-0cbc-4e29-a8e6-8d46acb50268" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Oct 30 19:40:39.430718 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:39.430682    6996 patch_prober.go:29] interesting pod/e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Oct 30 19:40:39.430897 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:39.430740    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw" podUID="b3a80a33-44df-464f-bdf1-5f9804cb29c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Oct 30 19:40:39.813598 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:39.813524    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-d05eeacd-6bb9664cb7-lmrkw"
Oct 30 19:40:39.813598 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:39.813595    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-d05eeacd-596f4f9cb6-tkmg2"
Oct 30 19:40:39.951511 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:39.944958    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f"
Oct 30 19:40:39.951511 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:39.946311    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f"
Oct 30 19:40:39.951762 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:39.951621    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f"
Oct 30 19:40:40.430331 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:40.430292    6996 patch_prober.go:29] interesting pod/e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Oct 30 19:40:40.430862 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:40.430351    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw" podUID="b3a80a33-44df-464f-bdf1-5f9804cb29c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Oct 30 19:40:41.430079 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:41.430045    6996 patch_prober.go:29] interesting pod/e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Oct 30 19:40:41.430247 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:41.430099    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw" podUID="b3a80a33-44df-464f-bdf1-5f9804cb29c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Oct 30 19:40:42.373597 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:42.373565    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-1d482894-56c95f9f44-m528s"]
Oct 30 19:40:42.394190 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:42.394160    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-1d482894-6fc5774b99-msqpt"]
Oct 30 19:40:42.436257 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:42.436223    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw"
Oct 30 19:40:42.437103 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:42.437080    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw"
Oct 30 19:40:42.438111 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:42.438068    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw"
Oct 30 19:40:44.035906 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:44.035872    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-1d482894-bbfc8cdb9-dpnsm"]
Oct 30 19:40:44.036501 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:44.036131    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1d482894-bbfc8cdb9-dpnsm" podUID="5ba269cd-36fe-4a68-ae4e-1c7e7c7076a1" containerName="maas-api" containerID="cri-o://3ae72d15bd2446bdbe0f60c5a5340f18abb5f3a844179f102a1059002f7695d4" gracePeriod=30
Oct 30 19:40:44.320251 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:44.320223    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1d482894-bbfc8cdb9-dpnsm"
Oct 30 19:40:44.398874 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:44.398839    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5ba269cd-36fe-4a68-ae4e-1c7e7c7076a1-metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ba269cd-36fe-4a68-ae4e-1c7e7c7076a1-metrics-certs\") pod \"5ba269cd-36fe-4a68-ae4e-1c7e7c7076a1\" (UID: \"5ba269cd-36fe-4a68-ae4e-1c7e7c7076a1\") "
Oct 30 19:40:44.399086 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:44.398888    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5ba269cd-36fe-4a68-ae4e-1c7e7c7076a1-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5ba269cd-36fe-4a68-ae4e-1c7e7c7076a1-maas-api-tls\") pod \"5ba269cd-36fe-4a68-ae4e-1c7e7c7076a1\" (UID: \"5ba269cd-36fe-4a68-ae4e-1c7e7c7076a1\") "
Oct 30 19:40:44.399164 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:44.399085    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5ba269cd-36fe-4a68-ae4e-1c7e7c7076a1-kube-api-access-lb2rc\" (UniqueName: \"kubernetes.io/projected/5ba269cd-36fe-4a68-ae4e-1c7e7c7076a1-kube-api-access-lb2rc\") pod \"5ba269cd-36fe-4a68-ae4e-1c7e7c7076a1\" (UID: \"5ba269cd-36fe-4a68-ae4e-1c7e7c7076a1\") "
Oct 30 19:40:44.399164 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:44.399136    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5ba269cd-36fe-4a68-ae4e-1c7e7c7076a1-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ba269cd-36fe-4a68-ae4e-1c7e7c7076a1-openshift-service-ca\") pod \"5ba269cd-36fe-4a68-ae4e-1c7e7c7076a1\" (UID: \"5ba269cd-36fe-4a68-ae4e-1c7e7c7076a1\") "
Oct 30 19:40:44.399608 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:44.399571    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ba269cd-36fe-4a68-ae4e-1c7e7c7076a1-openshift-service-ca" pod "5ba269cd-36fe-4a68-ae4e-1c7e7c7076a1" (UID: "5ba269cd-36fe-4a68-ae4e-1c7e7c7076a1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:40:44.401297 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:44.401271    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ba269cd-36fe-4a68-ae4e-1c7e7c7076a1-maas-api-tls" pod "5ba269cd-36fe-4a68-ae4e-1c7e7c7076a1" (UID: "5ba269cd-36fe-4a68-ae4e-1c7e7c7076a1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:40:44.401392 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:44.401297    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ba269cd-36fe-4a68-ae4e-1c7e7c7076a1-metrics-certs" pod "5ba269cd-36fe-4a68-ae4e-1c7e7c7076a1" (UID: "5ba269cd-36fe-4a68-ae4e-1c7e7c7076a1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:40:44.401392 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:44.401378    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ba269cd-36fe-4a68-ae4e-1c7e7c7076a1-kube-api-access-lb2rc" pod "5ba269cd-36fe-4a68-ae4e-1c7e7c7076a1" (UID: "5ba269cd-36fe-4a68-ae4e-1c7e7c7076a1"). InnerVolumeSpecName "kube-api-access-lb2rc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:40:44.500935 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:44.500898    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lb2rc\" (UniqueName: \"kubernetes.io/projected/5ba269cd-36fe-4a68-ae4e-1c7e7c7076a1-kube-api-access-lb2rc\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:40:44.500935 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:44.500929    6996 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ba269cd-36fe-4a68-ae4e-1c7e7c7076a1-openshift-service-ca\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:40:44.500935 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:44.500942    6996 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ba269cd-36fe-4a68-ae4e-1c7e7c7076a1-metrics-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:40:44.501185 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:44.500953    6996 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5ba269cd-36fe-4a68-ae4e-1c7e7c7076a1-maas-api-tls\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:40:44.608144 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:44.608110    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f"]
Oct 30 19:40:44.608334 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:44.608303    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f" podUID="8529bd14-0cbc-4e29-a8e6-8d46acb50268" containerName="istio-proxy" containerID="cri-o://09890536dfb76cf057f03f6bea205f116c4ba9f6d8f7f3c74112571e5ddaf011" gracePeriod=30
Oct 30 19:40:44.948363 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:44.948271    6996 generic.go:356] "Generic (PLEG): container finished" podID="5ba269cd-36fe-4a68-ae4e-1c7e7c7076a1" containerID="3ae72d15bd2446bdbe0f60c5a5340f18abb5f3a844179f102a1059002f7695d4" exitCode=0
Oct 30 19:40:44.948363 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:44.948339    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1d482894-bbfc8cdb9-dpnsm"
Oct 30 19:40:44.948629 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:44.948352    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1d482894-bbfc8cdb9-dpnsm" event={"ID":"5ba269cd-36fe-4a68-ae4e-1c7e7c7076a1","Type":"ContainerDied","Data":"3ae72d15bd2446bdbe0f60c5a5340f18abb5f3a844179f102a1059002f7695d4"}
Oct 30 19:40:44.948629 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:44.948458    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1d482894-bbfc8cdb9-dpnsm" event={"ID":"5ba269cd-36fe-4a68-ae4e-1c7e7c7076a1","Type":"ContainerDied","Data":"d37b5a23d958211710acf9197add407e7af10b5895d520eb9d7c8501372e9d03"}
Oct 30 19:40:44.948629 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:44.948508    6996 scope.go:122] "RemoveContainer" containerID="3ae72d15bd2446bdbe0f60c5a5340f18abb5f3a844179f102a1059002f7695d4"
Oct 30 19:40:44.957752 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:44.957736    6996 scope.go:122] "RemoveContainer" containerID="3ae72d15bd2446bdbe0f60c5a5340f18abb5f3a844179f102a1059002f7695d4"
Oct 30 19:40:44.958066 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:40:44.958045    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ae72d15bd2446bdbe0f60c5a5340f18abb5f3a844179f102a1059002f7695d4\": container with ID starting with 3ae72d15bd2446bdbe0f60c5a5340f18abb5f3a844179f102a1059002f7695d4 not found: ID does not exist" containerID="3ae72d15bd2446bdbe0f60c5a5340f18abb5f3a844179f102a1059002f7695d4"
Oct 30 19:40:44.958117 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:44.958074    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ae72d15bd2446bdbe0f60c5a5340f18abb5f3a844179f102a1059002f7695d4"} err="failed to get container status \"3ae72d15bd2446bdbe0f60c5a5340f18abb5f3a844179f102a1059002f7695d4\": rpc error: code = NotFound desc = could not find container \"3ae72d15bd2446bdbe0f60c5a5340f18abb5f3a844179f102a1059002f7695d4\": container with ID starting with 3ae72d15bd2446bdbe0f60c5a5340f18abb5f3a844179f102a1059002f7695d4 not found: ID does not exist"
Oct 30 19:40:44.982559 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:44.982532    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-1d482894-bbfc8cdb9-dpnsm"]
Oct 30 19:40:44.991178 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:44.991142    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-1d482894-bbfc8cdb9-dpnsm"]
Oct 30 19:40:46.791055 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:46.791022    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5ba269cd-36fe-4a68-ae4e-1c7e7c7076a1" path="/var/lib/kubelet/pods/5ba269cd-36fe-4a68-ae4e-1c7e7c7076a1/volumes"
Oct 30 19:40:46.856225 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:46.856189    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 19:40:47.374593 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:47.374516    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-1d482894-56c95f9f44-m528s" podUID="36ff5561-cb31-4a4c-9702-3749feae0ffc" containerName="payload-processing" containerID="cri-o://28e7d89462f22570ead08a3d00249390d31bc2966bc79c0433b9141ce6845e04" gracePeriod=25
Oct 30 19:40:47.395121 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:47.395089    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-1d482894-6fc5774b99-msqpt" podUID="15e26590-2ab2-4b01-aa34-fbedcc89e919" containerName="payload-pre-processing" containerID="cri-o://8dcb960a751b2c68dace04750af1a88ee78da715cfdef7593d9b1debe527e235" gracePeriod=25
Oct 30 19:40:47.671997 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:47.671966    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-1d482894-56c95f9f44-m528s"
Oct 30 19:40:47.715829 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:47.715796    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-1d482894-6fc5774b99-msqpt"
Oct 30 19:40:47.735659 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:47.735625    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/36ff5561-cb31-4a4c-9702-3749feae0ffc-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36ff5561-cb31-4a4c-9702-3749feae0ffc-plugins-config-volume\") pod \"36ff5561-cb31-4a4c-9702-3749feae0ffc\" (UID: \"36ff5561-cb31-4a4c-9702-3749feae0ffc\") "
Oct 30 19:40:47.735837 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:47.735710    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/36ff5561-cb31-4a4c-9702-3749feae0ffc-kube-api-access-26mq2\" (UniqueName: \"kubernetes.io/projected/36ff5561-cb31-4a4c-9702-3749feae0ffc-kube-api-access-26mq2\") pod \"36ff5561-cb31-4a4c-9702-3749feae0ffc\" (UID: \"36ff5561-cb31-4a4c-9702-3749feae0ffc\") "
Oct 30 19:40:47.741969 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:47.739830    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36ff5561-cb31-4a4c-9702-3749feae0ffc-plugins-config-volume" pod "36ff5561-cb31-4a4c-9702-3749feae0ffc" (UID: "36ff5561-cb31-4a4c-9702-3749feae0ffc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:40:47.746755 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:47.746721    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36ff5561-cb31-4a4c-9702-3749feae0ffc-kube-api-access-26mq2" pod "36ff5561-cb31-4a4c-9702-3749feae0ffc" (UID: "36ff5561-cb31-4a4c-9702-3749feae0ffc"). InnerVolumeSpecName "kube-api-access-26mq2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:40:47.837853 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:47.837815    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/15e26590-2ab2-4b01-aa34-fbedcc89e919-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15e26590-2ab2-4b01-aa34-fbedcc89e919-plugins-config-volume\") pod \"15e26590-2ab2-4b01-aa34-fbedcc89e919\" (UID: \"15e26590-2ab2-4b01-aa34-fbedcc89e919\") "
Oct 30 19:40:47.838382 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:47.838209    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15e26590-2ab2-4b01-aa34-fbedcc89e919-plugins-config-volume" pod "15e26590-2ab2-4b01-aa34-fbedcc89e919" (UID: "15e26590-2ab2-4b01-aa34-fbedcc89e919"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:40:47.838512 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:47.838355    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/15e26590-2ab2-4b01-aa34-fbedcc89e919-kube-api-access-qsbqs\" (UniqueName: \"kubernetes.io/projected/15e26590-2ab2-4b01-aa34-fbedcc89e919-kube-api-access-qsbqs\") pod \"15e26590-2ab2-4b01-aa34-fbedcc89e919\" (UID: \"15e26590-2ab2-4b01-aa34-fbedcc89e919\") "
Oct 30 19:40:47.838878 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:47.838842    6996 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36ff5561-cb31-4a4c-9702-3749feae0ffc-plugins-config-volume\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:40:47.838956 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:47.838881    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-26mq2\" (UniqueName: \"kubernetes.io/projected/36ff5561-cb31-4a4c-9702-3749feae0ffc-kube-api-access-26mq2\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:40:47.838956 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:47.838892    6996 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15e26590-2ab2-4b01-aa34-fbedcc89e919-plugins-config-volume\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:40:47.840471 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:47.840446    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15e26590-2ab2-4b01-aa34-fbedcc89e919-kube-api-access-qsbqs" pod "15e26590-2ab2-4b01-aa34-fbedcc89e919" (UID: "15e26590-2ab2-4b01-aa34-fbedcc89e919"). InnerVolumeSpecName "kube-api-access-qsbqs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:40:47.940395 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:47.940290    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qsbqs\" (UniqueName: \"kubernetes.io/projected/15e26590-2ab2-4b01-aa34-fbedcc89e919-kube-api-access-qsbqs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:40:47.975051 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:47.975009    6996 generic.go:356] "Generic (PLEG): container finished" podID="15e26590-2ab2-4b01-aa34-fbedcc89e919" containerID="8dcb960a751b2c68dace04750af1a88ee78da715cfdef7593d9b1debe527e235" exitCode=0
Oct 30 19:40:47.975244 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:47.975177    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-1d482894-6fc5774b99-msqpt" event={"ID":"15e26590-2ab2-4b01-aa34-fbedcc89e919","Type":"ContainerDied","Data":"8dcb960a751b2c68dace04750af1a88ee78da715cfdef7593d9b1debe527e235"}
Oct 30 19:40:47.975244 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:47.975236    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-1d482894-6fc5774b99-msqpt" event={"ID":"15e26590-2ab2-4b01-aa34-fbedcc89e919","Type":"ContainerDied","Data":"c81ddb1dcbd708e7fbc1f6fa41c0e6972091c625dacef5b9e9566935d9f3730a"}
Oct 30 19:40:47.975367 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:47.975268    6996 scope.go:122] "RemoveContainer" containerID="8dcb960a751b2c68dace04750af1a88ee78da715cfdef7593d9b1debe527e235"
Oct 30 19:40:47.975425 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:47.975371    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-1d482894-6fc5774b99-msqpt"
Oct 30 19:40:47.985550 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:47.984836    6996 generic.go:356] "Generic (PLEG): container finished" podID="36ff5561-cb31-4a4c-9702-3749feae0ffc" containerID="28e7d89462f22570ead08a3d00249390d31bc2966bc79c0433b9141ce6845e04" exitCode=0
Oct 30 19:40:47.985550 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:47.984887    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-1d482894-56c95f9f44-m528s" event={"ID":"36ff5561-cb31-4a4c-9702-3749feae0ffc","Type":"ContainerDied","Data":"28e7d89462f22570ead08a3d00249390d31bc2966bc79c0433b9141ce6845e04"}
Oct 30 19:40:47.985550 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:47.984911    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-1d482894-56c95f9f44-m528s"
Oct 30 19:40:47.985550 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:47.984936    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-1d482894-56c95f9f44-m528s" event={"ID":"36ff5561-cb31-4a4c-9702-3749feae0ffc","Type":"ContainerDied","Data":"bf615a7b6e740d3f9919f9d2a6522374947bac72cda2ab8e225903ac2f15fa83"}
Oct 30 19:40:48.006697 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:48.006657    6996 scope.go:122] "RemoveContainer" containerID="8dcb960a751b2c68dace04750af1a88ee78da715cfdef7593d9b1debe527e235"
Oct 30 19:40:48.007285 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:40:48.007259    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8dcb960a751b2c68dace04750af1a88ee78da715cfdef7593d9b1debe527e235\": container with ID starting with 8dcb960a751b2c68dace04750af1a88ee78da715cfdef7593d9b1debe527e235 not found: ID does not exist" containerID="8dcb960a751b2c68dace04750af1a88ee78da715cfdef7593d9b1debe527e235"
Oct 30 19:40:48.007397 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:48.007298    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8dcb960a751b2c68dace04750af1a88ee78da715cfdef7593d9b1debe527e235"} err="failed to get container status \"8dcb960a751b2c68dace04750af1a88ee78da715cfdef7593d9b1debe527e235\": rpc error: code = NotFound desc = could not find container \"8dcb960a751b2c68dace04750af1a88ee78da715cfdef7593d9b1debe527e235\": container with ID starting with 8dcb960a751b2c68dace04750af1a88ee78da715cfdef7593d9b1debe527e235 not found: ID does not exist"
Oct 30 19:40:48.007397 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:48.007322    6996 scope.go:122] "RemoveContainer" containerID="28e7d89462f22570ead08a3d00249390d31bc2966bc79c0433b9141ce6845e04"
Oct 30 19:40:48.009375 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:48.009354    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-1d482894-6fc5774b99-msqpt"]
Oct 30 19:40:48.012359 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:48.012337    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-1d482894-6fc5774b99-msqpt"]
Oct 30 19:40:48.016511 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:48.016462    6996 scope.go:122] "RemoveContainer" containerID="28e7d89462f22570ead08a3d00249390d31bc2966bc79c0433b9141ce6845e04"
Oct 30 19:40:48.016804 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:40:48.016783    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28e7d89462f22570ead08a3d00249390d31bc2966bc79c0433b9141ce6845e04\": container with ID starting with 28e7d89462f22570ead08a3d00249390d31bc2966bc79c0433b9141ce6845e04 not found: ID does not exist" containerID="28e7d89462f22570ead08a3d00249390d31bc2966bc79c0433b9141ce6845e04"
Oct 30 19:40:48.016850 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:48.016812    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28e7d89462f22570ead08a3d00249390d31bc2966bc79c0433b9141ce6845e04"} err="failed to get container status \"28e7d89462f22570ead08a3d00249390d31bc2966bc79c0433b9141ce6845e04\": rpc error: code = NotFound desc = could not find container \"28e7d89462f22570ead08a3d00249390d31bc2966bc79c0433b9141ce6845e04\": container with ID starting with 28e7d89462f22570ead08a3d00249390d31bc2966bc79c0433b9141ce6845e04 not found: ID does not exist"
Oct 30 19:40:48.026236 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:48.026212    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-1d482894-56c95f9f44-m528s"]
Oct 30 19:40:48.034125 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:48.034101    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-1d482894-56c95f9f44-m528s"]
Oct 30 19:40:48.791301 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:48.791211    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="15e26590-2ab2-4b01-aa34-fbedcc89e919" path="/var/lib/kubelet/pods/15e26590-2ab2-4b01-aa34-fbedcc89e919/volumes"
Oct 30 19:40:48.791722 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:48.791702    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="36ff5561-cb31-4a4c-9702-3749feae0ffc" path="/var/lib/kubelet/pods/36ff5561-cb31-4a4c-9702-3749feae0ffc/volumes"
Oct 30 19:40:49.655331 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:49.655297    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-3a55af38-564c4bcc96-cdl5f"]
Oct 30 19:40:49.655716 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:49.655561    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3a55af38-564c4bcc96-cdl5f" podUID="8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d" containerName="maas-api" containerID="cri-o://13a5203dbeb03bdf2a6df2e17224bc563b45b7fc6be6aa7f1ce1fcb532bdf19b" gracePeriod=30
Oct 30 19:40:49.723748 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:49.723701    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3a55af38-564c4bcc96-cdl5f" podUID="8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.59:8443/health\": dial tcp 10.134.0.59:8443: connect: connection refused"
Oct 30 19:40:49.933263 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:49.933210    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f"
Oct 30 19:40:49.964824 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:49.964786    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3a55af38-564c4bcc96-cdl5f"
Oct 30 19:40:50.005375 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:50.005335    6996 generic.go:356] "Generic (PLEG): container finished" podID="8529bd14-0cbc-4e29-a8e6-8d46acb50268" containerID="09890536dfb76cf057f03f6bea205f116c4ba9f6d8f7f3c74112571e5ddaf011" exitCode=0
Oct 30 19:40:50.005572 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:50.005404    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f"
Oct 30 19:40:50.005572 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:50.005417    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f" event={"ID":"8529bd14-0cbc-4e29-a8e6-8d46acb50268","Type":"ContainerDied","Data":"09890536dfb76cf057f03f6bea205f116c4ba9f6d8f7f3c74112571e5ddaf011"}
Oct 30 19:40:50.005572 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:50.005471    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f" event={"ID":"8529bd14-0cbc-4e29-a8e6-8d46acb50268","Type":"ContainerDied","Data":"50baf6252fa23d60e296c1783e3ca4488317f7042570869f86149ea5d960bb17"}
Oct 30 19:40:50.005572 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:50.005508    6996 scope.go:122] "RemoveContainer" containerID="09890536dfb76cf057f03f6bea205f116c4ba9f6d8f7f3c74112571e5ddaf011"
Oct 30 19:40:50.007290 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:50.007168    6996 generic.go:356] "Generic (PLEG): container finished" podID="8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d" containerID="13a5203dbeb03bdf2a6df2e17224bc563b45b7fc6be6aa7f1ce1fcb532bdf19b" exitCode=0
Oct 30 19:40:50.007290 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:50.007199    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3a55af38-564c4bcc96-cdl5f" event={"ID":"8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d","Type":"ContainerDied","Data":"13a5203dbeb03bdf2a6df2e17224bc563b45b7fc6be6aa7f1ce1fcb532bdf19b"}
Oct 30 19:40:50.007290 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:50.007224    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3a55af38-564c4bcc96-cdl5f"
Oct 30 19:40:50.007503 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:50.007231    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3a55af38-564c4bcc96-cdl5f" event={"ID":"8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d","Type":"ContainerDied","Data":"0d0341b9a53aba7e41f02e95d112299368f7b56921f4808ba3a74a0a9c90b41f"}
Oct 30 19:40:50.016920 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:50.016899    6996 scope.go:122] "RemoveContainer" containerID="ab97147e44f11a3cb1cffa87c40ea670f07d3d7232a80aca20df994b0b91a2b0"
Oct 30 19:40:50.028730 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:50.028704    6996 scope.go:122] "RemoveContainer" containerID="09890536dfb76cf057f03f6bea205f116c4ba9f6d8f7f3c74112571e5ddaf011"
Oct 30 19:40:50.029066 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:40:50.029045    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09890536dfb76cf057f03f6bea205f116c4ba9f6d8f7f3c74112571e5ddaf011\": container with ID starting with 09890536dfb76cf057f03f6bea205f116c4ba9f6d8f7f3c74112571e5ddaf011 not found: ID does not exist" containerID="09890536dfb76cf057f03f6bea205f116c4ba9f6d8f7f3c74112571e5ddaf011"
Oct 30 19:40:50.029171 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:50.029081    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09890536dfb76cf057f03f6bea205f116c4ba9f6d8f7f3c74112571e5ddaf011"} err="failed to get container status \"09890536dfb76cf057f03f6bea205f116c4ba9f6d8f7f3c74112571e5ddaf011\": rpc error: code = NotFound desc = could not find container \"09890536dfb76cf057f03f6bea205f116c4ba9f6d8f7f3c74112571e5ddaf011\": container with ID starting with 09890536dfb76cf057f03f6bea205f116c4ba9f6d8f7f3c74112571e5ddaf011 not found: ID does not exist"
Oct 30 19:40:50.029171 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:50.029107    6996 scope.go:122] "RemoveContainer" containerID="ab97147e44f11a3cb1cffa87c40ea670f07d3d7232a80aca20df994b0b91a2b0"
Oct 30 19:40:50.029446 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:40:50.029424    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab97147e44f11a3cb1cffa87c40ea670f07d3d7232a80aca20df994b0b91a2b0\": container with ID starting with ab97147e44f11a3cb1cffa87c40ea670f07d3d7232a80aca20df994b0b91a2b0 not found: ID does not exist" containerID="ab97147e44f11a3cb1cffa87c40ea670f07d3d7232a80aca20df994b0b91a2b0"
Oct 30 19:40:50.029554 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:50.029450    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab97147e44f11a3cb1cffa87c40ea670f07d3d7232a80aca20df994b0b91a2b0"} err="failed to get container status \"ab97147e44f11a3cb1cffa87c40ea670f07d3d7232a80aca20df994b0b91a2b0\": rpc error: code = NotFound desc = could not find container \"ab97147e44f11a3cb1cffa87c40ea670f07d3d7232a80aca20df994b0b91a2b0\": container with ID starting with ab97147e44f11a3cb1cffa87c40ea670f07d3d7232a80aca20df994b0b91a2b0 not found: ID does not exist"
Oct 30 19:40:50.029554 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:50.029465    6996 scope.go:122] "RemoveContainer" containerID="13a5203dbeb03bdf2a6df2e17224bc563b45b7fc6be6aa7f1ce1fcb532bdf19b"
Oct 30 19:40:50.038722 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:50.038702    6996 scope.go:122] "RemoveContainer" containerID="13a5203dbeb03bdf2a6df2e17224bc563b45b7fc6be6aa7f1ce1fcb532bdf19b"
Oct 30 19:40:50.039052 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:40:50.039033    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13a5203dbeb03bdf2a6df2e17224bc563b45b7fc6be6aa7f1ce1fcb532bdf19b\": container with ID starting with 13a5203dbeb03bdf2a6df2e17224bc563b45b7fc6be6aa7f1ce1fcb532bdf19b not found: ID does not exist" containerID="13a5203dbeb03bdf2a6df2e17224bc563b45b7fc6be6aa7f1ce1fcb532bdf19b"
Oct 30 19:40:50.039105 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:50.039066    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13a5203dbeb03bdf2a6df2e17224bc563b45b7fc6be6aa7f1ce1fcb532bdf19b"} err="failed to get container status \"13a5203dbeb03bdf2a6df2e17224bc563b45b7fc6be6aa7f1ce1fcb532bdf19b\": rpc error: code = NotFound desc = could not find container \"13a5203dbeb03bdf2a6df2e17224bc563b45b7fc6be6aa7f1ce1fcb532bdf19b\": container with ID starting with 13a5203dbeb03bdf2a6df2e17224bc563b45b7fc6be6aa7f1ce1fcb532bdf19b not found: ID does not exist"
Oct 30 19:40:50.063199 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:50.063169    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8529bd14-0cbc-4e29-a8e6-8d46acb50268-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8529bd14-0cbc-4e29-a8e6-8d46acb50268-istiod-ca-cert\") pod \"8529bd14-0cbc-4e29-a8e6-8d46acb50268\" (UID: \"8529bd14-0cbc-4e29-a8e6-8d46acb50268\") "
Oct 30 19:40:50.063352 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:50.063211    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8529bd14-0cbc-4e29-a8e6-8d46acb50268-kube-api-access-s2k59\" (UniqueName: \"kubernetes.io/projected/8529bd14-0cbc-4e29-a8e6-8d46acb50268-kube-api-access-s2k59\") pod \"8529bd14-0cbc-4e29-a8e6-8d46acb50268\" (UID: \"8529bd14-0cbc-4e29-a8e6-8d46acb50268\") "
Oct 30 19:40:50.063352 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:50.063229    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d-openshift-service-ca\") pod \"8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d\" (UID: \"8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d\") "
Oct 30 19:40:50.063352 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:50.063303    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8529bd14-0cbc-4e29-a8e6-8d46acb50268-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8529bd14-0cbc-4e29-a8e6-8d46acb50268-credential-socket\") pod \"8529bd14-0cbc-4e29-a8e6-8d46acb50268\" (UID: \"8529bd14-0cbc-4e29-a8e6-8d46acb50268\") "
Oct 30 19:40:50.063352 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:50.063338    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8529bd14-0cbc-4e29-a8e6-8d46acb50268-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8529bd14-0cbc-4e29-a8e6-8d46acb50268-istio-podinfo\") pod \"8529bd14-0cbc-4e29-a8e6-8d46acb50268\" (UID: \"8529bd14-0cbc-4e29-a8e6-8d46acb50268\") "
Oct 30 19:40:50.063607 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:50.063369    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d-kube-api-access-qdgbd\" (UniqueName: \"kubernetes.io/projected/8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d-kube-api-access-qdgbd\") pod \"8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d\" (UID: \"8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d\") "
Oct 30 19:40:50.063607 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:50.063406    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8529bd14-0cbc-4e29-a8e6-8d46acb50268-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8529bd14-0cbc-4e29-a8e6-8d46acb50268-workload-certs\") pod \"8529bd14-0cbc-4e29-a8e6-8d46acb50268\" (UID: \"8529bd14-0cbc-4e29-a8e6-8d46acb50268\") "
Oct 30 19:40:50.063607 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:50.063434    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8529bd14-0cbc-4e29-a8e6-8d46acb50268-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8529bd14-0cbc-4e29-a8e6-8d46acb50268-istio-envoy\") pod \"8529bd14-0cbc-4e29-a8e6-8d46acb50268\" (UID: \"8529bd14-0cbc-4e29-a8e6-8d46acb50268\") "
Oct 30 19:40:50.063607 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:50.063475    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8529bd14-0cbc-4e29-a8e6-8d46acb50268-istio-token\" (UniqueName: \"kubernetes.io/projected/8529bd14-0cbc-4e29-a8e6-8d46acb50268-istio-token\") pod \"8529bd14-0cbc-4e29-a8e6-8d46acb50268\" (UID: \"8529bd14-0cbc-4e29-a8e6-8d46acb50268\") "
Oct 30 19:40:50.063607 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:50.063517    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8529bd14-0cbc-4e29-a8e6-8d46acb50268-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8529bd14-0cbc-4e29-a8e6-8d46acb50268-istio-data\") pod \"8529bd14-0cbc-4e29-a8e6-8d46acb50268\" (UID: \"8529bd14-0cbc-4e29-a8e6-8d46acb50268\") "
Oct 30 19:40:50.063607 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:50.063556    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8529bd14-0cbc-4e29-a8e6-8d46acb50268-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8529bd14-0cbc-4e29-a8e6-8d46acb50268-workload-socket\") pod \"8529bd14-0cbc-4e29-a8e6-8d46acb50268\" (UID: \"8529bd14-0cbc-4e29-a8e6-8d46acb50268\") "
Oct 30 19:40:50.064993 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:50.063639    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d-maas-api-tls\") pod \"8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d\" (UID: \"8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d\") "
Oct 30 19:40:50.064993 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:50.063641    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8529bd14-0cbc-4e29-a8e6-8d46acb50268-istiod-ca-cert" pod "8529bd14-0cbc-4e29-a8e6-8d46acb50268" (UID: "8529bd14-0cbc-4e29-a8e6-8d46acb50268"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:40:50.064993 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:50.063642    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8529bd14-0cbc-4e29-a8e6-8d46acb50268-credential-socket" pod "8529bd14-0cbc-4e29-a8e6-8d46acb50268" (UID: "8529bd14-0cbc-4e29-a8e6-8d46acb50268"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:40:50.064993 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:50.063657    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d-openshift-service-ca" pod "8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d" (UID: "8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:40:50.064993 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:50.063981    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d-metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d-metrics-certs\") pod \"8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d\" (UID: \"8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d\") "
Oct 30 19:40:50.064993 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:50.064548    6996 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8529bd14-0cbc-4e29-a8e6-8d46acb50268-istiod-ca-cert\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:40:50.064993 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:50.064570    6996 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d-openshift-service-ca\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:40:50.064993 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:50.064584    6996 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8529bd14-0cbc-4e29-a8e6-8d46acb50268-credential-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:40:50.064993 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:50.063983    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8529bd14-0cbc-4e29-a8e6-8d46acb50268-workload-certs" pod "8529bd14-0cbc-4e29-a8e6-8d46acb50268" (UID: "8529bd14-0cbc-4e29-a8e6-8d46acb50268"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:40:50.064993 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:50.064244    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8529bd14-0cbc-4e29-a8e6-8d46acb50268-istio-data" pod "8529bd14-0cbc-4e29-a8e6-8d46acb50268" (UID: "8529bd14-0cbc-4e29-a8e6-8d46acb50268"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:40:50.064993 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:50.064395    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8529bd14-0cbc-4e29-a8e6-8d46acb50268-workload-socket" pod "8529bd14-0cbc-4e29-a8e6-8d46acb50268" (UID: "8529bd14-0cbc-4e29-a8e6-8d46acb50268"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:40:50.066222 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:50.066117    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8529bd14-0cbc-4e29-a8e6-8d46acb50268-istio-envoy" pod "8529bd14-0cbc-4e29-a8e6-8d46acb50268" (UID: "8529bd14-0cbc-4e29-a8e6-8d46acb50268"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:40:50.066437 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:50.066406    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8529bd14-0cbc-4e29-a8e6-8d46acb50268-istio-podinfo" pod "8529bd14-0cbc-4e29-a8e6-8d46acb50268" (UID: "8529bd14-0cbc-4e29-a8e6-8d46acb50268"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:40:50.066889 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:50.066810    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d-kube-api-access-qdgbd" pod "8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d" (UID: "8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d"). InnerVolumeSpecName "kube-api-access-qdgbd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:40:50.066991 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:50.066922    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8529bd14-0cbc-4e29-a8e6-8d46acb50268-kube-api-access-s2k59" pod "8529bd14-0cbc-4e29-a8e6-8d46acb50268" (UID: "8529bd14-0cbc-4e29-a8e6-8d46acb50268"). InnerVolumeSpecName "kube-api-access-s2k59". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:40:50.067200 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:50.067180    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d-maas-api-tls" pod "8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d" (UID: "8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:40:50.067576 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:50.067554    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8529bd14-0cbc-4e29-a8e6-8d46acb50268-istio-token" pod "8529bd14-0cbc-4e29-a8e6-8d46acb50268" (UID: "8529bd14-0cbc-4e29-a8e6-8d46acb50268"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:40:50.068359 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:50.068338    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d-metrics-certs" pod "8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d" (UID: "8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:40:50.165546 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:50.165445    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s2k59\" (UniqueName: \"kubernetes.io/projected/8529bd14-0cbc-4e29-a8e6-8d46acb50268-kube-api-access-s2k59\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:40:50.165546 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:50.165515    6996 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8529bd14-0cbc-4e29-a8e6-8d46acb50268-istio-podinfo\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:40:50.165546 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:50.165534    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qdgbd\" (UniqueName: \"kubernetes.io/projected/8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d-kube-api-access-qdgbd\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:40:50.165839 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:50.165547    6996 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8529bd14-0cbc-4e29-a8e6-8d46acb50268-workload-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:40:50.165839 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:50.165562    6996 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8529bd14-0cbc-4e29-a8e6-8d46acb50268-istio-envoy\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:40:50.165839 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:50.165575    6996 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8529bd14-0cbc-4e29-a8e6-8d46acb50268-istio-token\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:40:50.165839 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:50.165588    6996 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8529bd14-0cbc-4e29-a8e6-8d46acb50268-istio-data\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:40:50.165839 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:50.165622    6996 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8529bd14-0cbc-4e29-a8e6-8d46acb50268-workload-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:40:50.165839 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:50.165636    6996 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d-maas-api-tls\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:40:50.165839 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:50.165649    6996 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d-metrics-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:40:50.335220 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:50.335191    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-3a55af38-564c4bcc96-cdl5f"]
Oct 30 19:40:50.340226 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:50.340192    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-3a55af38-564c4bcc96-cdl5f"]
Oct 30 19:40:50.375084 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:50.375044    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f"]
Oct 30 19:40:50.382545 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:50.382514    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-1d482894-gw-openshift-default-777776b6d9-r4z4f"]
Oct 30 19:40:50.792369 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:50.792332    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8529bd14-0cbc-4e29-a8e6-8d46acb50268" path="/var/lib/kubelet/pods/8529bd14-0cbc-4e29-a8e6-8d46acb50268/volumes"
Oct 30 19:40:50.792978 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:50.792955    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d" path="/var/lib/kubelet/pods/8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d/volumes"
Oct 30 19:40:51.462451 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.462420    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2"]
Oct 30 19:40:51.463034 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.463007    6996 cpu_manager.go:414] "containerMap: removing container" podUID="8529bd14-0cbc-4e29-a8e6-8d46acb50268" containerName="istio-proxy"
Oct 30 19:40:51.463034 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.463032    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8529bd14-0cbc-4e29-a8e6-8d46acb50268" containerName="istio-proxy"
Oct 30 19:40:51.463217 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.463068    6996 cpu_manager.go:414] "containerMap: removing container" podUID="5ba269cd-36fe-4a68-ae4e-1c7e7c7076a1" containerName="maas-api"
Oct 30 19:40:51.463217 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.463076    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5ba269cd-36fe-4a68-ae4e-1c7e7c7076a1" containerName="maas-api"
Oct 30 19:40:51.463217 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.463093    6996 cpu_manager.go:414] "containerMap: removing container" podUID="36ff5561-cb31-4a4c-9702-3749feae0ffc" containerName="payload-processing"
Oct 30 19:40:51.463217 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.463101    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="36ff5561-cb31-4a4c-9702-3749feae0ffc" containerName="payload-processing"
Oct 30 19:40:51.463217 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.463116    6996 cpu_manager.go:414] "containerMap: removing container" podUID="8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d" containerName="maas-api"
Oct 30 19:40:51.463217 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.463123    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d" containerName="maas-api"
Oct 30 19:40:51.463217 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.463155    6996 cpu_manager.go:414] "containerMap: removing container" podUID="15e26590-2ab2-4b01-aa34-fbedcc89e919" containerName="payload-pre-processing"
Oct 30 19:40:51.463217 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.463164    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="15e26590-2ab2-4b01-aa34-fbedcc89e919" containerName="payload-pre-processing"
Oct 30 19:40:51.463633 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.463325    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="15e26590-2ab2-4b01-aa34-fbedcc89e919" containerName="payload-pre-processing"
Oct 30 19:40:51.463633 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.463338    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="8529bd14-0cbc-4e29-a8e6-8d46acb50268" containerName="istio-proxy"
Oct 30 19:40:51.463633 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.463351    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="8529bd14-0cbc-4e29-a8e6-8d46acb50268" containerName="istio-proxy"
Oct 30 19:40:51.463633 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.463366    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="5ba269cd-36fe-4a68-ae4e-1c7e7c7076a1" containerName="maas-api"
Oct 30 19:40:51.463633 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.463378    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="36ff5561-cb31-4a4c-9702-3749feae0ffc" containerName="payload-processing"
Oct 30 19:40:51.463633 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.463387    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="8a6e97a4-50fa-4d93-8f30-a79f3c30bb4d" containerName="maas-api"
Oct 30 19:40:51.468381 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.468361    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2"
Oct 30 19:40:51.507644 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.507615    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-f06af6-openshift-default-dockercfg-tcjql\""
Oct 30 19:40:51.518146 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.517848    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2"]
Oct 30 19:40:51.582183 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.582148    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/387db3d3-e192-44ca-b22b-e84f74c32a98-istiod-ca-cert\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2\" (UID: \"387db3d3-e192-44ca-b22b-e84f74c32a98\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2"
Oct 30 19:40:51.582370 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.582212    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/387db3d3-e192-44ca-b22b-e84f74c32a98-istio-podinfo\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2\" (UID: \"387db3d3-e192-44ca-b22b-e84f74c32a98\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2"
Oct 30 19:40:51.582370 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.582304    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/387db3d3-e192-44ca-b22b-e84f74c32a98-workload-certs\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2\" (UID: \"387db3d3-e192-44ca-b22b-e84f74c32a98\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2"
Oct 30 19:40:51.582370 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.582338    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/387db3d3-e192-44ca-b22b-e84f74c32a98-workload-socket\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2\" (UID: \"387db3d3-e192-44ca-b22b-e84f74c32a98\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2"
Oct 30 19:40:51.582472 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.582374    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/387db3d3-e192-44ca-b22b-e84f74c32a98-istio-token\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2\" (UID: \"387db3d3-e192-44ca-b22b-e84f74c32a98\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2"
Oct 30 19:40:51.582472 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.582454    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/387db3d3-e192-44ca-b22b-e84f74c32a98-istio-envoy\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2\" (UID: \"387db3d3-e192-44ca-b22b-e84f74c32a98\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2"
Oct 30 19:40:51.582557 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.582523    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/387db3d3-e192-44ca-b22b-e84f74c32a98-credential-socket\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2\" (UID: \"387db3d3-e192-44ca-b22b-e84f74c32a98\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2"
Oct 30 19:40:51.582588 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.582561    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5g7h\" (UniqueName: \"kubernetes.io/projected/387db3d3-e192-44ca-b22b-e84f74c32a98-kube-api-access-j5g7h\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2\" (UID: \"387db3d3-e192-44ca-b22b-e84f74c32a98\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2"
Oct 30 19:40:51.582621 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.582612    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/387db3d3-e192-44ca-b22b-e84f74c32a98-istio-data\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2\" (UID: \"387db3d3-e192-44ca-b22b-e84f74c32a98\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2"
Oct 30 19:40:51.641596 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.641558    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-c1fdd6-76556ddfb-w9t9z"]
Oct 30 19:40:51.642181 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.642160    6996 cpu_manager.go:414] "containerMap: removing container" podUID="8529bd14-0cbc-4e29-a8e6-8d46acb50268" containerName="istio-proxy"
Oct 30 19:40:51.642181 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.642181    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8529bd14-0cbc-4e29-a8e6-8d46acb50268" containerName="istio-proxy"
Oct 30 19:40:51.645433 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.645411    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-c1fdd6-76556ddfb-w9t9z"
Oct 30 19:40:51.663168 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.663144    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w3-c1fdd6\""
Oct 30 19:40:51.663298 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.663283    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w3-c1fdd6-dockercfg-4ghxg\""
Oct 30 19:40:51.667025 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.667002    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-c1fdd6-76556ddfb-w9t9z"]
Oct 30 19:40:51.677112 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.677087    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-c1fdd6-5d68c879bc-ccgjb"]
Oct 30 19:40:51.680837 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.680812    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-c1fdd6-5d68c879bc-ccgjb"
Oct 30 19:40:51.683753 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.683729    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/387db3d3-e192-44ca-b22b-e84f74c32a98-workload-certs\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2\" (UID: \"387db3d3-e192-44ca-b22b-e84f74c32a98\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2"
Oct 30 19:40:51.683876 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.683769    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/387db3d3-e192-44ca-b22b-e84f74c32a98-workload-socket\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2\" (UID: \"387db3d3-e192-44ca-b22b-e84f74c32a98\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2"
Oct 30 19:40:51.683876 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.683819    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/387db3d3-e192-44ca-b22b-e84f74c32a98-istio-token\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2\" (UID: \"387db3d3-e192-44ca-b22b-e84f74c32a98\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2"
Oct 30 19:40:51.683876 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.683863    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/387db3d3-e192-44ca-b22b-e84f74c32a98-istio-envoy\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2\" (UID: \"387db3d3-e192-44ca-b22b-e84f74c32a98\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2"
Oct 30 19:40:51.684029 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.684009    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/387db3d3-e192-44ca-b22b-e84f74c32a98-credential-socket\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2\" (UID: \"387db3d3-e192-44ca-b22b-e84f74c32a98\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2"
Oct 30 19:40:51.685635 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.684076    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5g7h\" (UniqueName: \"kubernetes.io/projected/387db3d3-e192-44ca-b22b-e84f74c32a98-kube-api-access-j5g7h\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2\" (UID: \"387db3d3-e192-44ca-b22b-e84f74c32a98\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2"
Oct 30 19:40:51.685635 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.684097    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/387db3d3-e192-44ca-b22b-e84f74c32a98-workload-certs\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2\" (UID: \"387db3d3-e192-44ca-b22b-e84f74c32a98\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2"
Oct 30 19:40:51.685635 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.684133    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/387db3d3-e192-44ca-b22b-e84f74c32a98-istio-data\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2\" (UID: \"387db3d3-e192-44ca-b22b-e84f74c32a98\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2"
Oct 30 19:40:51.685635 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.684163    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/387db3d3-e192-44ca-b22b-e84f74c32a98-workload-socket\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2\" (UID: \"387db3d3-e192-44ca-b22b-e84f74c32a98\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2"
Oct 30 19:40:51.685635 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.684273    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/387db3d3-e192-44ca-b22b-e84f74c32a98-istiod-ca-cert\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2\" (UID: \"387db3d3-e192-44ca-b22b-e84f74c32a98\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2"
Oct 30 19:40:51.685635 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.684340    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/387db3d3-e192-44ca-b22b-e84f74c32a98-credential-socket\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2\" (UID: \"387db3d3-e192-44ca-b22b-e84f74c32a98\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2"
Oct 30 19:40:51.685635 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.684371    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/387db3d3-e192-44ca-b22b-e84f74c32a98-istio-podinfo\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2\" (UID: \"387db3d3-e192-44ca-b22b-e84f74c32a98\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2"
Oct 30 19:40:51.685635 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.684411    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/387db3d3-e192-44ca-b22b-e84f74c32a98-istio-data\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2\" (UID: \"387db3d3-e192-44ca-b22b-e84f74c32a98\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2"
Oct 30 19:40:51.685635 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.685585    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/387db3d3-e192-44ca-b22b-e84f74c32a98-istiod-ca-cert\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2\" (UID: \"387db3d3-e192-44ca-b22b-e84f74c32a98\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2"
Oct 30 19:40:51.686430 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.686410    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/387db3d3-e192-44ca-b22b-e84f74c32a98-istio-envoy\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2\" (UID: \"387db3d3-e192-44ca-b22b-e84f74c32a98\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2"
Oct 30 19:40:51.687025 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.687007    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/387db3d3-e192-44ca-b22b-e84f74c32a98-istio-podinfo\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2\" (UID: \"387db3d3-e192-44ca-b22b-e84f74c32a98\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2"
Oct 30 19:40:51.692162 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.692143    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-c1fdd6-5d68c879bc-ccgjb"]
Oct 30 19:40:51.698325 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.698298    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/387db3d3-e192-44ca-b22b-e84f74c32a98-istio-token\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2\" (UID: \"387db3d3-e192-44ca-b22b-e84f74c32a98\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2"
Oct 30 19:40:51.698852 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.698834    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5g7h\" (UniqueName: \"kubernetes.io/projected/387db3d3-e192-44ca-b22b-e84f74c32a98-kube-api-access-j5g7h\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2\" (UID: \"387db3d3-e192-44ca-b22b-e84f74c32a98\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2"
Oct 30 19:40:51.783305 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.783270    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2"
Oct 30 19:40:51.785063 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.785033    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/71cedf55-9f63-4c56-b75f-ab67bb24d556-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-c1fdd6-76556ddfb-w9t9z\" (UID: \"71cedf55-9f63-4c56-b75f-ab67bb24d556\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-c1fdd6-76556ddfb-w9t9z"
Oct 30 19:40:51.785280 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.785073    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8d2w\" (UniqueName: \"kubernetes.io/projected/71cedf55-9f63-4c56-b75f-ab67bb24d556-kube-api-access-v8d2w\") pod \"payload-processing-e2e-worker-w3-c1fdd6-76556ddfb-w9t9z\" (UID: \"71cedf55-9f63-4c56-b75f-ab67bb24d556\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-c1fdd6-76556ddfb-w9t9z"
Oct 30 19:40:51.785280 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.785214    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-np75j\" (UniqueName: \"kubernetes.io/projected/dc5d56e5-1eef-44d3-97c0-84e9af887161-kube-api-access-np75j\") pod \"payload-pre-processing-e2e-worker-w3-c1fdd6-5d68c879bc-ccgjb\" (UID: \"dc5d56e5-1eef-44d3-97c0-84e9af887161\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-c1fdd6-5d68c879bc-ccgjb"
Oct 30 19:40:51.785373 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.785332    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc5d56e5-1eef-44d3-97c0-84e9af887161-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-c1fdd6-5d68c879bc-ccgjb\" (UID: \"dc5d56e5-1eef-44d3-97c0-84e9af887161\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-c1fdd6-5d68c879bc-ccgjb"
Oct 30 19:40:51.886575 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.886533    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/71cedf55-9f63-4c56-b75f-ab67bb24d556-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-c1fdd6-76556ddfb-w9t9z\" (UID: \"71cedf55-9f63-4c56-b75f-ab67bb24d556\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-c1fdd6-76556ddfb-w9t9z"
Oct 30 19:40:51.886996 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.886600    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8d2w\" (UniqueName: \"kubernetes.io/projected/71cedf55-9f63-4c56-b75f-ab67bb24d556-kube-api-access-v8d2w\") pod \"payload-processing-e2e-worker-w3-c1fdd6-76556ddfb-w9t9z\" (UID: \"71cedf55-9f63-4c56-b75f-ab67bb24d556\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-c1fdd6-76556ddfb-w9t9z"
Oct 30 19:40:51.886996 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.886669    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-np75j\" (UniqueName: \"kubernetes.io/projected/dc5d56e5-1eef-44d3-97c0-84e9af887161-kube-api-access-np75j\") pod \"payload-pre-processing-e2e-worker-w3-c1fdd6-5d68c879bc-ccgjb\" (UID: \"dc5d56e5-1eef-44d3-97c0-84e9af887161\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-c1fdd6-5d68c879bc-ccgjb"
Oct 30 19:40:51.886996 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.886826    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc5d56e5-1eef-44d3-97c0-84e9af887161-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-c1fdd6-5d68c879bc-ccgjb\" (UID: \"dc5d56e5-1eef-44d3-97c0-84e9af887161\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-c1fdd6-5d68c879bc-ccgjb"
Oct 30 19:40:51.887404 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.887377    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/71cedf55-9f63-4c56-b75f-ab67bb24d556-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-c1fdd6-76556ddfb-w9t9z\" (UID: \"71cedf55-9f63-4c56-b75f-ab67bb24d556\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-c1fdd6-76556ddfb-w9t9z"
Oct 30 19:40:51.887839 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.887815    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc5d56e5-1eef-44d3-97c0-84e9af887161-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-c1fdd6-5d68c879bc-ccgjb\" (UID: \"dc5d56e5-1eef-44d3-97c0-84e9af887161\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-c1fdd6-5d68c879bc-ccgjb"
Oct 30 19:40:51.905061 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.905030    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8d2w\" (UniqueName: \"kubernetes.io/projected/71cedf55-9f63-4c56-b75f-ab67bb24d556-kube-api-access-v8d2w\") pod \"payload-processing-e2e-worker-w3-c1fdd6-76556ddfb-w9t9z\" (UID: \"71cedf55-9f63-4c56-b75f-ab67bb24d556\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-c1fdd6-76556ddfb-w9t9z"
Oct 30 19:40:51.907071 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.907041    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-np75j\" (UniqueName: \"kubernetes.io/projected/dc5d56e5-1eef-44d3-97c0-84e9af887161-kube-api-access-np75j\") pod \"payload-pre-processing-e2e-worker-w3-c1fdd6-5d68c879bc-ccgjb\" (UID: \"dc5d56e5-1eef-44d3-97c0-84e9af887161\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-c1fdd6-5d68c879bc-ccgjb"
Oct 30 19:40:51.958185 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.958139    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-c1fdd6-76556ddfb-w9t9z"
Oct 30 19:40:51.968293 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.968250    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw"]
Oct 30 19:40:51.968631 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:51.968575    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw" podUID="b3a80a33-44df-464f-bdf1-5f9804cb29c5" containerName="istio-proxy" containerID="cri-o://4189e74b6b2a3e4b7583b1dfaf4dce14ea23b260a5d376ab9bc58fc08bd1a2ff" gracePeriod=30
Oct 30 19:40:52.029428 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:52.029397    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-c1fdd6-5d68c879bc-ccgjb"
Oct 30 19:40:52.071638 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:52.071575    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2"]
Oct 30 19:40:52.096003 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:52.095950    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:40:52.096150 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:52.096058    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:40:52.096150 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:52.096115    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:40:52.249435 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:52.249318    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-c1fdd6-76556ddfb-w9t9z"]
Oct 30 19:40:52.253184 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:40:52.253149    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71cedf55_9f63_4c56_b75f_ab67bb24d556.slice/crio-df391d0beb5fdcce9de92a7b4812e0f7986dd7404f3702d84bd34492e06741db WatchSource:0}: Error finding container df391d0beb5fdcce9de92a7b4812e0f7986dd7404f3702d84bd34492e06741db: Status 404 returned error can't find the container with id df391d0beb5fdcce9de92a7b4812e0f7986dd7404f3702d84bd34492e06741db
Oct 30 19:40:52.339534 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:52.339500    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-c1fdd6-5d68c879bc-ccgjb"]
Oct 30 19:40:52.344337 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:40:52.344308    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc5d56e5_1eef_44d3_97c0_84e9af887161.slice/crio-e94a528c11be2a42d16f1bfd4cb7ee1b2391a429a728a698da4c951360767817 WatchSource:0}: Error finding container e94a528c11be2a42d16f1bfd4cb7ee1b2391a429a728a698da4c951360767817: Status 404 returned error can't find the container with id e94a528c11be2a42d16f1bfd4cb7ee1b2391a429a728a698da4c951360767817
Oct 30 19:40:52.805713 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:52.805681    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-c1fdd6-c87447bf7-8r6gk"]
Oct 30 19:40:52.809720 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:52.809696    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-c1fdd6-c87447bf7-8r6gk"
Oct 30 19:40:52.813936 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:52.813913    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w3-c1fdd6-serving-cert\""
Oct 30 19:40:52.822455 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:52.822426    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-c1fdd6-c87447bf7-8r6gk"]
Oct 30 19:40:52.901982 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:52.901944    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/34320df2-abbd-4508-91e0-bad01e9f5c25-metrics-certs\") pod \"maas-api-e2e-worker-w3-c1fdd6-c87447bf7-8r6gk\" (UID: \"34320df2-abbd-4508-91e0-bad01e9f5c25\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-c1fdd6-c87447bf7-8r6gk"
Oct 30 19:40:52.902452 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:52.901989    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdhqz\" (UniqueName: \"kubernetes.io/projected/34320df2-abbd-4508-91e0-bad01e9f5c25-kube-api-access-xdhqz\") pod \"maas-api-e2e-worker-w3-c1fdd6-c87447bf7-8r6gk\" (UID: \"34320df2-abbd-4508-91e0-bad01e9f5c25\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-c1fdd6-c87447bf7-8r6gk"
Oct 30 19:40:52.902633 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:52.902595    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/34320df2-abbd-4508-91e0-bad01e9f5c25-maas-api-tls\") pod \"maas-api-e2e-worker-w3-c1fdd6-c87447bf7-8r6gk\" (UID: \"34320df2-abbd-4508-91e0-bad01e9f5c25\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-c1fdd6-c87447bf7-8r6gk"
Oct 30 19:40:52.902793 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:52.902761    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/34320df2-abbd-4508-91e0-bad01e9f5c25-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-c1fdd6-c87447bf7-8r6gk\" (UID: \"34320df2-abbd-4508-91e0-bad01e9f5c25\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-c1fdd6-c87447bf7-8r6gk"
Oct 30 19:40:53.005196 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:53.004616    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/34320df2-abbd-4508-91e0-bad01e9f5c25-metrics-certs\") pod \"maas-api-e2e-worker-w3-c1fdd6-c87447bf7-8r6gk\" (UID: \"34320df2-abbd-4508-91e0-bad01e9f5c25\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-c1fdd6-c87447bf7-8r6gk"
Oct 30 19:40:53.005196 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:53.004677    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdhqz\" (UniqueName: \"kubernetes.io/projected/34320df2-abbd-4508-91e0-bad01e9f5c25-kube-api-access-xdhqz\") pod \"maas-api-e2e-worker-w3-c1fdd6-c87447bf7-8r6gk\" (UID: \"34320df2-abbd-4508-91e0-bad01e9f5c25\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-c1fdd6-c87447bf7-8r6gk"
Oct 30 19:40:53.005196 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:53.004846    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/34320df2-abbd-4508-91e0-bad01e9f5c25-maas-api-tls\") pod \"maas-api-e2e-worker-w3-c1fdd6-c87447bf7-8r6gk\" (UID: \"34320df2-abbd-4508-91e0-bad01e9f5c25\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-c1fdd6-c87447bf7-8r6gk"
Oct 30 19:40:53.005196 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:53.004894    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/34320df2-abbd-4508-91e0-bad01e9f5c25-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-c1fdd6-c87447bf7-8r6gk\" (UID: \"34320df2-abbd-4508-91e0-bad01e9f5c25\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-c1fdd6-c87447bf7-8r6gk"
Oct 30 19:40:53.005951 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:53.005912    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/34320df2-abbd-4508-91e0-bad01e9f5c25-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-c1fdd6-c87447bf7-8r6gk\" (UID: \"34320df2-abbd-4508-91e0-bad01e9f5c25\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-c1fdd6-c87447bf7-8r6gk"
Oct 30 19:40:53.009418 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:53.009368    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/34320df2-abbd-4508-91e0-bad01e9f5c25-maas-api-tls\") pod \"maas-api-e2e-worker-w3-c1fdd6-c87447bf7-8r6gk\" (UID: \"34320df2-abbd-4508-91e0-bad01e9f5c25\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-c1fdd6-c87447bf7-8r6gk"
Oct 30 19:40:53.009631 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:53.009591    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/34320df2-abbd-4508-91e0-bad01e9f5c25-metrics-certs\") pod \"maas-api-e2e-worker-w3-c1fdd6-c87447bf7-8r6gk\" (UID: \"34320df2-abbd-4508-91e0-bad01e9f5c25\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-c1fdd6-c87447bf7-8r6gk"
Oct 30 19:40:53.027050 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:53.027026    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdhqz\" (UniqueName: \"kubernetes.io/projected/34320df2-abbd-4508-91e0-bad01e9f5c25-kube-api-access-xdhqz\") pod \"maas-api-e2e-worker-w3-c1fdd6-c87447bf7-8r6gk\" (UID: \"34320df2-abbd-4508-91e0-bad01e9f5c25\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-c1fdd6-c87447bf7-8r6gk"
Oct 30 19:40:53.027943 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:53.027909    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-c1fdd6-5d68c879bc-ccgjb" event={"ID":"dc5d56e5-1eef-44d3-97c0-84e9af887161","Type":"ContainerStarted","Data":"0fe16b0c2359fc34bdf49b70910a721d291931704937bbade12905f4a23132d1"}
Oct 30 19:40:53.028048 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:53.027975    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-c1fdd6-5d68c879bc-ccgjb"
Oct 30 19:40:53.028048 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:53.028003    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-c1fdd6-5d68c879bc-ccgjb" event={"ID":"dc5d56e5-1eef-44d3-97c0-84e9af887161","Type":"ContainerStarted","Data":"e94a528c11be2a42d16f1bfd4cb7ee1b2391a429a728a698da4c951360767817"}
Oct 30 19:40:53.029673 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:53.029649    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2" event={"ID":"387db3d3-e192-44ca-b22b-e84f74c32a98","Type":"ContainerStarted","Data":"6c17d0200b74861a4493ca95d965286cc1240d19fcd752d0b07c24015c4442a3"}
Oct 30 19:40:53.029773 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:53.029699    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2" event={"ID":"387db3d3-e192-44ca-b22b-e84f74c32a98","Type":"ContainerStarted","Data":"7d87b1070409a16895469ff2b7cafb1456641100a457cd03db1b4a27a1f52580"}
Oct 30 19:40:53.031222 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:53.031198    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-c1fdd6-76556ddfb-w9t9z" event={"ID":"71cedf55-9f63-4c56-b75f-ab67bb24d556","Type":"ContainerStarted","Data":"3c3409a29bf819acfdd659f6777163d4330cd0d2df4cc59d2965841b5a781cc7"}
Oct 30 19:40:53.031293 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:53.031239    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-c1fdd6-76556ddfb-w9t9z" event={"ID":"71cedf55-9f63-4c56-b75f-ab67bb24d556","Type":"ContainerStarted","Data":"df391d0beb5fdcce9de92a7b4812e0f7986dd7404f3702d84bd34492e06741db"}
Oct 30 19:40:53.031340 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:53.031335    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w3-c1fdd6-76556ddfb-w9t9z"
Oct 30 19:40:53.061602 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:53.061506    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-c1fdd6-5d68c879bc-ccgjb" podStartSLOduration=2.061475692 podStartE2EDuration="2.061475692s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:40:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:40:53.060047366 +0000 UTC m=+528.388165511" watchObservedRunningTime="2026-10-30 19:40:53.061475692 +0000 UTC m=+528.389593846"
Oct 30 19:40:53.126042 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:53.126012    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-c1fdd6-c87447bf7-8r6gk"
Oct 30 19:40:53.132841 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:53.132797    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w3-c1fdd6-76556ddfb-w9t9z" podStartSLOduration=2.132782293 podStartE2EDuration="2.132782293s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:40:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:40:53.098165837 +0000 UTC m=+528.426283992" watchObservedRunningTime="2026-10-30 19:40:53.132782293 +0000 UTC m=+528.460900446"
Oct 30 19:40:53.133623 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:53.133588    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2" podStartSLOduration=2.133577754 podStartE2EDuration="2.133577754s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:40:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:40:53.131733912 +0000 UTC m=+528.459852067" watchObservedRunningTime="2026-10-30 19:40:53.133577754 +0000 UTC m=+528.461696005"
Oct 30 19:40:53.425299 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:53.425267    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-c1fdd6-c87447bf7-8r6gk"]
Oct 30 19:40:53.426789 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:40:53.426761    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34320df2_abbd_4508_91e0_bad01e9f5c25.slice/crio-838273cc8dca0cc8e9b1645cf829cff31f329e67008b23fcd450b00c9811fb40 WatchSource:0}: Error finding container 838273cc8dca0cc8e9b1645cf829cff31f329e67008b23fcd450b00c9811fb40: Status 404 returned error can't find the container with id 838273cc8dca0cc8e9b1645cf829cff31f329e67008b23fcd450b00c9811fb40
Oct 30 19:40:53.784369 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:53.784289    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2"
Oct 30 19:40:53.789568 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:53.789542    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2"
Oct 30 19:40:54.039293 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:54.039201    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-c1fdd6-c87447bf7-8r6gk" event={"ID":"34320df2-abbd-4508-91e0-bad01e9f5c25","Type":"ContainerStarted","Data":"054931ed9d39b08455ddf923d7ba89892bab8bb4ec8297c8e0b162c2b98e153b"}
Oct 30 19:40:54.039293 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:54.039259    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-c1fdd6-c87447bf7-8r6gk" event={"ID":"34320df2-abbd-4508-91e0-bad01e9f5c25","Type":"ContainerStarted","Data":"838273cc8dca0cc8e9b1645cf829cff31f329e67008b23fcd450b00c9811fb40"}
Oct 30 19:40:54.039915 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:54.039892    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2"
Oct 30 19:40:54.040419 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:54.040404    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2"
Oct 30 19:40:54.061436 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:54.061388    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-c1fdd6-c87447bf7-8r6gk" podStartSLOduration=1.664263227 podStartE2EDuration="2.06137611s" totalImagesPullingTime="397.112883ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:40:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:40:54.060717506 +0000 UTC m=+529.388835673" watchObservedRunningTime="2026-10-30 19:40:54.06137611 +0000 UTC m=+529.389494305"
Oct 30 19:40:55.063011 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:55.062964    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-c1fdd6-c87447bf7-8r6gk"
Oct 30 19:40:55.865505 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:55.865459    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-d05eeacd-596f4f9cb6-tkmg2"]
Oct 30 19:40:55.884015 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:55.883976    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-d05eeacd-6bb9664cb7-lmrkw"]
Oct 30 19:40:56.393433 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:56.393390    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 19:40:57.116042 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:57.116005    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-f06af6-77784df88-qg6p6"]
Oct 30 19:40:57.120989 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:57.120958    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-f06af6-77784df88-qg6p6"
Oct 30 19:40:57.138396 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:57.138366    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-f06af6-77784df88-qg6p6"]
Oct 30 19:40:57.162023 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:57.161992    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0451a825-071c-4214-b5ea-800baa964cde-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-f06af6-77784df88-qg6p6\" (UID: \"0451a825-071c-4214-b5ea-800baa964cde\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-f06af6-77784df88-qg6p6"
Oct 30 19:40:57.162189 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:57.162168    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rpc8\" (UniqueName: \"kubernetes.io/projected/0451a825-071c-4214-b5ea-800baa964cde-kube-api-access-2rpc8\") pod \"payload-processing-e2e-worker-w2-f06af6-77784df88-qg6p6\" (UID: \"0451a825-071c-4214-b5ea-800baa964cde\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-f06af6-77784df88-qg6p6"
Oct 30 19:40:57.214635 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:57.213887    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w2-f06af6-dockercfg-j854r\""
Oct 30 19:40:57.214635 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:57.213986    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w2-f06af6\""
Oct 30 19:40:57.249303 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:57.249277    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw"
Oct 30 19:40:57.262938 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:57.262901    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b3a80a33-44df-464f-bdf1-5f9804cb29c5-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b3a80a33-44df-464f-bdf1-5f9804cb29c5-istio-data\") pod \"b3a80a33-44df-464f-bdf1-5f9804cb29c5\" (UID: \"b3a80a33-44df-464f-bdf1-5f9804cb29c5\") "
Oct 30 19:40:57.263093 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:57.262999    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b3a80a33-44df-464f-bdf1-5f9804cb29c5-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3a80a33-44df-464f-bdf1-5f9804cb29c5-workload-socket\") pod \"b3a80a33-44df-464f-bdf1-5f9804cb29c5\" (UID: \"b3a80a33-44df-464f-bdf1-5f9804cb29c5\") "
Oct 30 19:40:57.263093 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:57.263033    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b3a80a33-44df-464f-bdf1-5f9804cb29c5-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b3a80a33-44df-464f-bdf1-5f9804cb29c5-istiod-ca-cert\") pod \"b3a80a33-44df-464f-bdf1-5f9804cb29c5\" (UID: \"b3a80a33-44df-464f-bdf1-5f9804cb29c5\") "
Oct 30 19:40:57.263093 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:57.263081    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b3a80a33-44df-464f-bdf1-5f9804cb29c5-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3a80a33-44df-464f-bdf1-5f9804cb29c5-workload-certs\") pod \"b3a80a33-44df-464f-bdf1-5f9804cb29c5\" (UID: \"b3a80a33-44df-464f-bdf1-5f9804cb29c5\") "
Oct 30 19:40:57.263339 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:57.263103    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b3a80a33-44df-464f-bdf1-5f9804cb29c5-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b3a80a33-44df-464f-bdf1-5f9804cb29c5-istio-podinfo\") pod \"b3a80a33-44df-464f-bdf1-5f9804cb29c5\" (UID: \"b3a80a33-44df-464f-bdf1-5f9804cb29c5\") "
Oct 30 19:40:57.263339 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:57.263150    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b3a80a33-44df-464f-bdf1-5f9804cb29c5-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b3a80a33-44df-464f-bdf1-5f9804cb29c5-istio-envoy\") pod \"b3a80a33-44df-464f-bdf1-5f9804cb29c5\" (UID: \"b3a80a33-44df-464f-bdf1-5f9804cb29c5\") "
Oct 30 19:40:57.263339 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:57.263174    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3a80a33-44df-464f-bdf1-5f9804cb29c5-istio-data" pod "b3a80a33-44df-464f-bdf1-5f9804cb29c5" (UID: "b3a80a33-44df-464f-bdf1-5f9804cb29c5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:40:57.263339 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:57.263208    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b3a80a33-44df-464f-bdf1-5f9804cb29c5-kube-api-access-t74qd\" (UniqueName: \"kubernetes.io/projected/b3a80a33-44df-464f-bdf1-5f9804cb29c5-kube-api-access-t74qd\") pod \"b3a80a33-44df-464f-bdf1-5f9804cb29c5\" (UID: \"b3a80a33-44df-464f-bdf1-5f9804cb29c5\") "
Oct 30 19:40:57.263339 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:57.263267    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3a80a33-44df-464f-bdf1-5f9804cb29c5-workload-socket" pod "b3a80a33-44df-464f-bdf1-5f9804cb29c5" (UID: "b3a80a33-44df-464f-bdf1-5f9804cb29c5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:40:57.263339 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:57.263294    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b3a80a33-44df-464f-bdf1-5f9804cb29c5-istio-token\" (UniqueName: \"kubernetes.io/projected/b3a80a33-44df-464f-bdf1-5f9804cb29c5-istio-token\") pod \"b3a80a33-44df-464f-bdf1-5f9804cb29c5\" (UID: \"b3a80a33-44df-464f-bdf1-5f9804cb29c5\") "
Oct 30 19:40:57.263339 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:57.263342    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b3a80a33-44df-464f-bdf1-5f9804cb29c5-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3a80a33-44df-464f-bdf1-5f9804cb29c5-credential-socket\") pod \"b3a80a33-44df-464f-bdf1-5f9804cb29c5\" (UID: \"b3a80a33-44df-464f-bdf1-5f9804cb29c5\") "
Oct 30 19:40:57.264447 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:57.263372    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3a80a33-44df-464f-bdf1-5f9804cb29c5-workload-certs" pod "b3a80a33-44df-464f-bdf1-5f9804cb29c5" (UID: "b3a80a33-44df-464f-bdf1-5f9804cb29c5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:40:57.264447 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:57.263531    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3a80a33-44df-464f-bdf1-5f9804cb29c5-istiod-ca-cert" pod "b3a80a33-44df-464f-bdf1-5f9804cb29c5" (UID: "b3a80a33-44df-464f-bdf1-5f9804cb29c5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:40:57.264447 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:57.263687    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3a80a33-44df-464f-bdf1-5f9804cb29c5-credential-socket" pod "b3a80a33-44df-464f-bdf1-5f9804cb29c5" (UID: "b3a80a33-44df-464f-bdf1-5f9804cb29c5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:40:57.264447 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:57.263732    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rpc8\" (UniqueName: \"kubernetes.io/projected/0451a825-071c-4214-b5ea-800baa964cde-kube-api-access-2rpc8\") pod \"payload-processing-e2e-worker-w2-f06af6-77784df88-qg6p6\" (UID: \"0451a825-071c-4214-b5ea-800baa964cde\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-f06af6-77784df88-qg6p6"
Oct 30 19:40:57.264447 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:57.263912    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0451a825-071c-4214-b5ea-800baa964cde-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-f06af6-77784df88-qg6p6\" (UID: \"0451a825-071c-4214-b5ea-800baa964cde\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-f06af6-77784df88-qg6p6"
Oct 30 19:40:57.264447 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:57.264052    6996 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3a80a33-44df-464f-bdf1-5f9804cb29c5-credential-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:40:57.264447 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:57.264070    6996 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b3a80a33-44df-464f-bdf1-5f9804cb29c5-istio-data\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:40:57.264447 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:57.264086    6996 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3a80a33-44df-464f-bdf1-5f9804cb29c5-workload-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:40:57.264447 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:57.264101    6996 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b3a80a33-44df-464f-bdf1-5f9804cb29c5-istiod-ca-cert\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:40:57.264447 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:57.264117    6996 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3a80a33-44df-464f-bdf1-5f9804cb29c5-workload-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:40:57.264966 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:57.264629    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0451a825-071c-4214-b5ea-800baa964cde-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-f06af6-77784df88-qg6p6\" (UID: \"0451a825-071c-4214-b5ea-800baa964cde\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-f06af6-77784df88-qg6p6"
Oct 30 19:40:57.265535 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:57.265497    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b3a80a33-44df-464f-bdf1-5f9804cb29c5-istio-podinfo" pod "b3a80a33-44df-464f-bdf1-5f9804cb29c5" (UID: "b3a80a33-44df-464f-bdf1-5f9804cb29c5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:40:57.265669 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:57.265653    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3a80a33-44df-464f-bdf1-5f9804cb29c5-istio-envoy" pod "b3a80a33-44df-464f-bdf1-5f9804cb29c5" (UID: "b3a80a33-44df-464f-bdf1-5f9804cb29c5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:40:57.265845 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:57.265825    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3a80a33-44df-464f-bdf1-5f9804cb29c5-kube-api-access-t74qd" pod "b3a80a33-44df-464f-bdf1-5f9804cb29c5" (UID: "b3a80a33-44df-464f-bdf1-5f9804cb29c5"). InnerVolumeSpecName "kube-api-access-t74qd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:40:57.265845 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:57.265838    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3a80a33-44df-464f-bdf1-5f9804cb29c5-istio-token" pod "b3a80a33-44df-464f-bdf1-5f9804cb29c5" (UID: "b3a80a33-44df-464f-bdf1-5f9804cb29c5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:40:57.277332 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:57.277303    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rpc8\" (UniqueName: \"kubernetes.io/projected/0451a825-071c-4214-b5ea-800baa964cde-kube-api-access-2rpc8\") pod \"payload-processing-e2e-worker-w2-f06af6-77784df88-qg6p6\" (UID: \"0451a825-071c-4214-b5ea-800baa964cde\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-f06af6-77784df88-qg6p6"
Oct 30 19:40:57.365250 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:57.365214    6996 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b3a80a33-44df-464f-bdf1-5f9804cb29c5-istio-podinfo\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:40:57.365250 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:57.365244    6996 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b3a80a33-44df-464f-bdf1-5f9804cb29c5-istio-envoy\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:40:57.365250 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:57.365255    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t74qd\" (UniqueName: \"kubernetes.io/projected/b3a80a33-44df-464f-bdf1-5f9804cb29c5-kube-api-access-t74qd\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:40:57.365582 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:57.365266    6996 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3a80a33-44df-464f-bdf1-5f9804cb29c5-istio-token\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:40:57.438075 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:57.438044    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-f06af6-77784df88-qg6p6"
Oct 30 19:40:57.727698 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:57.727672    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-f06af6-77784df88-qg6p6"]
Oct 30 19:40:57.729805 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:40:57.729779    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0451a825_071c_4214_b5ea_800baa964cde.slice/crio-62f624a03dbac977429655b69f9fbbc0a4a2109e23358cf2e6218183c50c8194 WatchSource:0}: Error finding container 62f624a03dbac977429655b69f9fbbc0a4a2109e23358cf2e6218183c50c8194: Status 404 returned error can't find the container with id 62f624a03dbac977429655b69f9fbbc0a4a2109e23358cf2e6218183c50c8194
Oct 30 19:40:58.005995 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:58.005914    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-f06af6-7f99b6c86c-tdw5h"]
Oct 30 19:40:58.007397 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:58.007364    6996 cpu_manager.go:414] "containerMap: removing container" podUID="b3a80a33-44df-464f-bdf1-5f9804cb29c5" containerName="istio-proxy"
Oct 30 19:40:58.007397 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:58.007394    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b3a80a33-44df-464f-bdf1-5f9804cb29c5" containerName="istio-proxy"
Oct 30 19:40:58.007660 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:58.007425    6996 cpu_manager.go:414] "containerMap: removing container" podUID="b3a80a33-44df-464f-bdf1-5f9804cb29c5" containerName="istio-proxy"
Oct 30 19:40:58.007660 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:58.007433    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b3a80a33-44df-464f-bdf1-5f9804cb29c5" containerName="istio-proxy"
Oct 30 19:40:58.007660 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:58.007655    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="b3a80a33-44df-464f-bdf1-5f9804cb29c5" containerName="istio-proxy"
Oct 30 19:40:58.007819 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:58.007678    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="b3a80a33-44df-464f-bdf1-5f9804cb29c5" containerName="istio-proxy"
Oct 30 19:40:58.013883 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:58.013853    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-f06af6-7f99b6c86c-tdw5h"
Oct 30 19:40:58.019845 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:58.019822    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w2-f06af6-serving-cert\""
Oct 30 19:40:58.021231 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:58.021207    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-f06af6-7f99b6c86c-tdw5h"]
Oct 30 19:40:58.073841 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:58.073797    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqj78\" (UniqueName: \"kubernetes.io/projected/6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a-kube-api-access-lqj78\") pod \"maas-api-e2e-worker-w2-f06af6-7f99b6c86c-tdw5h\" (UID: \"6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-f06af6-7f99b6c86c-tdw5h"
Oct 30 19:40:58.074015 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:58.073864    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-f06af6-7f99b6c86c-tdw5h\" (UID: \"6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-f06af6-7f99b6c86c-tdw5h"
Oct 30 19:40:58.074015 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:58.074003    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a-maas-api-tls\") pod \"maas-api-e2e-worker-w2-f06af6-7f99b6c86c-tdw5h\" (UID: \"6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-f06af6-7f99b6c86c-tdw5h"
Oct 30 19:40:58.074134 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:58.074055    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a-metrics-certs\") pod \"maas-api-e2e-worker-w2-f06af6-7f99b6c86c-tdw5h\" (UID: \"6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-f06af6-7f99b6c86c-tdw5h"
Oct 30 19:40:58.082439 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:58.082412    6996 generic.go:356] "Generic (PLEG): container finished" podID="b3a80a33-44df-464f-bdf1-5f9804cb29c5" containerID="4189e74b6b2a3e4b7583b1dfaf4dce14ea23b260a5d376ab9bc58fc08bd1a2ff" exitCode=0
Oct 30 19:40:58.082585 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:58.082490    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw"
Oct 30 19:40:58.082585 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:58.082519    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw" event={"ID":"b3a80a33-44df-464f-bdf1-5f9804cb29c5","Type":"ContainerDied","Data":"4189e74b6b2a3e4b7583b1dfaf4dce14ea23b260a5d376ab9bc58fc08bd1a2ff"}
Oct 30 19:40:58.082585 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:58.082582    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw" event={"ID":"b3a80a33-44df-464f-bdf1-5f9804cb29c5","Type":"ContainerDied","Data":"4825c7c95e566c0f52a2d978ac810bbbdfcf5ec4e6b7e1db7ed8e0f1cc611df8"}
Oct 30 19:40:58.082761 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:58.082612    6996 scope.go:122] "RemoveContainer" containerID="4189e74b6b2a3e4b7583b1dfaf4dce14ea23b260a5d376ab9bc58fc08bd1a2ff"
Oct 30 19:40:58.084307 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:58.084270    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-f06af6-77784df88-qg6p6" event={"ID":"0451a825-071c-4214-b5ea-800baa964cde","Type":"ContainerStarted","Data":"03553bdb3ed85902eb05ef420ac75c54d97c5785e6f2bc3a1bf95eafccd8e97b"}
Oct 30 19:40:58.084401 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:58.084315    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-f06af6-77784df88-qg6p6" event={"ID":"0451a825-071c-4214-b5ea-800baa964cde","Type":"ContainerStarted","Data":"62f624a03dbac977429655b69f9fbbc0a4a2109e23358cf2e6218183c50c8194"}
Oct 30 19:40:58.084612 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:58.084587    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w2-f06af6-77784df88-qg6p6"
Oct 30 19:40:58.093651 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:58.093633    6996 scope.go:122] "RemoveContainer" containerID="fab12fe15a32f1bf9e890dc0309116c75de5dbeae9a82076cb670207e5afab16"
Oct 30 19:40:58.103019 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:58.103002    6996 scope.go:122] "RemoveContainer" containerID="4189e74b6b2a3e4b7583b1dfaf4dce14ea23b260a5d376ab9bc58fc08bd1a2ff"
Oct 30 19:40:58.103297 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:40:58.103277    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4189e74b6b2a3e4b7583b1dfaf4dce14ea23b260a5d376ab9bc58fc08bd1a2ff\": container with ID starting with 4189e74b6b2a3e4b7583b1dfaf4dce14ea23b260a5d376ab9bc58fc08bd1a2ff not found: ID does not exist" containerID="4189e74b6b2a3e4b7583b1dfaf4dce14ea23b260a5d376ab9bc58fc08bd1a2ff"
Oct 30 19:40:58.103341 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:58.103306    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4189e74b6b2a3e4b7583b1dfaf4dce14ea23b260a5d376ab9bc58fc08bd1a2ff"} err="failed to get container status \"4189e74b6b2a3e4b7583b1dfaf4dce14ea23b260a5d376ab9bc58fc08bd1a2ff\": rpc error: code = NotFound desc = could not find container \"4189e74b6b2a3e4b7583b1dfaf4dce14ea23b260a5d376ab9bc58fc08bd1a2ff\": container with ID starting with 4189e74b6b2a3e4b7583b1dfaf4dce14ea23b260a5d376ab9bc58fc08bd1a2ff not found: ID does not exist"
Oct 30 19:40:58.103341 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:58.103323    6996 scope.go:122] "RemoveContainer" containerID="fab12fe15a32f1bf9e890dc0309116c75de5dbeae9a82076cb670207e5afab16"
Oct 30 19:40:58.103594 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:40:58.103571    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fab12fe15a32f1bf9e890dc0309116c75de5dbeae9a82076cb670207e5afab16\": container with ID starting with fab12fe15a32f1bf9e890dc0309116c75de5dbeae9a82076cb670207e5afab16 not found: ID does not exist" containerID="fab12fe15a32f1bf9e890dc0309116c75de5dbeae9a82076cb670207e5afab16"
Oct 30 19:40:58.103638 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:58.103605    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fab12fe15a32f1bf9e890dc0309116c75de5dbeae9a82076cb670207e5afab16"} err="failed to get container status \"fab12fe15a32f1bf9e890dc0309116c75de5dbeae9a82076cb670207e5afab16\": rpc error: code = NotFound desc = could not find container \"fab12fe15a32f1bf9e890dc0309116c75de5dbeae9a82076cb670207e5afab16\": container with ID starting with fab12fe15a32f1bf9e890dc0309116c75de5dbeae9a82076cb670207e5afab16 not found: ID does not exist"
Oct 30 19:40:58.114835 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:58.114796    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w2-f06af6-77784df88-qg6p6" podStartSLOduration=1.114784009 podStartE2EDuration="1.114784009s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:40:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:40:58.111978187 +0000 UTC m=+533.440096353" watchObservedRunningTime="2026-10-30 19:40:58.114784009 +0000 UTC m=+533.442902197"
Oct 30 19:40:58.139572 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:58.139543    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw"]
Oct 30 19:40:58.140745 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:58.140725    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-b33c04be-openshift-default-7b5d78d77-q29rw"]
Oct 30 19:40:58.175361 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:58.175334    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a-maas-api-tls\") pod \"maas-api-e2e-worker-w2-f06af6-7f99b6c86c-tdw5h\" (UID: \"6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-f06af6-7f99b6c86c-tdw5h"
Oct 30 19:40:58.175543 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:58.175422    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a-metrics-certs\") pod \"maas-api-e2e-worker-w2-f06af6-7f99b6c86c-tdw5h\" (UID: \"6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-f06af6-7f99b6c86c-tdw5h"
Oct 30 19:40:58.175543 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:40:58.175521    6996 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w2-f06af6-serving-cert: secret "maas-api-e2e-worker-w2-f06af6-serving-cert" not found
Oct 30 19:40:58.175659 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:58.175553    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqj78\" (UniqueName: \"kubernetes.io/projected/6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a-kube-api-access-lqj78\") pod \"maas-api-e2e-worker-w2-f06af6-7f99b6c86c-tdw5h\" (UID: \"6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-f06af6-7f99b6c86c-tdw5h"
Oct 30 19:40:58.175659 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:40:58.175595    6996 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a-maas-api-tls podName:6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a nodeName:}" failed. No retries permitted until 2026-10-30 19:40:58.675574633 +0000 UTC m=+534.003692797 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a-maas-api-tls") pod "maas-api-e2e-worker-w2-f06af6-7f99b6c86c-tdw5h" (UID: "6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a") : secret "maas-api-e2e-worker-w2-f06af6-serving-cert" not found
Oct 30 19:40:58.175776 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:58.175685    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-f06af6-7f99b6c86c-tdw5h\" (UID: \"6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-f06af6-7f99b6c86c-tdw5h"
Oct 30 19:40:58.176850 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:58.176827    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-f06af6-7f99b6c86c-tdw5h\" (UID: \"6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-f06af6-7f99b6c86c-tdw5h"
Oct 30 19:40:58.178077 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:58.178056    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a-metrics-certs\") pod \"maas-api-e2e-worker-w2-f06af6-7f99b6c86c-tdw5h\" (UID: \"6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-f06af6-7f99b6c86c-tdw5h"
Oct 30 19:40:58.188643 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:58.188621    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqj78\" (UniqueName: \"kubernetes.io/projected/6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a-kube-api-access-lqj78\") pod \"maas-api-e2e-worker-w2-f06af6-7f99b6c86c-tdw5h\" (UID: \"6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-f06af6-7f99b6c86c-tdw5h"
Oct 30 19:40:58.681078 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:58.681047    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a-maas-api-tls\") pod \"maas-api-e2e-worker-w2-f06af6-7f99b6c86c-tdw5h\" (UID: \"6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-f06af6-7f99b6c86c-tdw5h"
Oct 30 19:40:58.683474 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:58.683451    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a-maas-api-tls\") pod \"maas-api-e2e-worker-w2-f06af6-7f99b6c86c-tdw5h\" (UID: \"6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-f06af6-7f99b6c86c-tdw5h"
Oct 30 19:40:58.795915 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:58.795865    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b3a80a33-44df-464f-bdf1-5f9804cb29c5" path="/var/lib/kubelet/pods/b3a80a33-44df-464f-bdf1-5f9804cb29c5/volumes"
Oct 30 19:40:58.936507 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:58.936417    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-f06af6-7f99b6c86c-tdw5h"
Oct 30 19:40:59.229112 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:40:59.229078    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-f06af6-7f99b6c86c-tdw5h"]
Oct 30 19:40:59.230903 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:40:59.230879    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a0eb5d6_3bc0_4c49_9fd1_50fbc70d716a.slice/crio-8b6ce75b50711c5bc42af0f91abda8932e479c6f9eb9eb4f93f7d6e17977fd73 WatchSource:0}: Error finding container 8b6ce75b50711c5bc42af0f91abda8932e479c6f9eb9eb4f93f7d6e17977fd73: Status 404 returned error can't find the container with id 8b6ce75b50711c5bc42af0f91abda8932e479c6f9eb9eb4f93f7d6e17977fd73
Oct 30 19:41:00.099675 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:00.099637    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-f06af6-7f99b6c86c-tdw5h" event={"ID":"6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a","Type":"ContainerStarted","Data":"4cd55008fc5acfa4051d2e74b8e657826e9c18d5859900ad2df3bfaff7eab51f"}
Oct 30 19:41:00.100054 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:00.099699    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-f06af6-7f99b6c86c-tdw5h" event={"ID":"6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a","Type":"ContainerStarted","Data":"8b6ce75b50711c5bc42af0f91abda8932e479c6f9eb9eb4f93f7d6e17977fd73"}
Oct 30 19:41:00.100054 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:00.099856    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-f06af6-7f99b6c86c-tdw5h"
Oct 30 19:41:00.121439 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:00.121389    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-f06af6-7f99b6c86c-tdw5h" podStartSLOduration=2.702397192 podStartE2EDuration="3.121375965s" totalImagesPullingTime="418.978773ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:40:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:41:00.118753916 +0000 UTC m=+535.446872069" watchObservedRunningTime="2026-10-30 19:41:00.121375965 +0000 UTC m=+535.449494117"
Oct 30 19:41:00.447879 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:00.447792    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-2a3e1b-6db86c8568-g56mm"]
Oct 30 19:41:00.452714 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:00.452679    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-2a3e1b-6db86c8568-g56mm"
Oct 30 19:41:00.466638 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:00.466595    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-2a3e1b-6db86c8568-g56mm"]
Oct 30 19:41:00.500528 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:00.500469    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f32d849b-5292-444c-8369-87385b8d5348-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-2a3e1b-6db86c8568-g56mm\" (UID: \"f32d849b-5292-444c-8369-87385b8d5348\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-2a3e1b-6db86c8568-g56mm"
Oct 30 19:41:00.500699 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:00.500678    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kxkf\" (UniqueName: \"kubernetes.io/projected/f32d849b-5292-444c-8369-87385b8d5348-kube-api-access-2kxkf\") pod \"maas-api-e2e-worker-w4-2a3e1b-6db86c8568-g56mm\" (UID: \"f32d849b-5292-444c-8369-87385b8d5348\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-2a3e1b-6db86c8568-g56mm"
Oct 30 19:41:00.500800 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:00.500721    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f32d849b-5292-444c-8369-87385b8d5348-metrics-certs\") pod \"maas-api-e2e-worker-w4-2a3e1b-6db86c8568-g56mm\" (UID: \"f32d849b-5292-444c-8369-87385b8d5348\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-2a3e1b-6db86c8568-g56mm"
Oct 30 19:41:00.500800 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:00.500758    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f32d849b-5292-444c-8369-87385b8d5348-maas-api-tls\") pod \"maas-api-e2e-worker-w4-2a3e1b-6db86c8568-g56mm\" (UID: \"f32d849b-5292-444c-8369-87385b8d5348\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-2a3e1b-6db86c8568-g56mm"
Oct 30 19:41:00.502223 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:00.502204    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w4-2a3e1b-serving-cert\""
Oct 30 19:41:00.601454 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:00.601418    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f32d849b-5292-444c-8369-87385b8d5348-maas-api-tls\") pod \"maas-api-e2e-worker-w4-2a3e1b-6db86c8568-g56mm\" (UID: \"f32d849b-5292-444c-8369-87385b8d5348\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-2a3e1b-6db86c8568-g56mm"
Oct 30 19:41:00.601628 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:00.601466    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f32d849b-5292-444c-8369-87385b8d5348-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-2a3e1b-6db86c8568-g56mm\" (UID: \"f32d849b-5292-444c-8369-87385b8d5348\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-2a3e1b-6db86c8568-g56mm"
Oct 30 19:41:00.601628 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:00.601581    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kxkf\" (UniqueName: \"kubernetes.io/projected/f32d849b-5292-444c-8369-87385b8d5348-kube-api-access-2kxkf\") pod \"maas-api-e2e-worker-w4-2a3e1b-6db86c8568-g56mm\" (UID: \"f32d849b-5292-444c-8369-87385b8d5348\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-2a3e1b-6db86c8568-g56mm"
Oct 30 19:41:00.601628 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:41:00.601600    6996 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w4-2a3e1b-serving-cert: secret "maas-api-e2e-worker-w4-2a3e1b-serving-cert" not found
Oct 30 19:41:00.601793 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:00.601606    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f32d849b-5292-444c-8369-87385b8d5348-metrics-certs\") pod \"maas-api-e2e-worker-w4-2a3e1b-6db86c8568-g56mm\" (UID: \"f32d849b-5292-444c-8369-87385b8d5348\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-2a3e1b-6db86c8568-g56mm"
Oct 30 19:41:00.601793 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:41:00.601675    6996 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f32d849b-5292-444c-8369-87385b8d5348-maas-api-tls podName:f32d849b-5292-444c-8369-87385b8d5348 nodeName:}" failed. No retries permitted until 2026-10-30 19:41:01.101654288 +0000 UTC m=+536.429772444 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f32d849b-5292-444c-8369-87385b8d5348-maas-api-tls") pod "maas-api-e2e-worker-w4-2a3e1b-6db86c8568-g56mm" (UID: "f32d849b-5292-444c-8369-87385b8d5348") : secret "maas-api-e2e-worker-w4-2a3e1b-serving-cert" not found
Oct 30 19:41:00.602257 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:00.602236    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f32d849b-5292-444c-8369-87385b8d5348-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-2a3e1b-6db86c8568-g56mm\" (UID: \"f32d849b-5292-444c-8369-87385b8d5348\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-2a3e1b-6db86c8568-g56mm"
Oct 30 19:41:00.604127 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:00.604102    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f32d849b-5292-444c-8369-87385b8d5348-metrics-certs\") pod \"maas-api-e2e-worker-w4-2a3e1b-6db86c8568-g56mm\" (UID: \"f32d849b-5292-444c-8369-87385b8d5348\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-2a3e1b-6db86c8568-g56mm"
Oct 30 19:41:00.621532 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:00.621510    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kxkf\" (UniqueName: \"kubernetes.io/projected/f32d849b-5292-444c-8369-87385b8d5348-kube-api-access-2kxkf\") pod \"maas-api-e2e-worker-w4-2a3e1b-6db86c8568-g56mm\" (UID: \"f32d849b-5292-444c-8369-87385b8d5348\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-2a3e1b-6db86c8568-g56mm"
Oct 30 19:41:00.866311 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:00.866253    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-d05eeacd-596f4f9cb6-tkmg2" podUID="0f0dbb94-ef0a-4531-9b45-2bb2dff8d130" containerName="payload-processing" containerID="cri-o://73471f672fe6466ac73ee2014ff8d8e737249ad9a59c60f3559ed9471dfa4547" gracePeriod=25
Oct 30 19:41:00.884869 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:00.884800    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-d05eeacd-6bb9664cb7-lmrkw" podUID="468d7149-0982-44ce-a793-4d3e4cf04719" containerName="payload-pre-processing" containerID="cri-o://f0a333f03e12c50bb413b98cf43a9f579db5873f214dc20e273933b7ff5dbe08" gracePeriod=25
Oct 30 19:41:01.077086 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:01.077051    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-c1fdd6-c87447bf7-8r6gk"
Oct 30 19:41:01.110041 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:01.109398    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f32d849b-5292-444c-8369-87385b8d5348-maas-api-tls\") pod \"maas-api-e2e-worker-w4-2a3e1b-6db86c8568-g56mm\" (UID: \"f32d849b-5292-444c-8369-87385b8d5348\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-2a3e1b-6db86c8568-g56mm"
Oct 30 19:41:01.113297 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:01.113267    6996 generic.go:356] "Generic (PLEG): container finished" podID="468d7149-0982-44ce-a793-4d3e4cf04719" containerID="f0a333f03e12c50bb413b98cf43a9f579db5873f214dc20e273933b7ff5dbe08" exitCode=0
Oct 30 19:41:01.113460 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:01.113439    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-d05eeacd-6bb9664cb7-lmrkw" event={"ID":"468d7149-0982-44ce-a793-4d3e4cf04719","Type":"ContainerDied","Data":"f0a333f03e12c50bb413b98cf43a9f579db5873f214dc20e273933b7ff5dbe08"}
Oct 30 19:41:01.114251 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:01.114224    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f32d849b-5292-444c-8369-87385b8d5348-maas-api-tls\") pod \"maas-api-e2e-worker-w4-2a3e1b-6db86c8568-g56mm\" (UID: \"f32d849b-5292-444c-8369-87385b8d5348\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-2a3e1b-6db86c8568-g56mm"
Oct 30 19:41:01.125677 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:01.124046    6996 generic.go:356] "Generic (PLEG): container finished" podID="0f0dbb94-ef0a-4531-9b45-2bb2dff8d130" containerID="73471f672fe6466ac73ee2014ff8d8e737249ad9a59c60f3559ed9471dfa4547" exitCode=0
Oct 30 19:41:01.126874 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:01.126266    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-d05eeacd-596f4f9cb6-tkmg2" event={"ID":"0f0dbb94-ef0a-4531-9b45-2bb2dff8d130","Type":"ContainerDied","Data":"73471f672fe6466ac73ee2014ff8d8e737249ad9a59c60f3559ed9471dfa4547"}
Oct 30 19:41:01.128588 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:01.128470    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 19:41:01.185582 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:01.183975    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-d05eeacd-596f4f9cb6-tkmg2"
Oct 30 19:41:01.211786 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:01.211014    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0f0dbb94-ef0a-4531-9b45-2bb2dff8d130-kube-api-access-d5lt2\" (UniqueName: \"kubernetes.io/projected/0f0dbb94-ef0a-4531-9b45-2bb2dff8d130-kube-api-access-d5lt2\") pod \"0f0dbb94-ef0a-4531-9b45-2bb2dff8d130\" (UID: \"0f0dbb94-ef0a-4531-9b45-2bb2dff8d130\") "
Oct 30 19:41:01.211949 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:01.211452    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0f0dbb94-ef0a-4531-9b45-2bb2dff8d130-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f0dbb94-ef0a-4531-9b45-2bb2dff8d130-plugins-config-volume\") pod \"0f0dbb94-ef0a-4531-9b45-2bb2dff8d130\" (UID: \"0f0dbb94-ef0a-4531-9b45-2bb2dff8d130\") "
Oct 30 19:41:01.211949 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:01.211827    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f0dbb94-ef0a-4531-9b45-2bb2dff8d130-plugins-config-volume" pod "0f0dbb94-ef0a-4531-9b45-2bb2dff8d130" (UID: "0f0dbb94-ef0a-4531-9b45-2bb2dff8d130"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:41:01.221619 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:41:01.221118    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 19:41:01.226512 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:01.223226    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-d05eeacd-6bb9664cb7-lmrkw"
Oct 30 19:41:01.230835 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:01.230806    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f0dbb94-ef0a-4531-9b45-2bb2dff8d130-kube-api-access-d5lt2" pod "0f0dbb94-ef0a-4531-9b45-2bb2dff8d130" (UID: "0f0dbb94-ef0a-4531-9b45-2bb2dff8d130"). InnerVolumeSpecName "kube-api-access-d5lt2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:41:01.231589 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:01.231570    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d5lt2\" (UniqueName: \"kubernetes.io/projected/0f0dbb94-ef0a-4531-9b45-2bb2dff8d130-kube-api-access-d5lt2\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:41:01.231589 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:01.231589    6996 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f0dbb94-ef0a-4531-9b45-2bb2dff8d130-plugins-config-volume\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:41:01.332405 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:01.332368    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/468d7149-0982-44ce-a793-4d3e4cf04719-kube-api-access-zvzp5\" (UniqueName: \"kubernetes.io/projected/468d7149-0982-44ce-a793-4d3e4cf04719-kube-api-access-zvzp5\") pod \"468d7149-0982-44ce-a793-4d3e4cf04719\" (UID: \"468d7149-0982-44ce-a793-4d3e4cf04719\") "
Oct 30 19:41:01.332595 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:01.332536    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/468d7149-0982-44ce-a793-4d3e4cf04719-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/468d7149-0982-44ce-a793-4d3e4cf04719-plugins-config-volume\") pod \"468d7149-0982-44ce-a793-4d3e4cf04719\" (UID: \"468d7149-0982-44ce-a793-4d3e4cf04719\") "
Oct 30 19:41:01.332931 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:01.332902    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/468d7149-0982-44ce-a793-4d3e4cf04719-plugins-config-volume" pod "468d7149-0982-44ce-a793-4d3e4cf04719" (UID: "468d7149-0982-44ce-a793-4d3e4cf04719"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:41:01.335225 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:01.335201    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/468d7149-0982-44ce-a793-4d3e4cf04719-kube-api-access-zvzp5" pod "468d7149-0982-44ce-a793-4d3e4cf04719" (UID: "468d7149-0982-44ce-a793-4d3e4cf04719"). InnerVolumeSpecName "kube-api-access-zvzp5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:41:01.372073 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:01.372038    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-2a3e1b-6db86c8568-g56mm"
Oct 30 19:41:01.435870 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:01.434453    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zvzp5\" (UniqueName: \"kubernetes.io/projected/468d7149-0982-44ce-a793-4d3e4cf04719-kube-api-access-zvzp5\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:41:01.435870 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:01.434515    6996 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/468d7149-0982-44ce-a793-4d3e4cf04719-plugins-config-volume\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:41:01.684282 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:01.684252    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-2a3e1b-6db86c8568-g56mm"]
Oct 30 19:41:01.686702 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:41:01.686670    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf32d849b_5292_444c_8369_87385b8d5348.slice/crio-417812bf77ba1cd84d4f356b0b74731e7ccc617891081515358277e82e0ce4f9 WatchSource:0}: Error finding container 417812bf77ba1cd84d4f356b0b74731e7ccc617891081515358277e82e0ce4f9: Status 404 returned error can't find the container with id 417812bf77ba1cd84d4f356b0b74731e7ccc617891081515358277e82e0ce4f9
Oct 30 19:41:02.133613 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:02.133580    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-d05eeacd-6bb9664cb7-lmrkw"
Oct 30 19:41:02.133613 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:02.133601    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-d05eeacd-6bb9664cb7-lmrkw" event={"ID":"468d7149-0982-44ce-a793-4d3e4cf04719","Type":"ContainerDied","Data":"a0a3426c3c8782a98906ff0aed334091e21102b06c45e3302a7d4fb2f03f00fc"}
Oct 30 19:41:02.134105 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:02.133664    6996 scope.go:122] "RemoveContainer" containerID="f0a333f03e12c50bb413b98cf43a9f579db5873f214dc20e273933b7ff5dbe08"
Oct 30 19:41:02.135095 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:02.135062    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-2a3e1b-6db86c8568-g56mm" event={"ID":"f32d849b-5292-444c-8369-87385b8d5348","Type":"ContainerStarted","Data":"417812bf77ba1cd84d4f356b0b74731e7ccc617891081515358277e82e0ce4f9"}
Oct 30 19:41:02.136441 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:02.136412    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-d05eeacd-596f4f9cb6-tkmg2" event={"ID":"0f0dbb94-ef0a-4531-9b45-2bb2dff8d130","Type":"ContainerDied","Data":"bce8a157eff3bf25e7cf4478275bb6a46706fb8d2b02eb36ff4eec8aec7e9211"}
Oct 30 19:41:02.136441 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:02.136434    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-d05eeacd-596f4f9cb6-tkmg2"
Oct 30 19:41:02.180318 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:02.180288    6996 scope.go:122] "RemoveContainer" containerID="73471f672fe6466ac73ee2014ff8d8e737249ad9a59c60f3559ed9471dfa4547"
Oct 30 19:41:02.205387 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:02.205345    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-d05eeacd-596f4f9cb6-tkmg2"]
Oct 30 19:41:02.214395 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:02.214364    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-d05eeacd-596f4f9cb6-tkmg2"]
Oct 30 19:41:02.234831 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:02.234798    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-d05eeacd-6bb9664cb7-lmrkw"]
Oct 30 19:41:02.259512 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:02.237321    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-d05eeacd-6bb9664cb7-lmrkw"]
Oct 30 19:41:02.791012 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:02.790964    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0f0dbb94-ef0a-4531-9b45-2bb2dff8d130" path="/var/lib/kubelet/pods/0f0dbb94-ef0a-4531-9b45-2bb2dff8d130/volumes"
Oct 30 19:41:02.791471 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:02.791449    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="468d7149-0982-44ce-a793-4d3e4cf04719" path="/var/lib/kubelet/pods/468d7149-0982-44ce-a793-4d3e4cf04719/volumes"
Oct 30 19:41:03.169456 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:03.169421    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-2a3e1b-6db86c8568-g56mm" event={"ID":"f32d849b-5292-444c-8369-87385b8d5348","Type":"ContainerStarted","Data":"d25ef271cdd63ee1bc4fa885ca1914ad20d1a36152441ec8529246a8239921ec"}
Oct 30 19:41:03.170044 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:03.170025    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-2a3e1b-6db86c8568-g56mm"
Oct 30 19:41:03.198223 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:03.198156    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-2a3e1b-6db86c8568-g56mm" podStartSLOduration=2.618261098 podStartE2EDuration="3.198134575s" totalImagesPullingTime="579.873477ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:41:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:41:03.197097466 +0000 UTC m=+538.525215642" watchObservedRunningTime="2026-10-30 19:41:03.198134575 +0000 UTC m=+538.526252729"
Oct 30 19:41:04.062617 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:04.058663    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-c1fdd6-5d68c879bc-ccgjb"
Oct 30 19:41:04.062617 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:04.059833    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w3-c1fdd6-76556ddfb-w9t9z"
Oct 30 19:41:05.420168 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:05.420052    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 19:41:06.095336 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:06.094405    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-8836a8-7fbfb4b97d-x9fs9"]
Oct 30 19:41:06.095336 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:06.095300    6996 cpu_manager.go:414] "containerMap: removing container" podUID="468d7149-0982-44ce-a793-4d3e4cf04719" containerName="payload-pre-processing"
Oct 30 19:41:06.095336 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:06.095322    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="468d7149-0982-44ce-a793-4d3e4cf04719" containerName="payload-pre-processing"
Oct 30 19:41:06.095336 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:06.095335    6996 cpu_manager.go:414] "containerMap: removing container" podUID="0f0dbb94-ef0a-4531-9b45-2bb2dff8d130" containerName="payload-processing"
Oct 30 19:41:06.095336 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:06.095342    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0f0dbb94-ef0a-4531-9b45-2bb2dff8d130" containerName="payload-processing"
Oct 30 19:41:06.095797 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:06.095454    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="468d7149-0982-44ce-a793-4d3e4cf04719" containerName="payload-pre-processing"
Oct 30 19:41:06.095797 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:06.095464    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="0f0dbb94-ef0a-4531-9b45-2bb2dff8d130" containerName="payload-processing"
Oct 30 19:41:06.100891 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:06.100865    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8836a8-7fbfb4b97d-x9fs9"
Oct 30 19:41:06.115261 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:06.115234    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w0-8836a8-serving-cert\""
Oct 30 19:41:06.116126 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:06.116098    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-8836a8-7fbfb4b97d-x9fs9"]
Oct 30 19:41:06.133335 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:06.133304    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-f06af6-7f99b6c86c-tdw5h"
Oct 30 19:41:06.230052 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:06.230016    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e538f71a-8196-4fad-a86b-4750900feb8c-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-8836a8-7fbfb4b97d-x9fs9\" (UID: \"e538f71a-8196-4fad-a86b-4750900feb8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8836a8-7fbfb4b97d-x9fs9"
Oct 30 19:41:06.230281 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:06.230257    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjfcg\" (UniqueName: \"kubernetes.io/projected/e538f71a-8196-4fad-a86b-4750900feb8c-kube-api-access-xjfcg\") pod \"maas-api-e2e-worker-w0-8836a8-7fbfb4b97d-x9fs9\" (UID: \"e538f71a-8196-4fad-a86b-4750900feb8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8836a8-7fbfb4b97d-x9fs9"
Oct 30 19:41:06.230393 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:06.230359    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e538f71a-8196-4fad-a86b-4750900feb8c-maas-api-tls\") pod \"maas-api-e2e-worker-w0-8836a8-7fbfb4b97d-x9fs9\" (UID: \"e538f71a-8196-4fad-a86b-4750900feb8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8836a8-7fbfb4b97d-x9fs9"
Oct 30 19:41:06.230529 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:06.230508    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e538f71a-8196-4fad-a86b-4750900feb8c-metrics-certs\") pod \"maas-api-e2e-worker-w0-8836a8-7fbfb4b97d-x9fs9\" (UID: \"e538f71a-8196-4fad-a86b-4750900feb8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8836a8-7fbfb4b97d-x9fs9"
Oct 30 19:41:06.332148 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:06.332110    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e538f71a-8196-4fad-a86b-4750900feb8c-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-8836a8-7fbfb4b97d-x9fs9\" (UID: \"e538f71a-8196-4fad-a86b-4750900feb8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8836a8-7fbfb4b97d-x9fs9"
Oct 30 19:41:06.332148 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:06.332152    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjfcg\" (UniqueName: \"kubernetes.io/projected/e538f71a-8196-4fad-a86b-4750900feb8c-kube-api-access-xjfcg\") pod \"maas-api-e2e-worker-w0-8836a8-7fbfb4b97d-x9fs9\" (UID: \"e538f71a-8196-4fad-a86b-4750900feb8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8836a8-7fbfb4b97d-x9fs9"
Oct 30 19:41:06.332380 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:06.332204    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e538f71a-8196-4fad-a86b-4750900feb8c-maas-api-tls\") pod \"maas-api-e2e-worker-w0-8836a8-7fbfb4b97d-x9fs9\" (UID: \"e538f71a-8196-4fad-a86b-4750900feb8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8836a8-7fbfb4b97d-x9fs9"
Oct 30 19:41:06.332380 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:06.332254    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e538f71a-8196-4fad-a86b-4750900feb8c-metrics-certs\") pod \"maas-api-e2e-worker-w0-8836a8-7fbfb4b97d-x9fs9\" (UID: \"e538f71a-8196-4fad-a86b-4750900feb8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8836a8-7fbfb4b97d-x9fs9"
Oct 30 19:41:06.332931 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:06.332905    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e538f71a-8196-4fad-a86b-4750900feb8c-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-8836a8-7fbfb4b97d-x9fs9\" (UID: \"e538f71a-8196-4fad-a86b-4750900feb8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8836a8-7fbfb4b97d-x9fs9"
Oct 30 19:41:06.334785 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:06.334760    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e538f71a-8196-4fad-a86b-4750900feb8c-maas-api-tls\") pod \"maas-api-e2e-worker-w0-8836a8-7fbfb4b97d-x9fs9\" (UID: \"e538f71a-8196-4fad-a86b-4750900feb8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8836a8-7fbfb4b97d-x9fs9"
Oct 30 19:41:06.334887 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:06.334792    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e538f71a-8196-4fad-a86b-4750900feb8c-metrics-certs\") pod \"maas-api-e2e-worker-w0-8836a8-7fbfb4b97d-x9fs9\" (UID: \"e538f71a-8196-4fad-a86b-4750900feb8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8836a8-7fbfb4b97d-x9fs9"
Oct 30 19:41:06.365413 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:06.365353    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjfcg\" (UniqueName: \"kubernetes.io/projected/e538f71a-8196-4fad-a86b-4750900feb8c-kube-api-access-xjfcg\") pod \"maas-api-e2e-worker-w0-8836a8-7fbfb4b97d-x9fs9\" (UID: \"e538f71a-8196-4fad-a86b-4750900feb8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8836a8-7fbfb4b97d-x9fs9"
Oct 30 19:41:06.416195 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:06.416165    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8836a8-7fbfb4b97d-x9fs9"
Oct 30 19:41:06.734133 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:06.733941    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-8836a8-7fbfb4b97d-x9fs9"]
Oct 30 19:41:07.201002 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:07.200957    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8836a8-7fbfb4b97d-x9fs9" event={"ID":"e538f71a-8196-4fad-a86b-4750900feb8c","Type":"ContainerStarted","Data":"3c213ce7c652e176dccb88562256154852d32b769bb45c52ef5daea655d20c69"}
Oct 30 19:41:08.210384 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:08.210331    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8836a8-7fbfb4b97d-x9fs9" event={"ID":"e538f71a-8196-4fad-a86b-4750900feb8c","Type":"ContainerStarted","Data":"11bc7eb4258b08d007e332da83f9dc24ef86d3a87ae1810e3c7009e862f11217"}
Oct 30 19:41:08.210858 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:08.210525    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8836a8-7fbfb4b97d-x9fs9"
Oct 30 19:41:08.526761 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:08.526640    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8836a8-7fbfb4b97d-x9fs9" podStartSLOduration=2.05666331 podStartE2EDuration="2.526623926s" totalImagesPullingTime="469.960616ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:41:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:41:08.23010379 +0000 UTC m=+543.558221943" watchObservedRunningTime="2026-10-30 19:41:08.526623926 +0000 UTC m=+543.854742079"
Oct 30 19:41:08.530131 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:08.530073    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-7a745c-74784c5fb8-mgg8p"]
Oct 30 19:41:08.534933 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:08.534905    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-7a745c-74784c5fb8-mgg8p"
Oct 30 19:41:08.541324 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:08.541302    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w1-7a745c-serving-cert\""
Oct 30 19:41:08.561732 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:08.561682    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-7a745c-74784c5fb8-mgg8p"]
Oct 30 19:41:08.598327 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:08.598294    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc6b5019-f6ed-44c3-8390-8525a8e75910-maas-api-tls\") pod \"maas-api-e2e-worker-w1-7a745c-74784c5fb8-mgg8p\" (UID: \"bc6b5019-f6ed-44c3-8390-8525a8e75910\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-7a745c-74784c5fb8-mgg8p"
Oct 30 19:41:08.598497 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:08.598348    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc6b5019-f6ed-44c3-8390-8525a8e75910-metrics-certs\") pod \"maas-api-e2e-worker-w1-7a745c-74784c5fb8-mgg8p\" (UID: \"bc6b5019-f6ed-44c3-8390-8525a8e75910\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-7a745c-74784c5fb8-mgg8p"
Oct 30 19:41:08.598497 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:08.598437    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tg6ws\" (UniqueName: \"kubernetes.io/projected/bc6b5019-f6ed-44c3-8390-8525a8e75910-kube-api-access-tg6ws\") pod \"maas-api-e2e-worker-w1-7a745c-74784c5fb8-mgg8p\" (UID: \"bc6b5019-f6ed-44c3-8390-8525a8e75910\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-7a745c-74784c5fb8-mgg8p"
Oct 30 19:41:08.598602 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:08.598548    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc6b5019-f6ed-44c3-8390-8525a8e75910-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-7a745c-74784c5fb8-mgg8p\" (UID: \"bc6b5019-f6ed-44c3-8390-8525a8e75910\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-7a745c-74784c5fb8-mgg8p"
Oct 30 19:41:08.699998 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:08.699951    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc6b5019-f6ed-44c3-8390-8525a8e75910-maas-api-tls\") pod \"maas-api-e2e-worker-w1-7a745c-74784c5fb8-mgg8p\" (UID: \"bc6b5019-f6ed-44c3-8390-8525a8e75910\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-7a745c-74784c5fb8-mgg8p"
Oct 30 19:41:08.700161 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:08.700016    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc6b5019-f6ed-44c3-8390-8525a8e75910-metrics-certs\") pod \"maas-api-e2e-worker-w1-7a745c-74784c5fb8-mgg8p\" (UID: \"bc6b5019-f6ed-44c3-8390-8525a8e75910\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-7a745c-74784c5fb8-mgg8p"
Oct 30 19:41:08.700161 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:08.700056    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tg6ws\" (UniqueName: \"kubernetes.io/projected/bc6b5019-f6ed-44c3-8390-8525a8e75910-kube-api-access-tg6ws\") pod \"maas-api-e2e-worker-w1-7a745c-74784c5fb8-mgg8p\" (UID: \"bc6b5019-f6ed-44c3-8390-8525a8e75910\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-7a745c-74784c5fb8-mgg8p"
Oct 30 19:41:08.700161 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:08.700125    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc6b5019-f6ed-44c3-8390-8525a8e75910-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-7a745c-74784c5fb8-mgg8p\" (UID: \"bc6b5019-f6ed-44c3-8390-8525a8e75910\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-7a745c-74784c5fb8-mgg8p"
Oct 30 19:41:08.701000 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:08.700968    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc6b5019-f6ed-44c3-8390-8525a8e75910-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-7a745c-74784c5fb8-mgg8p\" (UID: \"bc6b5019-f6ed-44c3-8390-8525a8e75910\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-7a745c-74784c5fb8-mgg8p"
Oct 30 19:41:08.702518 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:08.702495    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc6b5019-f6ed-44c3-8390-8525a8e75910-maas-api-tls\") pod \"maas-api-e2e-worker-w1-7a745c-74784c5fb8-mgg8p\" (UID: \"bc6b5019-f6ed-44c3-8390-8525a8e75910\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-7a745c-74784c5fb8-mgg8p"
Oct 30 19:41:08.702648 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:08.702629    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc6b5019-f6ed-44c3-8390-8525a8e75910-metrics-certs\") pod \"maas-api-e2e-worker-w1-7a745c-74784c5fb8-mgg8p\" (UID: \"bc6b5019-f6ed-44c3-8390-8525a8e75910\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-7a745c-74784c5fb8-mgg8p"
Oct 30 19:41:08.714830 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:08.714802    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tg6ws\" (UniqueName: \"kubernetes.io/projected/bc6b5019-f6ed-44c3-8390-8525a8e75910-kube-api-access-tg6ws\") pod \"maas-api-e2e-worker-w1-7a745c-74784c5fb8-mgg8p\" (UID: \"bc6b5019-f6ed-44c3-8390-8525a8e75910\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-7a745c-74784c5fb8-mgg8p"
Oct 30 19:41:08.862682 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:08.862648    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-7a745c-74784c5fb8-mgg8p"
Oct 30 19:41:09.095025 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:09.094984    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w2-f06af6-77784df88-qg6p6"
Oct 30 19:41:09.158017 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:09.157990    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-7a745c-74784c5fb8-mgg8p"]
Oct 30 19:41:09.161435 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:41:09.161399    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc6b5019_f6ed_44c3_8390_8525a8e75910.slice/crio-316bcd70db29ebea51b8a2a2a89ff9fe0358a654047561abe33aa56d7eed45db WatchSource:0}: Error finding container 316bcd70db29ebea51b8a2a2a89ff9fe0358a654047561abe33aa56d7eed45db: Status 404 returned error can't find the container with id 316bcd70db29ebea51b8a2a2a89ff9fe0358a654047561abe33aa56d7eed45db
Oct 30 19:41:09.185886 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:09.185857    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-2a3e1b-6db86c8568-g56mm"
Oct 30 19:41:09.218695 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:09.218653    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-7a745c-74784c5fb8-mgg8p" event={"ID":"bc6b5019-f6ed-44c3-8390-8525a8e75910","Type":"ContainerStarted","Data":"316bcd70db29ebea51b8a2a2a89ff9fe0358a654047561abe33aa56d7eed45db"}
Oct 30 19:41:10.080664 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:10.080617    6996 patch_prober.go:29] interesting pod/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:41:10.080853 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:10.080683    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2" podUID="387db3d3-e192-44ca-b22b-e84f74c32a98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:41:10.228414 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:10.228370    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-7a745c-74784c5fb8-mgg8p" event={"ID":"bc6b5019-f6ed-44c3-8390-8525a8e75910","Type":"ContainerStarted","Data":"a049874ece5fdc772f649a9e9eed6f0c90269bec1920b395e22928b1d42eb155"}
Oct 30 19:41:10.228943 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:10.228627    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-7a745c-74784c5fb8-mgg8p"
Oct 30 19:41:10.249170 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:10.249093    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-7a745c-74784c5fb8-mgg8p" podStartSLOduration=1.809100741 podStartE2EDuration="2.24907227s" totalImagesPullingTime="439.971529ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:41:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:41:10.248231146 +0000 UTC m=+545.576349314" watchObservedRunningTime="2026-10-30 19:41:10.24907227 +0000 UTC m=+545.577190476"
Oct 30 19:41:11.236079 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:11.236044    6996 generic.go:356] "Generic (PLEG): container finished" podID="387db3d3-e192-44ca-b22b-e84f74c32a98" containerID="6c17d0200b74861a4493ca95d965286cc1240d19fcd752d0b07c24015c4442a3" exitCode=0
Oct 30 19:41:11.236685 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:11.236120    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2" event={"ID":"387db3d3-e192-44ca-b22b-e84f74c32a98","Type":"ContainerDied","Data":"6c17d0200b74861a4493ca95d965286cc1240d19fcd752d0b07c24015c4442a3"}
Oct 30 19:41:11.236685 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:11.236656    6996 scope.go:122] "RemoveContainer" containerID="6c17d0200b74861a4493ca95d965286cc1240d19fcd752d0b07c24015c4442a3"
Oct 30 19:41:11.237892 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:11.237863    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:41:11.237980 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:11.237936    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:41:11.237980 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:11.237968    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:41:12.267539 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:12.267471    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2" event={"ID":"387db3d3-e192-44ca-b22b-e84f74c32a98","Type":"ContainerStarted","Data":"a6d46a3b14eae6336de683d86c01a3dd0b0362ecbb1f9ba41fa832f91a513f16"}
Oct 30 19:41:12.784218 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:12.784179    6996 patch_prober.go:29] interesting pod/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Oct 30 19:41:12.784401 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:12.784234    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2" podUID="387db3d3-e192-44ca-b22b-e84f74c32a98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Oct 30 19:41:12.791797 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:12.791773    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2"
Oct 30 19:41:13.784010 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:13.783975    6996 patch_prober.go:29] interesting pod/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Oct 30 19:41:13.784516 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:13.784028    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2" podUID="387db3d3-e192-44ca-b22b-e84f74c32a98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Oct 30 19:41:14.223973 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:14.223940    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8836a8-7fbfb4b97d-x9fs9"
Oct 30 19:41:14.784453 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:14.784407    6996 patch_prober.go:29] interesting pod/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Oct 30 19:41:14.784949 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:14.784509    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2" podUID="387db3d3-e192-44ca-b22b-e84f74c32a98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Oct 30 19:41:14.866608 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:14.866565    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 19:41:15.783791 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:15.783708    6996 patch_prober.go:29] interesting pod/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Oct 30 19:41:15.783791 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:15.783763    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2" podUID="387db3d3-e192-44ca-b22b-e84f74c32a98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Oct 30 19:41:16.264021 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:16.263988    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-7a745c-74784c5fb8-mgg8p"
Oct 30 19:41:16.783786 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:16.783753    6996 patch_prober.go:29] interesting pod/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Oct 30 19:41:16.784054 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:16.784025    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2" podUID="387db3d3-e192-44ca-b22b-e84f74c32a98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Oct 30 19:41:17.789397 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:17.789366    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2"
Oct 30 19:41:17.789814 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:17.789650    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2"
Oct 30 19:41:17.790445 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:17.790430    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2"
Oct 30 19:41:21.166765 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:21.166731    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 19:41:31.190117 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:41:31.190078    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 19:41:31.754175 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:31.754133    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 19:41:42.767318 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:42.767274    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 19:41:46.052159 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:46.052123    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v"]
Oct 30 19:41:46.057120 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:46.057100    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v"
Oct 30 19:41:46.065702 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:46.065682    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-c1fdd6-openshift-default-dockercfg-7mt7d\""
Oct 30 19:41:46.077446 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:46.070131    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v"]
Oct 30 19:41:46.149053 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:46.149016    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b99ab818-3be9-406e-be18-c0ed26c43bea-istiod-ca-cert\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v\" (UID: \"b99ab818-3be9-406e-be18-c0ed26c43bea\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v"
Oct 30 19:41:46.149207 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:46.149068    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b99ab818-3be9-406e-be18-c0ed26c43bea-workload-certs\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v\" (UID: \"b99ab818-3be9-406e-be18-c0ed26c43bea\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v"
Oct 30 19:41:46.149207 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:46.149139    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b99ab818-3be9-406e-be18-c0ed26c43bea-workload-socket\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v\" (UID: \"b99ab818-3be9-406e-be18-c0ed26c43bea\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v"
Oct 30 19:41:46.149286 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:46.149247    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b99ab818-3be9-406e-be18-c0ed26c43bea-istio-token\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v\" (UID: \"b99ab818-3be9-406e-be18-c0ed26c43bea\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v"
Oct 30 19:41:46.149321 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:46.149310    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b99ab818-3be9-406e-be18-c0ed26c43bea-istio-data\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v\" (UID: \"b99ab818-3be9-406e-be18-c0ed26c43bea\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v"
Oct 30 19:41:46.149357 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:46.149328    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b99ab818-3be9-406e-be18-c0ed26c43bea-istio-envoy\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v\" (UID: \"b99ab818-3be9-406e-be18-c0ed26c43bea\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v"
Oct 30 19:41:46.149357 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:46.149344    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b99ab818-3be9-406e-be18-c0ed26c43bea-istio-podinfo\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v\" (UID: \"b99ab818-3be9-406e-be18-c0ed26c43bea\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v"
Oct 30 19:41:46.149473 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:46.149456    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lr5rb\" (UniqueName: \"kubernetes.io/projected/b99ab818-3be9-406e-be18-c0ed26c43bea-kube-api-access-lr5rb\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v\" (UID: \"b99ab818-3be9-406e-be18-c0ed26c43bea\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v"
Oct 30 19:41:46.149540 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:46.149516    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b99ab818-3be9-406e-be18-c0ed26c43bea-credential-socket\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v\" (UID: \"b99ab818-3be9-406e-be18-c0ed26c43bea\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v"
Oct 30 19:41:46.251133 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:46.251097    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b99ab818-3be9-406e-be18-c0ed26c43bea-credential-socket\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v\" (UID: \"b99ab818-3be9-406e-be18-c0ed26c43bea\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v"
Oct 30 19:41:46.251340 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:46.251159    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b99ab818-3be9-406e-be18-c0ed26c43bea-istiod-ca-cert\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v\" (UID: \"b99ab818-3be9-406e-be18-c0ed26c43bea\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v"
Oct 30 19:41:46.251340 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:46.251181    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b99ab818-3be9-406e-be18-c0ed26c43bea-workload-certs\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v\" (UID: \"b99ab818-3be9-406e-be18-c0ed26c43bea\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v"
Oct 30 19:41:46.251340 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:46.251205    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b99ab818-3be9-406e-be18-c0ed26c43bea-workload-socket\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v\" (UID: \"b99ab818-3be9-406e-be18-c0ed26c43bea\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v"
Oct 30 19:41:46.251340 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:46.251260    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b99ab818-3be9-406e-be18-c0ed26c43bea-istio-token\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v\" (UID: \"b99ab818-3be9-406e-be18-c0ed26c43bea\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v"
Oct 30 19:41:46.251340 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:46.251316    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b99ab818-3be9-406e-be18-c0ed26c43bea-istio-data\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v\" (UID: \"b99ab818-3be9-406e-be18-c0ed26c43bea\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v"
Oct 30 19:41:46.251340 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:46.251336    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b99ab818-3be9-406e-be18-c0ed26c43bea-istio-envoy\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v\" (UID: \"b99ab818-3be9-406e-be18-c0ed26c43bea\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v"
Oct 30 19:41:46.251719 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:46.251362    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b99ab818-3be9-406e-be18-c0ed26c43bea-istio-podinfo\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v\" (UID: \"b99ab818-3be9-406e-be18-c0ed26c43bea\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v"
Oct 30 19:41:46.251719 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:46.251391    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lr5rb\" (UniqueName: \"kubernetes.io/projected/b99ab818-3be9-406e-be18-c0ed26c43bea-kube-api-access-lr5rb\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v\" (UID: \"b99ab818-3be9-406e-be18-c0ed26c43bea\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v"
Oct 30 19:41:46.251719 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:46.251613    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b99ab818-3be9-406e-be18-c0ed26c43bea-credential-socket\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v\" (UID: \"b99ab818-3be9-406e-be18-c0ed26c43bea\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v"
Oct 30 19:41:46.251719 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:46.251638    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b99ab818-3be9-406e-be18-c0ed26c43bea-workload-socket\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v\" (UID: \"b99ab818-3be9-406e-be18-c0ed26c43bea\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v"
Oct 30 19:41:46.251931 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:46.251797    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b99ab818-3be9-406e-be18-c0ed26c43bea-istio-data\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v\" (UID: \"b99ab818-3be9-406e-be18-c0ed26c43bea\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v"
Oct 30 19:41:46.251931 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:46.251902    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b99ab818-3be9-406e-be18-c0ed26c43bea-workload-certs\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v\" (UID: \"b99ab818-3be9-406e-be18-c0ed26c43bea\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v"
Oct 30 19:41:46.252076 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:46.252054    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b99ab818-3be9-406e-be18-c0ed26c43bea-istiod-ca-cert\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v\" (UID: \"b99ab818-3be9-406e-be18-c0ed26c43bea\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v"
Oct 30 19:41:46.253686 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:46.253658    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b99ab818-3be9-406e-be18-c0ed26c43bea-istio-envoy\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v\" (UID: \"b99ab818-3be9-406e-be18-c0ed26c43bea\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v"
Oct 30 19:41:46.253940 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:46.253923    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b99ab818-3be9-406e-be18-c0ed26c43bea-istio-podinfo\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v\" (UID: \"b99ab818-3be9-406e-be18-c0ed26c43bea\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v"
Oct 30 19:41:46.263161 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:46.263134    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lr5rb\" (UniqueName: \"kubernetes.io/projected/b99ab818-3be9-406e-be18-c0ed26c43bea-kube-api-access-lr5rb\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v\" (UID: \"b99ab818-3be9-406e-be18-c0ed26c43bea\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v"
Oct 30 19:41:46.263296 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:46.263234    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b99ab818-3be9-406e-be18-c0ed26c43bea-istio-token\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v\" (UID: \"b99ab818-3be9-406e-be18-c0ed26c43bea\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v"
Oct 30 19:41:46.388640 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:46.388607    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v"
Oct 30 19:41:46.684554 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:46.684530    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v"]
Oct 30 19:41:46.686281 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:41:46.686253    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb99ab818_3be9_406e_be18_c0ed26c43bea.slice/crio-98879e1b59c007b8a0972675b2be82eb6dedeea506170f15d35fb69b53b1f298 WatchSource:0}: Error finding container 98879e1b59c007b8a0972675b2be82eb6dedeea506170f15d35fb69b53b1f298: Status 404 returned error can't find the container with id 98879e1b59c007b8a0972675b2be82eb6dedeea506170f15d35fb69b53b1f298
Oct 30 19:41:46.688939 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:46.688894    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:41:46.689007 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:46.688981    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:41:46.689045 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:46.689032    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:41:47.479540 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:47.479190    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v" event={"ID":"b99ab818-3be9-406e-be18-c0ed26c43bea","Type":"ContainerStarted","Data":"0e38bd02fc5f548539f1c3a269077b42a10645acd3329bd78a14d359aaebe30f"}
Oct 30 19:41:47.479540 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:47.479262    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v" event={"ID":"b99ab818-3be9-406e-be18-c0ed26c43bea","Type":"ContainerStarted","Data":"98879e1b59c007b8a0972675b2be82eb6dedeea506170f15d35fb69b53b1f298"}
Oct 30 19:41:47.513738 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:47.513675    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v" podStartSLOduration=1.513655565 podStartE2EDuration="1.513655565s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:41:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:41:47.508787416 +0000 UTC m=+582.836905570" watchObservedRunningTime="2026-10-30 19:41:47.513655565 +0000 UTC m=+582.841773719"
Oct 30 19:41:48.388767 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:48.388719    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v"
Oct 30 19:41:48.390859 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:48.390829    6996 patch_prober.go:29] interesting pod/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Oct 30 19:41:48.391095 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:48.391066    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v" podUID="b99ab818-3be9-406e-be18-c0ed26c43bea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Oct 30 19:41:48.766781 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:48.766691    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr"]
Oct 30 19:41:48.771171 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:48.771146    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr"
Oct 30 19:41:48.779835 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:48.779806    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-2a3e1b\"/\"default-dockercfg-9wcd7\""
Oct 30 19:41:48.780721 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:48.780512    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-2a3e1b\"/\"kube-root-ca.crt\""
Oct 30 19:41:48.781434 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:48.780572    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-2a3e1b\"/\"openshift-service-ca.crt\""
Oct 30 19:41:48.781618 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:48.780671    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-2a3e1b\"/\"premium-w4-2a3e1b-kserve-self-signed-certs\""
Oct 30 19:41:48.781981 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:48.781959    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr"]
Oct 30 19:41:48.790336 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:48.790315    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0e470010-625d-443f-81ec-9bf965988212-model-cache\") pod \"premium-w4-2a3e1b-kserve-7599454564-4dcwr\" (UID: \"0e470010-625d-443f-81ec-9bf965988212\") " pod="e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr"
Oct 30 19:41:48.790588 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:48.790561    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0e470010-625d-443f-81ec-9bf965988212-tmp-dir\") pod \"premium-w4-2a3e1b-kserve-7599454564-4dcwr\" (UID: \"0e470010-625d-443f-81ec-9bf965988212\") " pod="e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr"
Oct 30 19:41:48.790696 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:48.790606    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvdtg\" (UniqueName: \"kubernetes.io/projected/0e470010-625d-443f-81ec-9bf965988212-kube-api-access-wvdtg\") pod \"premium-w4-2a3e1b-kserve-7599454564-4dcwr\" (UID: \"0e470010-625d-443f-81ec-9bf965988212\") " pod="e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr"
Oct 30 19:41:48.790696 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:48.790638    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0e470010-625d-443f-81ec-9bf965988212-home\") pod \"premium-w4-2a3e1b-kserve-7599454564-4dcwr\" (UID: \"0e470010-625d-443f-81ec-9bf965988212\") " pod="e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr"
Oct 30 19:41:48.790696 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:48.790659    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0e470010-625d-443f-81ec-9bf965988212-dshm\") pod \"premium-w4-2a3e1b-kserve-7599454564-4dcwr\" (UID: \"0e470010-625d-443f-81ec-9bf965988212\") " pod="e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr"
Oct 30 19:41:48.790877 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:48.790710    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e470010-625d-443f-81ec-9bf965988212-tls-certs\") pod \"premium-w4-2a3e1b-kserve-7599454564-4dcwr\" (UID: \"0e470010-625d-443f-81ec-9bf965988212\") " pod="e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr"
Oct 30 19:41:48.892265 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:48.892227    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0e470010-625d-443f-81ec-9bf965988212-model-cache\") pod \"premium-w4-2a3e1b-kserve-7599454564-4dcwr\" (UID: \"0e470010-625d-443f-81ec-9bf965988212\") " pod="e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr"
Oct 30 19:41:48.892446 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:48.892334    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0e470010-625d-443f-81ec-9bf965988212-tmp-dir\") pod \"premium-w4-2a3e1b-kserve-7599454564-4dcwr\" (UID: \"0e470010-625d-443f-81ec-9bf965988212\") " pod="e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr"
Oct 30 19:41:48.892446 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:48.892365    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvdtg\" (UniqueName: \"kubernetes.io/projected/0e470010-625d-443f-81ec-9bf965988212-kube-api-access-wvdtg\") pod \"premium-w4-2a3e1b-kserve-7599454564-4dcwr\" (UID: \"0e470010-625d-443f-81ec-9bf965988212\") " pod="e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr"
Oct 30 19:41:48.892446 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:48.892386    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0e470010-625d-443f-81ec-9bf965988212-home\") pod \"premium-w4-2a3e1b-kserve-7599454564-4dcwr\" (UID: \"0e470010-625d-443f-81ec-9bf965988212\") " pod="e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr"
Oct 30 19:41:48.892446 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:48.892402    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0e470010-625d-443f-81ec-9bf965988212-dshm\") pod \"premium-w4-2a3e1b-kserve-7599454564-4dcwr\" (UID: \"0e470010-625d-443f-81ec-9bf965988212\") " pod="e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr"
Oct 30 19:41:48.892446 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:48.892439    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e470010-625d-443f-81ec-9bf965988212-tls-certs\") pod \"premium-w4-2a3e1b-kserve-7599454564-4dcwr\" (UID: \"0e470010-625d-443f-81ec-9bf965988212\") " pod="e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr"
Oct 30 19:41:48.892832 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:48.892800    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0e470010-625d-443f-81ec-9bf965988212-model-cache\") pod \"premium-w4-2a3e1b-kserve-7599454564-4dcwr\" (UID: \"0e470010-625d-443f-81ec-9bf965988212\") " pod="e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr"
Oct 30 19:41:48.892943 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:48.892885    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0e470010-625d-443f-81ec-9bf965988212-tmp-dir\") pod \"premium-w4-2a3e1b-kserve-7599454564-4dcwr\" (UID: \"0e470010-625d-443f-81ec-9bf965988212\") " pod="e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr"
Oct 30 19:41:48.892943 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:48.892885    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0e470010-625d-443f-81ec-9bf965988212-home\") pod \"premium-w4-2a3e1b-kserve-7599454564-4dcwr\" (UID: \"0e470010-625d-443f-81ec-9bf965988212\") " pod="e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr"
Oct 30 19:41:48.894839 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:48.894812    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0e470010-625d-443f-81ec-9bf965988212-dshm\") pod \"premium-w4-2a3e1b-kserve-7599454564-4dcwr\" (UID: \"0e470010-625d-443f-81ec-9bf965988212\") " pod="e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr"
Oct 30 19:41:48.895068 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:48.895050    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e470010-625d-443f-81ec-9bf965988212-tls-certs\") pod \"premium-w4-2a3e1b-kserve-7599454564-4dcwr\" (UID: \"0e470010-625d-443f-81ec-9bf965988212\") " pod="e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr"
Oct 30 19:41:48.922383 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:48.922349    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvdtg\" (UniqueName: \"kubernetes.io/projected/0e470010-625d-443f-81ec-9bf965988212-kube-api-access-wvdtg\") pod \"premium-w4-2a3e1b-kserve-7599454564-4dcwr\" (UID: \"0e470010-625d-443f-81ec-9bf965988212\") " pod="e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr"
Oct 30 19:41:49.096160 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:49.096126    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr"
Oct 30 19:41:49.391552 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:49.391068    6996 patch_prober.go:29] interesting pod/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Oct 30 19:41:49.391552 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:49.391129    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v" podUID="b99ab818-3be9-406e-be18-c0ed26c43bea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Oct 30 19:41:49.474504 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:49.473070    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr"]
Oct 30 19:41:49.512665 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:41:49.512502    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0e470010_625d_443f_81ec_9bf965988212.slice/crio-01ea573abfd479ce3f1ed75336667b85ca0b8841885f82e63cacbe3688045ffc WatchSource:0}: Error finding container 01ea573abfd479ce3f1ed75336667b85ca0b8841885f82e63cacbe3688045ffc: Status 404 returned error can't find the container with id 01ea573abfd479ce3f1ed75336667b85ca0b8841885f82e63cacbe3688045ffc
Oct 30 19:41:50.389248 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:50.389215    6996 patch_prober.go:29] interesting pod/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Oct 30 19:41:50.389657 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:50.389270    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v" podUID="b99ab818-3be9-406e-be18-c0ed26c43bea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Oct 30 19:41:50.515898 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:50.515867    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr" event={"ID":"0e470010-625d-443f-81ec-9bf965988212","Type":"ContainerStarted","Data":"1ea96199554c4db1446c246d6c52c0fb1b5334aeb9ab8ad5d1d30748d398d204"}
Oct 30 19:41:50.516080 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:50.515905    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr" event={"ID":"0e470010-625d-443f-81ec-9bf965988212","Type":"ContainerStarted","Data":"01ea573abfd479ce3f1ed75336667b85ca0b8841885f82e63cacbe3688045ffc"}
Oct 30 19:41:51.097093 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:51.097056    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr"
Oct 30 19:41:51.389466 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:51.389375    6996 patch_prober.go:29] interesting pod/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Oct 30 19:41:51.389466 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:51.389437    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v" podUID="b99ab818-3be9-406e-be18-c0ed26c43bea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Oct 30 19:41:52.389470 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:52.389432    6996 patch_prober.go:29] interesting pod/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Oct 30 19:41:52.389882 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:52.389507    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v" podUID="b99ab818-3be9-406e-be18-c0ed26c43bea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Oct 30 19:41:52.575605 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:52.575542    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr" podStartSLOduration=4.575527945 podStartE2EDuration="4.575527945s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:41:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:41:50.568845038 +0000 UTC m=+585.896963192" watchObservedRunningTime="2026-10-30 19:41:52.575527945 +0000 UTC m=+587.903646170"
Oct 30 19:41:52.576773 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:52.576755    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 19:41:54.144235 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:54.144191    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v"
Oct 30 19:41:54.144676 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:54.144605    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v"
Oct 30 19:41:54.145524 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:54.145474    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v"
Oct 30 19:41:59.096337 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:59.096302    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr"
Oct 30 19:41:59.110640 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:59.110612    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr"
Oct 30 19:41:59.584669 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:41:59.584640    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr"
Oct 30 19:42:01.195681 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:42:01.195645    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 19:42:03.959685 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:03.959645    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s"]
Oct 30 19:42:03.965571 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:03.965546    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s"
Oct 30 19:42:03.984881 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:03.984851    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-7a745c\"/\"openshift-service-ca.crt\""
Oct 30 19:42:03.985030 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:03.984904    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-7a745c\"/\"kube-root-ca.crt\""
Oct 30 19:42:03.985250 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:03.985223    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-7a745c\"/\"default-dockercfg-zlh4x\""
Oct 30 19:42:03.985551 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:03.985281    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-7a745c\"/\"premium-w1-7a745c-kserve-self-signed-certs\""
Oct 30 19:42:03.988994 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:03.988973    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s"]
Oct 30 19:42:04.044759 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:04.044728    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae905dfb-f30d-480a-899b-ea676abc1d8d-tmp-dir\") pod \"premium-w1-7a745c-kserve-c584476d-qvm9s\" (UID: \"ae905dfb-f30d-480a-899b-ea676abc1d8d\") " pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s"
Oct 30 19:42:04.044759 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:04.044763    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2mgk\" (UniqueName: \"kubernetes.io/projected/ae905dfb-f30d-480a-899b-ea676abc1d8d-kube-api-access-c2mgk\") pod \"premium-w1-7a745c-kserve-c584476d-qvm9s\" (UID: \"ae905dfb-f30d-480a-899b-ea676abc1d8d\") " pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s"
Oct 30 19:42:04.044942 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:04.044835    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae905dfb-f30d-480a-899b-ea676abc1d8d-tls-certs\") pod \"premium-w1-7a745c-kserve-c584476d-qvm9s\" (UID: \"ae905dfb-f30d-480a-899b-ea676abc1d8d\") " pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s"
Oct 30 19:42:04.044942 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:04.044918    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ae905dfb-f30d-480a-899b-ea676abc1d8d-model-cache\") pod \"premium-w1-7a745c-kserve-c584476d-qvm9s\" (UID: \"ae905dfb-f30d-480a-899b-ea676abc1d8d\") " pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s"
Oct 30 19:42:04.045028 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:04.044971    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ae905dfb-f30d-480a-899b-ea676abc1d8d-home\") pod \"premium-w1-7a745c-kserve-c584476d-qvm9s\" (UID: \"ae905dfb-f30d-480a-899b-ea676abc1d8d\") " pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s"
Oct 30 19:42:04.045074 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:04.045060    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ae905dfb-f30d-480a-899b-ea676abc1d8d-dshm\") pod \"premium-w1-7a745c-kserve-c584476d-qvm9s\" (UID: \"ae905dfb-f30d-480a-899b-ea676abc1d8d\") " pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s"
Oct 30 19:42:04.145666 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:04.145632    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae905dfb-f30d-480a-899b-ea676abc1d8d-tls-certs\") pod \"premium-w1-7a745c-kserve-c584476d-qvm9s\" (UID: \"ae905dfb-f30d-480a-899b-ea676abc1d8d\") " pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s"
Oct 30 19:42:04.145874 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:04.145680    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ae905dfb-f30d-480a-899b-ea676abc1d8d-model-cache\") pod \"premium-w1-7a745c-kserve-c584476d-qvm9s\" (UID: \"ae905dfb-f30d-480a-899b-ea676abc1d8d\") " pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s"
Oct 30 19:42:04.145874 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:04.145745    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ae905dfb-f30d-480a-899b-ea676abc1d8d-home\") pod \"premium-w1-7a745c-kserve-c584476d-qvm9s\" (UID: \"ae905dfb-f30d-480a-899b-ea676abc1d8d\") " pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s"
Oct 30 19:42:04.145874 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:04.145790    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ae905dfb-f30d-480a-899b-ea676abc1d8d-dshm\") pod \"premium-w1-7a745c-kserve-c584476d-qvm9s\" (UID: \"ae905dfb-f30d-480a-899b-ea676abc1d8d\") " pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s"
Oct 30 19:42:04.146109 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:04.145881    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae905dfb-f30d-480a-899b-ea676abc1d8d-tmp-dir\") pod \"premium-w1-7a745c-kserve-c584476d-qvm9s\" (UID: \"ae905dfb-f30d-480a-899b-ea676abc1d8d\") " pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s"
Oct 30 19:42:04.146109 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:04.145909    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2mgk\" (UniqueName: \"kubernetes.io/projected/ae905dfb-f30d-480a-899b-ea676abc1d8d-kube-api-access-c2mgk\") pod \"premium-w1-7a745c-kserve-c584476d-qvm9s\" (UID: \"ae905dfb-f30d-480a-899b-ea676abc1d8d\") " pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s"
Oct 30 19:42:04.146229 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:04.146192    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ae905dfb-f30d-480a-899b-ea676abc1d8d-model-cache\") pod \"premium-w1-7a745c-kserve-c584476d-qvm9s\" (UID: \"ae905dfb-f30d-480a-899b-ea676abc1d8d\") " pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s"
Oct 30 19:42:04.146285 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:04.146197    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ae905dfb-f30d-480a-899b-ea676abc1d8d-home\") pod \"premium-w1-7a745c-kserve-c584476d-qvm9s\" (UID: \"ae905dfb-f30d-480a-899b-ea676abc1d8d\") " pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s"
Oct 30 19:42:04.146285 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:04.146262    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae905dfb-f30d-480a-899b-ea676abc1d8d-tmp-dir\") pod \"premium-w1-7a745c-kserve-c584476d-qvm9s\" (UID: \"ae905dfb-f30d-480a-899b-ea676abc1d8d\") " pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s"
Oct 30 19:42:04.148282 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:04.148260    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ae905dfb-f30d-480a-899b-ea676abc1d8d-dshm\") pod \"premium-w1-7a745c-kserve-c584476d-qvm9s\" (UID: \"ae905dfb-f30d-480a-899b-ea676abc1d8d\") " pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s"
Oct 30 19:42:04.148392 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:04.148260    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae905dfb-f30d-480a-899b-ea676abc1d8d-tls-certs\") pod \"premium-w1-7a745c-kserve-c584476d-qvm9s\" (UID: \"ae905dfb-f30d-480a-899b-ea676abc1d8d\") " pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s"
Oct 30 19:42:04.159348 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:04.159326    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2mgk\" (UniqueName: \"kubernetes.io/projected/ae905dfb-f30d-480a-899b-ea676abc1d8d-kube-api-access-c2mgk\") pod \"premium-w1-7a745c-kserve-c584476d-qvm9s\" (UID: \"ae905dfb-f30d-480a-899b-ea676abc1d8d\") " pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s"
Oct 30 19:42:04.279082 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:04.279004    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s"
Oct 30 19:42:04.545008 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:04.544933    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 19:42:04.574391 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:04.574367    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s"]
Oct 30 19:42:04.576815 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:42:04.576788    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podae905dfb_f30d_480a_899b_ea676abc1d8d.slice/crio-de12172389f6110cc3f1f4ea9c1dba85538360bcfed66d08dd6103098ffbfb41 WatchSource:0}: Error finding container de12172389f6110cc3f1f4ea9c1dba85538360bcfed66d08dd6103098ffbfb41: Status 404 returned error can't find the container with id de12172389f6110cc3f1f4ea9c1dba85538360bcfed66d08dd6103098ffbfb41
Oct 30 19:42:04.607849 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:04.607816    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s" event={"ID":"ae905dfb-f30d-480a-899b-ea676abc1d8d","Type":"ContainerStarted","Data":"de12172389f6110cc3f1f4ea9c1dba85538360bcfed66d08dd6103098ffbfb41"}
Oct 30 19:42:05.616594 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:05.616545    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s" event={"ID":"ae905dfb-f30d-480a-899b-ea676abc1d8d","Type":"ContainerStarted","Data":"ff871009009b34989b690b556cdb584fb6ca5e78b6839c1f8d5b9bd8dda8455b"}
Oct 30 19:42:05.639581 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:05.639537    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s" podStartSLOduration=2.63952293 podStartE2EDuration="2.63952293s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:42:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:42:05.637702229 +0000 UTC m=+600.965820382" watchObservedRunningTime="2026-10-30 19:42:05.63952293 +0000 UTC m=+600.967641085"
Oct 30 19:42:06.279860 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:06.279821    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s"
Oct 30 19:42:08.396951 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:08.396908    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc"]
Oct 30 19:42:08.402200 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:08.402179    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc"
Oct 30 19:42:08.407449 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:08.407425    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-2a3e1b-openshift-default-dockercfg-gv7kl\""
Oct 30 19:42:08.427508 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:08.427453    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc"]
Oct 30 19:42:08.497681 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:08.497638    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7180ebbf-2671-4778-81a7-57a8aae4a783-istio-envoy\") pod \"e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc\" (UID: \"7180ebbf-2671-4778-81a7-57a8aae4a783\") " pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc"
Oct 30 19:42:08.497871 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:08.497696    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7180ebbf-2671-4778-81a7-57a8aae4a783-workload-certs\") pod \"e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc\" (UID: \"7180ebbf-2671-4778-81a7-57a8aae4a783\") " pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc"
Oct 30 19:42:08.497871 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:08.497787    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7180ebbf-2671-4778-81a7-57a8aae4a783-credential-socket\") pod \"e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc\" (UID: \"7180ebbf-2671-4778-81a7-57a8aae4a783\") " pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc"
Oct 30 19:42:08.497871 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:08.497833    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7180ebbf-2671-4778-81a7-57a8aae4a783-istiod-ca-cert\") pod \"e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc\" (UID: \"7180ebbf-2671-4778-81a7-57a8aae4a783\") " pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc"
Oct 30 19:42:08.497871 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:08.497853    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7180ebbf-2671-4778-81a7-57a8aae4a783-istio-podinfo\") pod \"e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc\" (UID: \"7180ebbf-2671-4778-81a7-57a8aae4a783\") " pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc"
Oct 30 19:42:08.497871 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:08.497869    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpkjg\" (UniqueName: \"kubernetes.io/projected/7180ebbf-2671-4778-81a7-57a8aae4a783-kube-api-access-cpkjg\") pod \"e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc\" (UID: \"7180ebbf-2671-4778-81a7-57a8aae4a783\") " pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc"
Oct 30 19:42:08.498061 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:08.498010    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7180ebbf-2671-4778-81a7-57a8aae4a783-workload-socket\") pod \"e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc\" (UID: \"7180ebbf-2671-4778-81a7-57a8aae4a783\") " pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc"
Oct 30 19:42:08.498061 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:08.498039    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7180ebbf-2671-4778-81a7-57a8aae4a783-istio-token\") pod \"e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc\" (UID: \"7180ebbf-2671-4778-81a7-57a8aae4a783\") " pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc"
Oct 30 19:42:08.498193 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:08.498177    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7180ebbf-2671-4778-81a7-57a8aae4a783-istio-data\") pod \"e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc\" (UID: \"7180ebbf-2671-4778-81a7-57a8aae4a783\") " pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc"
Oct 30 19:42:08.599167 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:08.599128    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7180ebbf-2671-4778-81a7-57a8aae4a783-istio-envoy\") pod \"e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc\" (UID: \"7180ebbf-2671-4778-81a7-57a8aae4a783\") " pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc"
Oct 30 19:42:08.599370 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:08.599181    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7180ebbf-2671-4778-81a7-57a8aae4a783-workload-certs\") pod \"e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc\" (UID: \"7180ebbf-2671-4778-81a7-57a8aae4a783\") " pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc"
Oct 30 19:42:08.599370 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:08.599236    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7180ebbf-2671-4778-81a7-57a8aae4a783-credential-socket\") pod \"e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc\" (UID: \"7180ebbf-2671-4778-81a7-57a8aae4a783\") " pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc"
Oct 30 19:42:08.599370 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:08.599258    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7180ebbf-2671-4778-81a7-57a8aae4a783-istiod-ca-cert\") pod \"e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc\" (UID: \"7180ebbf-2671-4778-81a7-57a8aae4a783\") " pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc"
Oct 30 19:42:08.599370 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:08.599283    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7180ebbf-2671-4778-81a7-57a8aae4a783-istio-podinfo\") pod \"e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc\" (UID: \"7180ebbf-2671-4778-81a7-57a8aae4a783\") " pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc"
Oct 30 19:42:08.599370 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:08.599307    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpkjg\" (UniqueName: \"kubernetes.io/projected/7180ebbf-2671-4778-81a7-57a8aae4a783-kube-api-access-cpkjg\") pod \"e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc\" (UID: \"7180ebbf-2671-4778-81a7-57a8aae4a783\") " pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc"
Oct 30 19:42:08.599370 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:08.599351    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7180ebbf-2671-4778-81a7-57a8aae4a783-workload-socket\") pod \"e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc\" (UID: \"7180ebbf-2671-4778-81a7-57a8aae4a783\") " pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc"
Oct 30 19:42:08.599721 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:08.599377    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7180ebbf-2671-4778-81a7-57a8aae4a783-istio-token\") pod \"e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc\" (UID: \"7180ebbf-2671-4778-81a7-57a8aae4a783\") " pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc"
Oct 30 19:42:08.599721 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:08.599541    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7180ebbf-2671-4778-81a7-57a8aae4a783-istio-data\") pod \"e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc\" (UID: \"7180ebbf-2671-4778-81a7-57a8aae4a783\") " pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc"
Oct 30 19:42:08.599829 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:08.599798    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7180ebbf-2671-4778-81a7-57a8aae4a783-credential-socket\") pod \"e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc\" (UID: \"7180ebbf-2671-4778-81a7-57a8aae4a783\") " pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc"
Oct 30 19:42:08.599884 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:08.599829    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7180ebbf-2671-4778-81a7-57a8aae4a783-workload-socket\") pod \"e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc\" (UID: \"7180ebbf-2671-4778-81a7-57a8aae4a783\") " pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc"
Oct 30 19:42:08.599948 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:08.599882    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7180ebbf-2671-4778-81a7-57a8aae4a783-istio-data\") pod \"e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc\" (UID: \"7180ebbf-2671-4778-81a7-57a8aae4a783\") " pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc"
Oct 30 19:42:08.599948 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:08.599923    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7180ebbf-2671-4778-81a7-57a8aae4a783-workload-certs\") pod \"e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc\" (UID: \"7180ebbf-2671-4778-81a7-57a8aae4a783\") " pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc"
Oct 30 19:42:08.600146 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:08.600127    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7180ebbf-2671-4778-81a7-57a8aae4a783-istiod-ca-cert\") pod \"e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc\" (UID: \"7180ebbf-2671-4778-81a7-57a8aae4a783\") " pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc"
Oct 30 19:42:08.601605 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:08.601587    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7180ebbf-2671-4778-81a7-57a8aae4a783-istio-envoy\") pod \"e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc\" (UID: \"7180ebbf-2671-4778-81a7-57a8aae4a783\") " pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc"
Oct 30 19:42:08.601724 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:08.601708    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7180ebbf-2671-4778-81a7-57a8aae4a783-istio-podinfo\") pod \"e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc\" (UID: \"7180ebbf-2671-4778-81a7-57a8aae4a783\") " pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc"
Oct 30 19:42:08.611754 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:08.611715    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpkjg\" (UniqueName: \"kubernetes.io/projected/7180ebbf-2671-4778-81a7-57a8aae4a783-kube-api-access-cpkjg\") pod \"e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc\" (UID: \"7180ebbf-2671-4778-81a7-57a8aae4a783\") " pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc"
Oct 30 19:42:08.612360 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:08.612342    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7180ebbf-2671-4778-81a7-57a8aae4a783-istio-token\") pod \"e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc\" (UID: \"7180ebbf-2671-4778-81a7-57a8aae4a783\") " pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc"
Oct 30 19:42:08.716770 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:08.716678    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc"
Oct 30 19:42:09.011857 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:09.011826    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc"]
Oct 30 19:42:09.013955 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:42:09.013920    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7180ebbf_2671_4778_81a7_57a8aae4a783.slice/crio-44f22fc183cd425c089f4c00fb7f833b527d0c4a8f8cafa333f540bd584c6d70 WatchSource:0}: Error finding container 44f22fc183cd425c089f4c00fb7f833b527d0c4a8f8cafa333f540bd584c6d70: Status 404 returned error can't find the container with id 44f22fc183cd425c089f4c00fb7f833b527d0c4a8f8cafa333f540bd584c6d70
Oct 30 19:42:09.016543 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:09.016474    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:42:09.016643 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:09.016593    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:42:09.016707 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:09.016674    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:42:09.666054 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:09.665999    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc" event={"ID":"7180ebbf-2671-4778-81a7-57a8aae4a783","Type":"ContainerStarted","Data":"bff4c9621e743d156397f4765bc0b85e01dd4b592bc26e11ec2f83a706173f90"}
Oct 30 19:42:09.667112 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:09.667084    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc" event={"ID":"7180ebbf-2671-4778-81a7-57a8aae4a783","Type":"ContainerStarted","Data":"44f22fc183cd425c089f4c00fb7f833b527d0c4a8f8cafa333f540bd584c6d70"}
Oct 30 19:42:09.694356 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:09.694293    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc" podStartSLOduration=1.694272037 podStartE2EDuration="1.694272037s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:42:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:42:09.693294492 +0000 UTC m=+605.021412658" watchObservedRunningTime="2026-10-30 19:42:09.694272037 +0000 UTC m=+605.022390192"
Oct 30 19:42:09.717729 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:09.717689    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc"
Oct 30 19:42:10.719411 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:10.719377    6996 patch_prober.go:29] interesting pod/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Oct 30 19:42:10.719836 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:10.719435    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc" podUID="7180ebbf-2671-4778-81a7-57a8aae4a783" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Oct 30 19:42:11.717876 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:11.717833    6996 patch_prober.go:29] interesting pod/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Oct 30 19:42:11.718100 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:11.717898    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc" podUID="7180ebbf-2671-4778-81a7-57a8aae4a783" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Oct 30 19:42:12.718066 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:12.718027    6996 patch_prober.go:29] interesting pod/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Oct 30 19:42:12.718635 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:12.718085    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc" podUID="7180ebbf-2671-4778-81a7-57a8aae4a783" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Oct 30 19:42:13.727502 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:13.721530    6996 patch_prober.go:29] interesting pod/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Oct 30 19:42:13.728037 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:13.727646    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc" podUID="7180ebbf-2671-4778-81a7-57a8aae4a783" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Oct 30 19:42:14.279899 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:14.279854    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s"
Oct 30 19:42:14.293317 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:14.293289    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s"
Oct 30 19:42:14.717474 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:14.717437    6996 patch_prober.go:29] interesting pod/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Oct 30 19:42:14.717789 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:14.717758    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc" podUID="7180ebbf-2671-4778-81a7-57a8aae4a783" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Oct 30 19:42:14.775493 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:14.775446    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s"
Oct 30 19:42:15.783153 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:15.783117    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc"
Oct 30 19:42:15.783654 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:15.783461    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc"
Oct 30 19:42:15.784429 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:15.784406    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc"
Oct 30 19:42:19.686149 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:19.686114    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 19:42:31.194417 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:42:31.194375    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 19:42:37.051000 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:37.050957    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 19:42:44.797820 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:44.797788    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv"]
Oct 30 19:42:44.803435 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:44.803409    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv"
Oct 30 19:42:44.807017 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:44.806987    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv"]
Oct 30 19:42:44.808386 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:44.808352    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-7a745c-openshift-default-dockercfg-svrhl\""
Oct 30 19:42:44.878530 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:44.878493    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b55d9c75-1365-4bac-852e-326f4380cca7-istiod-ca-cert\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv\" (UID: \"b55d9c75-1365-4bac-852e-326f4380cca7\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv"
Oct 30 19:42:44.878530 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:44.878533    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpfh7\" (UniqueName: \"kubernetes.io/projected/b55d9c75-1365-4bac-852e-326f4380cca7-kube-api-access-kpfh7\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv\" (UID: \"b55d9c75-1365-4bac-852e-326f4380cca7\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv"
Oct 30 19:42:44.878783 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:44.878672    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b55d9c75-1365-4bac-852e-326f4380cca7-workload-socket\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv\" (UID: \"b55d9c75-1365-4bac-852e-326f4380cca7\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv"
Oct 30 19:42:44.878950 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:44.878918    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b55d9c75-1365-4bac-852e-326f4380cca7-istio-token\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv\" (UID: \"b55d9c75-1365-4bac-852e-326f4380cca7\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv"
Oct 30 19:42:44.879062 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:44.879038    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b55d9c75-1365-4bac-852e-326f4380cca7-istio-envoy\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv\" (UID: \"b55d9c75-1365-4bac-852e-326f4380cca7\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv"
Oct 30 19:42:44.879147 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:44.879130    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b55d9c75-1365-4bac-852e-326f4380cca7-istio-data\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv\" (UID: \"b55d9c75-1365-4bac-852e-326f4380cca7\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv"
Oct 30 19:42:44.879195 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:44.879178    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b55d9c75-1365-4bac-852e-326f4380cca7-workload-certs\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv\" (UID: \"b55d9c75-1365-4bac-852e-326f4380cca7\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv"
Oct 30 19:42:44.879319 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:44.879297    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b55d9c75-1365-4bac-852e-326f4380cca7-credential-socket\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv\" (UID: \"b55d9c75-1365-4bac-852e-326f4380cca7\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv"
Oct 30 19:42:44.879363 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:44.879336    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b55d9c75-1365-4bac-852e-326f4380cca7-istio-podinfo\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv\" (UID: \"b55d9c75-1365-4bac-852e-326f4380cca7\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv"
Oct 30 19:42:44.980090 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:44.980055    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b55d9c75-1365-4bac-852e-326f4380cca7-workload-certs\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv\" (UID: \"b55d9c75-1365-4bac-852e-326f4380cca7\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv"
Oct 30 19:42:44.980289 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:44.980123    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b55d9c75-1365-4bac-852e-326f4380cca7-credential-socket\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv\" (UID: \"b55d9c75-1365-4bac-852e-326f4380cca7\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv"
Oct 30 19:42:44.980289 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:44.980143    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b55d9c75-1365-4bac-852e-326f4380cca7-istio-podinfo\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv\" (UID: \"b55d9c75-1365-4bac-852e-326f4380cca7\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv"
Oct 30 19:42:44.980289 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:44.980199    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b55d9c75-1365-4bac-852e-326f4380cca7-istiod-ca-cert\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv\" (UID: \"b55d9c75-1365-4bac-852e-326f4380cca7\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv"
Oct 30 19:42:44.980289 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:44.980215    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpfh7\" (UniqueName: \"kubernetes.io/projected/b55d9c75-1365-4bac-852e-326f4380cca7-kube-api-access-kpfh7\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv\" (UID: \"b55d9c75-1365-4bac-852e-326f4380cca7\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv"
Oct 30 19:42:44.980289 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:44.980256    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b55d9c75-1365-4bac-852e-326f4380cca7-workload-socket\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv\" (UID: \"b55d9c75-1365-4bac-852e-326f4380cca7\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv"
Oct 30 19:42:44.980613 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:44.980335    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b55d9c75-1365-4bac-852e-326f4380cca7-istio-token\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv\" (UID: \"b55d9c75-1365-4bac-852e-326f4380cca7\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv"
Oct 30 19:42:44.980613 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:44.980375    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b55d9c75-1365-4bac-852e-326f4380cca7-istio-envoy\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv\" (UID: \"b55d9c75-1365-4bac-852e-326f4380cca7\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv"
Oct 30 19:42:44.980727 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:44.980678    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b55d9c75-1365-4bac-852e-326f4380cca7-istio-data\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv\" (UID: \"b55d9c75-1365-4bac-852e-326f4380cca7\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv"
Oct 30 19:42:44.980826 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:44.980793    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b55d9c75-1365-4bac-852e-326f4380cca7-credential-socket\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv\" (UID: \"b55d9c75-1365-4bac-852e-326f4380cca7\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv"
Oct 30 19:42:44.980826 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:44.980808    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b55d9c75-1365-4bac-852e-326f4380cca7-workload-socket\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv\" (UID: \"b55d9c75-1365-4bac-852e-326f4380cca7\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv"
Oct 30 19:42:44.980950 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:44.980898    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b55d9c75-1365-4bac-852e-326f4380cca7-workload-certs\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv\" (UID: \"b55d9c75-1365-4bac-852e-326f4380cca7\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv"
Oct 30 19:42:44.981104 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:44.981081    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b55d9c75-1365-4bac-852e-326f4380cca7-istio-data\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv\" (UID: \"b55d9c75-1365-4bac-852e-326f4380cca7\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv"
Oct 30 19:42:44.981169 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:44.981151    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b55d9c75-1365-4bac-852e-326f4380cca7-istiod-ca-cert\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv\" (UID: \"b55d9c75-1365-4bac-852e-326f4380cca7\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv"
Oct 30 19:42:44.982574 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:44.982555    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b55d9c75-1365-4bac-852e-326f4380cca7-istio-envoy\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv\" (UID: \"b55d9c75-1365-4bac-852e-326f4380cca7\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv"
Oct 30 19:42:44.982699 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:44.982682    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b55d9c75-1365-4bac-852e-326f4380cca7-istio-podinfo\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv\" (UID: \"b55d9c75-1365-4bac-852e-326f4380cca7\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv"
Oct 30 19:42:44.992666 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:44.992645    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b55d9c75-1365-4bac-852e-326f4380cca7-istio-token\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv\" (UID: \"b55d9c75-1365-4bac-852e-326f4380cca7\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv"
Oct 30 19:42:44.992982 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:44.992962    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpfh7\" (UniqueName: \"kubernetes.io/projected/b55d9c75-1365-4bac-852e-326f4380cca7-kube-api-access-kpfh7\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv\" (UID: \"b55d9c75-1365-4bac-852e-326f4380cca7\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv"
Oct 30 19:42:45.120951 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:45.120921    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv"
Oct 30 19:42:45.414957 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:45.414932    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv"]
Oct 30 19:42:45.417075 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:42:45.417046    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb55d9c75_1365_4bac_852e_326f4380cca7.slice/crio-ff0cb40db78fdab142becb97060df1921d6522cbaa02771830c8fb557a96d833 WatchSource:0}: Error finding container ff0cb40db78fdab142becb97060df1921d6522cbaa02771830c8fb557a96d833: Status 404 returned error can't find the container with id ff0cb40db78fdab142becb97060df1921d6522cbaa02771830c8fb557a96d833
Oct 30 19:42:45.419648 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:45.419616    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:42:45.419737 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:45.419679    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:42:45.419737 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:45.419711    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:42:45.974029 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:45.973986    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv" event={"ID":"b55d9c75-1365-4bac-852e-326f4380cca7","Type":"ContainerStarted","Data":"2c1eeb7a523b74593fc04116358c713dacee6477b1fe5c487c1ac55cf8907c6c"}
Oct 30 19:42:45.974533 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:45.974086    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv" event={"ID":"b55d9c75-1365-4bac-852e-326f4380cca7","Type":"ContainerStarted","Data":"ff0cb40db78fdab142becb97060df1921d6522cbaa02771830c8fb557a96d833"}
Oct 30 19:42:46.006738 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:46.006686    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv" podStartSLOduration=2.006670559 podStartE2EDuration="2.006670559s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:42:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:42:46.002625547 +0000 UTC m=+641.330743706" watchObservedRunningTime="2026-10-30 19:42:46.006670559 +0000 UTC m=+641.334788712"
Oct 30 19:42:46.121168 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:46.121130    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv"
Oct 30 19:42:46.122389 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:46.122363    6996 patch_prober.go:29] interesting pod/e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Oct 30 19:42:46.122553 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:46.122418    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv" podUID="b55d9c75-1365-4bac-852e-326f4380cca7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Oct 30 19:42:47.122786 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:47.122747    6996 patch_prober.go:29] interesting pod/e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Oct 30 19:42:47.123450 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:47.122807    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv" podUID="b55d9c75-1365-4bac-852e-326f4380cca7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Oct 30 19:42:47.935801 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:47.935764    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["models-as-a-service/payload-processing-external-model-54696766b9-rvzmg"]
Oct 30 19:42:47.945408 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:47.945380    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-54696766b9-rvzmg"
Oct 30 19:42:47.956735 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:47.956701    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-54696766b9-rvzmg"]
Oct 30 19:42:47.960343 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:47.959669    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"agc-cred-077d59be6edd\""
Oct 30 19:42:47.960343 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:47.959886    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"payload-processing-external-model-plugins\""
Oct 30 19:42:47.960343 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:47.959952    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"agc-cred-84ddec04140b\""
Oct 30 19:42:47.960343 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:47.960016    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"payload-processing-external-model-dockercfg-8p64v\""
Oct 30 19:42:47.960343 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:47.960116    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"kube-root-ca.crt\""
Oct 30 19:42:47.960343 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:47.960182    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"openshift-service-ca.crt\""
Oct 30 19:42:47.960343 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:47.960269    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"routing-overlay\""
Oct 30 19:42:48.021208 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:48.021173    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/4e5665dc-3c3a-49e4-83de-cd514a209c0b-provider-credentials\") pod \"payload-processing-external-model-54696766b9-rvzmg\" (UID: \"4e5665dc-3c3a-49e4-83de-cd514a209c0b\") " pod="models-as-a-service/payload-processing-external-model-54696766b9-rvzmg"
Oct 30 19:42:48.021208 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:48.021211    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4e5665dc-3c3a-49e4-83de-cd514a209c0b-plugins-config-volume\") pod \"payload-processing-external-model-54696766b9-rvzmg\" (UID: \"4e5665dc-3c3a-49e4-83de-cd514a209c0b\") " pod="models-as-a-service/payload-processing-external-model-54696766b9-rvzmg"
Oct 30 19:42:48.021415 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:48.021267    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdzbf\" (UniqueName: \"kubernetes.io/projected/4e5665dc-3c3a-49e4-83de-cd514a209c0b-kube-api-access-sdzbf\") pod \"payload-processing-external-model-54696766b9-rvzmg\" (UID: \"4e5665dc-3c3a-49e4-83de-cd514a209c0b\") " pod="models-as-a-service/payload-processing-external-model-54696766b9-rvzmg"
Oct 30 19:42:48.021415 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:48.021398    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/4e5665dc-3c3a-49e4-83de-cd514a209c0b-routing-overlay-volume\") pod \"payload-processing-external-model-54696766b9-rvzmg\" (UID: \"4e5665dc-3c3a-49e4-83de-cd514a209c0b\") " pod="models-as-a-service/payload-processing-external-model-54696766b9-rvzmg"
Oct 30 19:42:48.121305 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:48.121263    6996 patch_prober.go:29] interesting pod/e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Oct 30 19:42:48.121599 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:48.121563    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv" podUID="b55d9c75-1365-4bac-852e-326f4380cca7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Oct 30 19:42:48.122376 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:48.122345    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/4e5665dc-3c3a-49e4-83de-cd514a209c0b-routing-overlay-volume\") pod \"payload-processing-external-model-54696766b9-rvzmg\" (UID: \"4e5665dc-3c3a-49e4-83de-cd514a209c0b\") " pod="models-as-a-service/payload-processing-external-model-54696766b9-rvzmg"
Oct 30 19:42:48.122738 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:48.122724    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/4e5665dc-3c3a-49e4-83de-cd514a209c0b-provider-credentials\") pod \"payload-processing-external-model-54696766b9-rvzmg\" (UID: \"4e5665dc-3c3a-49e4-83de-cd514a209c0b\") " pod="models-as-a-service/payload-processing-external-model-54696766b9-rvzmg"
Oct 30 19:42:48.122860 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:48.122848    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4e5665dc-3c3a-49e4-83de-cd514a209c0b-plugins-config-volume\") pod \"payload-processing-external-model-54696766b9-rvzmg\" (UID: \"4e5665dc-3c3a-49e4-83de-cd514a209c0b\") " pod="models-as-a-service/payload-processing-external-model-54696766b9-rvzmg"
Oct 30 19:42:48.123250 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:48.123219    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/4e5665dc-3c3a-49e4-83de-cd514a209c0b-routing-overlay-volume\") pod \"payload-processing-external-model-54696766b9-rvzmg\" (UID: \"4e5665dc-3c3a-49e4-83de-cd514a209c0b\") " pod="models-as-a-service/payload-processing-external-model-54696766b9-rvzmg"
Oct 30 19:42:48.123330 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:48.123315    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdzbf\" (UniqueName: \"kubernetes.io/projected/4e5665dc-3c3a-49e4-83de-cd514a209c0b-kube-api-access-sdzbf\") pod \"payload-processing-external-model-54696766b9-rvzmg\" (UID: \"4e5665dc-3c3a-49e4-83de-cd514a209c0b\") " pod="models-as-a-service/payload-processing-external-model-54696766b9-rvzmg"
Oct 30 19:42:48.124349 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:48.124299    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4e5665dc-3c3a-49e4-83de-cd514a209c0b-plugins-config-volume\") pod \"payload-processing-external-model-54696766b9-rvzmg\" (UID: \"4e5665dc-3c3a-49e4-83de-cd514a209c0b\") " pod="models-as-a-service/payload-processing-external-model-54696766b9-rvzmg"
Oct 30 19:42:48.128192 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:48.128170    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/4e5665dc-3c3a-49e4-83de-cd514a209c0b-provider-credentials\") pod \"payload-processing-external-model-54696766b9-rvzmg\" (UID: \"4e5665dc-3c3a-49e4-83de-cd514a209c0b\") " pod="models-as-a-service/payload-processing-external-model-54696766b9-rvzmg"
Oct 30 19:42:48.144267 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:48.144241    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdzbf\" (UniqueName: \"kubernetes.io/projected/4e5665dc-3c3a-49e4-83de-cd514a209c0b-kube-api-access-sdzbf\") pod \"payload-processing-external-model-54696766b9-rvzmg\" (UID: \"4e5665dc-3c3a-49e4-83de-cd514a209c0b\") " pod="models-as-a-service/payload-processing-external-model-54696766b9-rvzmg"
Oct 30 19:42:48.284552 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:48.283699    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-54696766b9-rvzmg"
Oct 30 19:42:48.705153 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:48.703826    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-54696766b9-rvzmg"]
Oct 30 19:42:49.011076 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:49.010647    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-54696766b9-rvzmg" event={"ID":"4e5665dc-3c3a-49e4-83de-cd514a209c0b","Type":"ContainerStarted","Data":"c5aa6e021717d768f91c3f90cd95eb6af951462e7bbba07ecebe1a0608992dc6"}
Oct 30 19:42:49.011076 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:49.010716    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-54696766b9-rvzmg" event={"ID":"4e5665dc-3c3a-49e4-83de-cd514a209c0b","Type":"ContainerStarted","Data":"ef5af5d32f61cf8c749167337f460274c55b0106bcd298c4d6332ffb6bb1acc5"}
Oct 30 19:42:49.011076 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:49.010762    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="models-as-a-service/payload-processing-external-model-54696766b9-rvzmg"
Oct 30 19:42:49.032852 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:49.032677    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="models-as-a-service/payload-processing-external-model-54696766b9-rvzmg" podStartSLOduration=2.032657644 podStartE2EDuration="2.032657644s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:42:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:42:49.029783694 +0000 UTC m=+644.357901850" watchObservedRunningTime="2026-10-30 19:42:49.032657644 +0000 UTC m=+644.360775799"
Oct 30 19:42:49.121413 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:49.121372    6996 patch_prober.go:29] interesting pod/e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Oct 30 19:42:49.121622 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:49.121432    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv" podUID="b55d9c75-1365-4bac-852e-326f4380cca7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Oct 30 19:42:50.129646 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:50.129603    6996 patch_prober.go:29] interesting pod/e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Oct 30 19:42:50.130319 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:50.130257    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv" podUID="b55d9c75-1365-4bac-852e-326f4380cca7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Oct 30 19:42:50.586514 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:50.579566    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr" podUID="0e470010-625d-443f-81ec-9bf965988212" containerName="main" probeResult="failure" output="Get \"https://10.134.0.72:8000/ready\": context deadline exceeded"
Oct 30 19:42:50.813514 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:50.813439    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s" podUID="ae905dfb-f30d-480a-899b-ea676abc1d8d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.73:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:42:51.164513 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:51.141600    6996 patch_prober.go:29] interesting pod/e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Oct 30 19:42:51.164513 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:51.141663    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv" podUID="b55d9c75-1365-4bac-852e-326f4380cca7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Oct 30 19:42:51.605504 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:51.582577    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr" podUID="0e470010-625d-443f-81ec-9bf965988212" containerName="main" probeResult="failure" output="Get \"https://10.134.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:42:51.605504 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:51.583028    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr"
Oct 30 19:42:51.791508 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:51.761340    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 19:42:51.820625 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:51.820581    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s" podUID="ae905dfb-f30d-480a-899b-ea676abc1d8d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.73:8000/ready\": context deadline exceeded"
Oct 30 19:42:51.850786 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:51.825572    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s"
Oct 30 19:42:51.944560 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:51.944047    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s"
Oct 30 19:42:52.005499 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:52.005387    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr"
Oct 30 19:42:52.122251 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:52.122211    6996 patch_prober.go:29] interesting pod/e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Oct 30 19:42:52.122443 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:52.122271    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv" podUID="b55d9c75-1365-4bac-852e-326f4380cca7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Oct 30 19:42:53.151543 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:53.147671    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv"
Oct 30 19:42:53.151543 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:53.149181    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv"
Oct 30 19:42:53.156934 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:42:53.155545    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv"
Oct 30 19:43:00.055058 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:00.055026    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="models-as-a-service/payload-processing-external-model-54696766b9-rvzmg"
Oct 30 19:43:01.173666 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:01.173636    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw"]
Oct 30 19:43:01.178628 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:01.178604    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw"
Oct 30 19:43:01.197316 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:01.197282    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw"]
Oct 30 19:43:01.199332 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:43:01.199298    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 19:43:01.289091 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:01.289054    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e41df835-fccd-45bb-bfca-ca1f81f6261a-credential-socket\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw\" (UID: \"e41df835-fccd-45bb-bfca-ca1f81f6261a\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw"
Oct 30 19:43:01.289266 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:01.289121    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e41df835-fccd-45bb-bfca-ca1f81f6261a-workload-certs\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw\" (UID: \"e41df835-fccd-45bb-bfca-ca1f81f6261a\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw"
Oct 30 19:43:01.289266 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:01.289177    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e41df835-fccd-45bb-bfca-ca1f81f6261a-istio-data\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw\" (UID: \"e41df835-fccd-45bb-bfca-ca1f81f6261a\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw"
Oct 30 19:43:01.289348 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:01.289312    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qq6t\" (UniqueName: \"kubernetes.io/projected/e41df835-fccd-45bb-bfca-ca1f81f6261a-kube-api-access-9qq6t\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw\" (UID: \"e41df835-fccd-45bb-bfca-ca1f81f6261a\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw"
Oct 30 19:43:01.289348 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:01.289340    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e41df835-fccd-45bb-bfca-ca1f81f6261a-workload-socket\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw\" (UID: \"e41df835-fccd-45bb-bfca-ca1f81f6261a\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw"
Oct 30 19:43:01.289412 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:01.289357    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e41df835-fccd-45bb-bfca-ca1f81f6261a-istio-token\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw\" (UID: \"e41df835-fccd-45bb-bfca-ca1f81f6261a\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw"
Oct 30 19:43:01.289445 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:01.289413    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e41df835-fccd-45bb-bfca-ca1f81f6261a-istio-envoy\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw\" (UID: \"e41df835-fccd-45bb-bfca-ca1f81f6261a\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw"
Oct 30 19:43:01.289509 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:01.289472    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e41df835-fccd-45bb-bfca-ca1f81f6261a-istio-podinfo\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw\" (UID: \"e41df835-fccd-45bb-bfca-ca1f81f6261a\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw"
Oct 30 19:43:01.289562 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:01.289520    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e41df835-fccd-45bb-bfca-ca1f81f6261a-istiod-ca-cert\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw\" (UID: \"e41df835-fccd-45bb-bfca-ca1f81f6261a\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw"
Oct 30 19:43:01.390499 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:01.390446    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e41df835-fccd-45bb-bfca-ca1f81f6261a-istiod-ca-cert\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw\" (UID: \"e41df835-fccd-45bb-bfca-ca1f81f6261a\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw"
Oct 30 19:43:01.390709 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:01.390564    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e41df835-fccd-45bb-bfca-ca1f81f6261a-credential-socket\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw\" (UID: \"e41df835-fccd-45bb-bfca-ca1f81f6261a\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw"
Oct 30 19:43:01.390709 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:01.390633    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e41df835-fccd-45bb-bfca-ca1f81f6261a-workload-certs\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw\" (UID: \"e41df835-fccd-45bb-bfca-ca1f81f6261a\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw"
Oct 30 19:43:01.390709 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:01.390647    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e41df835-fccd-45bb-bfca-ca1f81f6261a-istio-data\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw\" (UID: \"e41df835-fccd-45bb-bfca-ca1f81f6261a\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw"
Oct 30 19:43:01.390893 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:01.390719    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qq6t\" (UniqueName: \"kubernetes.io/projected/e41df835-fccd-45bb-bfca-ca1f81f6261a-kube-api-access-9qq6t\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw\" (UID: \"e41df835-fccd-45bb-bfca-ca1f81f6261a\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw"
Oct 30 19:43:01.390893 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:01.390740    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e41df835-fccd-45bb-bfca-ca1f81f6261a-workload-socket\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw\" (UID: \"e41df835-fccd-45bb-bfca-ca1f81f6261a\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw"
Oct 30 19:43:01.390893 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:01.390758    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e41df835-fccd-45bb-bfca-ca1f81f6261a-istio-token\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw\" (UID: \"e41df835-fccd-45bb-bfca-ca1f81f6261a\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw"
Oct 30 19:43:01.390893 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:01.390788    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e41df835-fccd-45bb-bfca-ca1f81f6261a-istio-envoy\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw\" (UID: \"e41df835-fccd-45bb-bfca-ca1f81f6261a\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw"
Oct 30 19:43:01.390893 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:01.390809    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e41df835-fccd-45bb-bfca-ca1f81f6261a-istio-podinfo\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw\" (UID: \"e41df835-fccd-45bb-bfca-ca1f81f6261a\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw"
Oct 30 19:43:01.391169 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:01.391017    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e41df835-fccd-45bb-bfca-ca1f81f6261a-credential-socket\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw\" (UID: \"e41df835-fccd-45bb-bfca-ca1f81f6261a\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw"
Oct 30 19:43:01.391169 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:01.391112    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e41df835-fccd-45bb-bfca-ca1f81f6261a-workload-certs\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw\" (UID: \"e41df835-fccd-45bb-bfca-ca1f81f6261a\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw"
Oct 30 19:43:01.391169 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:01.391129    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e41df835-fccd-45bb-bfca-ca1f81f6261a-istio-data\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw\" (UID: \"e41df835-fccd-45bb-bfca-ca1f81f6261a\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw"
Oct 30 19:43:01.391334 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:01.391172    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e41df835-fccd-45bb-bfca-ca1f81f6261a-workload-socket\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw\" (UID: \"e41df835-fccd-45bb-bfca-ca1f81f6261a\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw"
Oct 30 19:43:01.391390 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:01.391343    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e41df835-fccd-45bb-bfca-ca1f81f6261a-istiod-ca-cert\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw\" (UID: \"e41df835-fccd-45bb-bfca-ca1f81f6261a\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw"
Oct 30 19:43:01.393128 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:01.393105    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e41df835-fccd-45bb-bfca-ca1f81f6261a-istio-envoy\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw\" (UID: \"e41df835-fccd-45bb-bfca-ca1f81f6261a\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw"
Oct 30 19:43:01.393294 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:01.393276    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e41df835-fccd-45bb-bfca-ca1f81f6261a-istio-podinfo\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw\" (UID: \"e41df835-fccd-45bb-bfca-ca1f81f6261a\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw"
Oct 30 19:43:01.405851 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:01.405785    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qq6t\" (UniqueName: \"kubernetes.io/projected/e41df835-fccd-45bb-bfca-ca1f81f6261a-kube-api-access-9qq6t\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw\" (UID: \"e41df835-fccd-45bb-bfca-ca1f81f6261a\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw"
Oct 30 19:43:01.407169 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:01.407136    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e41df835-fccd-45bb-bfca-ca1f81f6261a-istio-token\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw\" (UID: \"e41df835-fccd-45bb-bfca-ca1f81f6261a\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw"
Oct 30 19:43:01.495359 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:01.495270    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw"
Oct 30 19:43:01.794540 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:01.794510    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw"]
Oct 30 19:43:01.796854 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:43:01.796828    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode41df835_fccd_45bb_bfca_ca1f81f6261a.slice/crio-fc82f523aecfe9c5effea5085c752db1f0891affe4674a3e6b2c298a2f596d06 WatchSource:0}: Error finding container fc82f523aecfe9c5effea5085c752db1f0891affe4674a3e6b2c298a2f596d06: Status 404 returned error can't find the container with id fc82f523aecfe9c5effea5085c752db1f0891affe4674a3e6b2c298a2f596d06
Oct 30 19:43:01.799252 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:01.799222    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:43:01.799328 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:01.799290    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:43:01.799328 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:01.799320    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:43:02.698821 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:02.698786    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw" event={"ID":"e41df835-fccd-45bb-bfca-ca1f81f6261a","Type":"ContainerStarted","Data":"e79c87eafd6a76089707fac19fee20748a19a2eb6ef2509076ec6e6be096c035"}
Oct 30 19:43:02.699271 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:02.698843    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw" event={"ID":"e41df835-fccd-45bb-bfca-ca1f81f6261a","Type":"ContainerStarted","Data":"fc82f523aecfe9c5effea5085c752db1f0891affe4674a3e6b2c298a2f596d06"}
Oct 30 19:43:02.724737 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:02.724339    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw" podStartSLOduration=1.724320144 podStartE2EDuration="1.724320144s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:43:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:43:02.723658738 +0000 UTC m=+658.051776892" watchObservedRunningTime="2026-10-30 19:43:02.724320144 +0000 UTC m=+658.052438311"
Oct 30 19:43:03.495886 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:03.495847    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw"
Oct 30 19:43:03.497880 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:03.497851    6996 patch_prober.go:29] interesting pod/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Oct 30 19:43:03.498049 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:03.497899    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw" podUID="e41df835-fccd-45bb-bfca-ca1f81f6261a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Oct 30 19:43:04.496525 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:04.496469    6996 patch_prober.go:29] interesting pod/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Oct 30 19:43:04.496939 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:04.496542    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw" podUID="e41df835-fccd-45bb-bfca-ca1f81f6261a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Oct 30 19:43:05.496113 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:05.496076    6996 patch_prober.go:29] interesting pod/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Oct 30 19:43:05.496275 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:05.496140    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw" podUID="e41df835-fccd-45bb-bfca-ca1f81f6261a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Oct 30 19:43:06.496257 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:06.496220    6996 patch_prober.go:29] interesting pod/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Oct 30 19:43:06.496706 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:06.496287    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw" podUID="e41df835-fccd-45bb-bfca-ca1f81f6261a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Oct 30 19:43:07.495725 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:07.495683    6996 patch_prober.go:29] interesting pod/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Oct 30 19:43:07.495905 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:07.495751    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw" podUID="e41df835-fccd-45bb-bfca-ca1f81f6261a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Oct 30 19:43:08.773055 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:08.773015    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 19:43:09.169279 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:09.169244    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw"
Oct 30 19:43:09.170186 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:09.170164    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw"
Oct 30 19:43:09.172902 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:09.172877    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw"
Oct 30 19:43:11.487568 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:11.487529    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp"]
Oct 30 19:43:11.512508 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:11.505256    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp"
Oct 30 19:43:11.649503 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:11.634032    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf35d600-7eac-42ff-b1f9-79ad561fea3c-workload-certs\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp\" (UID: \"cf35d600-7eac-42ff-b1f9-79ad561fea3c\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp"
Oct 30 19:43:11.653502 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:11.649852    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf35d600-7eac-42ff-b1f9-79ad561fea3c-credential-socket\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp\" (UID: \"cf35d600-7eac-42ff-b1f9-79ad561fea3c\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp"
Oct 30 19:43:11.659597 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:11.659563    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf35d600-7eac-42ff-b1f9-79ad561fea3c-istio-token\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp\" (UID: \"cf35d600-7eac-42ff-b1f9-79ad561fea3c\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp"
Oct 30 19:43:11.659776 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:11.659757    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf35d600-7eac-42ff-b1f9-79ad561fea3c-istiod-ca-cert\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp\" (UID: \"cf35d600-7eac-42ff-b1f9-79ad561fea3c\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp"
Oct 30 19:43:11.682651 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:11.650914    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-8836a8-openshift-default-dockercfg-dvhtk\""
Oct 30 19:43:11.687542 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:11.665566    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf35d600-7eac-42ff-b1f9-79ad561fea3c-istio-envoy\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp\" (UID: \"cf35d600-7eac-42ff-b1f9-79ad561fea3c\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp"
Oct 30 19:43:11.687805 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:11.687783    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf35d600-7eac-42ff-b1f9-79ad561fea3c-istio-data\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp\" (UID: \"cf35d600-7eac-42ff-b1f9-79ad561fea3c\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp"
Oct 30 19:43:11.695752 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:11.695712    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf35d600-7eac-42ff-b1f9-79ad561fea3c-workload-socket\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp\" (UID: \"cf35d600-7eac-42ff-b1f9-79ad561fea3c\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp"
Oct 30 19:43:11.703620 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:11.703581    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf35d600-7eac-42ff-b1f9-79ad561fea3c-istio-podinfo\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp\" (UID: \"cf35d600-7eac-42ff-b1f9-79ad561fea3c\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp"
Oct 30 19:43:11.703942 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:11.703919    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vkgw\" (UniqueName: \"kubernetes.io/projected/cf35d600-7eac-42ff-b1f9-79ad561fea3c-kube-api-access-7vkgw\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp\" (UID: \"cf35d600-7eac-42ff-b1f9-79ad561fea3c\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp"
Oct 30 19:43:11.853758 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:11.853713    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp"]
Oct 30 19:43:11.870360 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:11.870316    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf35d600-7eac-42ff-b1f9-79ad561fea3c-workload-socket\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp\" (UID: \"cf35d600-7eac-42ff-b1f9-79ad561fea3c\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp"
Oct 30 19:43:11.870682 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:11.870656    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf35d600-7eac-42ff-b1f9-79ad561fea3c-istio-podinfo\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp\" (UID: \"cf35d600-7eac-42ff-b1f9-79ad561fea3c\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp"
Oct 30 19:43:11.871012 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:11.870987    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vkgw\" (UniqueName: \"kubernetes.io/projected/cf35d600-7eac-42ff-b1f9-79ad561fea3c-kube-api-access-7vkgw\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp\" (UID: \"cf35d600-7eac-42ff-b1f9-79ad561fea3c\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp"
Oct 30 19:43:11.871301 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:11.871278    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf35d600-7eac-42ff-b1f9-79ad561fea3c-workload-certs\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp\" (UID: \"cf35d600-7eac-42ff-b1f9-79ad561fea3c\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp"
Oct 30 19:43:11.984191 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:11.984152    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf35d600-7eac-42ff-b1f9-79ad561fea3c-credential-socket\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp\" (UID: \"cf35d600-7eac-42ff-b1f9-79ad561fea3c\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp"
Oct 30 19:43:11.984492 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:11.984454    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf35d600-7eac-42ff-b1f9-79ad561fea3c-istio-token\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp\" (UID: \"cf35d600-7eac-42ff-b1f9-79ad561fea3c\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp"
Oct 30 19:43:11.984645 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:11.984627    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf35d600-7eac-42ff-b1f9-79ad561fea3c-istiod-ca-cert\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp\" (UID: \"cf35d600-7eac-42ff-b1f9-79ad561fea3c\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp"
Oct 30 19:43:11.984766 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:11.984751    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf35d600-7eac-42ff-b1f9-79ad561fea3c-istio-envoy\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp\" (UID: \"cf35d600-7eac-42ff-b1f9-79ad561fea3c\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp"
Oct 30 19:43:11.984872 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:11.984859    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf35d600-7eac-42ff-b1f9-79ad561fea3c-istio-data\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp\" (UID: \"cf35d600-7eac-42ff-b1f9-79ad561fea3c\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp"
Oct 30 19:43:11.986849 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:11.872391    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf35d600-7eac-42ff-b1f9-79ad561fea3c-workload-socket\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp\" (UID: \"cf35d600-7eac-42ff-b1f9-79ad561fea3c\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp"
Oct 30 19:43:11.987043 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:11.986628    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf35d600-7eac-42ff-b1f9-79ad561fea3c-istiod-ca-cert\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp\" (UID: \"cf35d600-7eac-42ff-b1f9-79ad561fea3c\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp"
Oct 30 19:43:11.987315 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:11.986896    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf35d600-7eac-42ff-b1f9-79ad561fea3c-credential-socket\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp\" (UID: \"cf35d600-7eac-42ff-b1f9-79ad561fea3c\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp"
Oct 30 19:43:11.987432 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:11.875319    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf35d600-7eac-42ff-b1f9-79ad561fea3c-workload-certs\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp\" (UID: \"cf35d600-7eac-42ff-b1f9-79ad561fea3c\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp"
Oct 30 19:43:12.028821 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:11.918957    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf35d600-7eac-42ff-b1f9-79ad561fea3c-istio-podinfo\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp\" (UID: \"cf35d600-7eac-42ff-b1f9-79ad561fea3c\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp"
Oct 30 19:43:12.029186 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:11.968113    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vkgw\" (UniqueName: \"kubernetes.io/projected/cf35d600-7eac-42ff-b1f9-79ad561fea3c-kube-api-access-7vkgw\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp\" (UID: \"cf35d600-7eac-42ff-b1f9-79ad561fea3c\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp"
Oct 30 19:43:12.055990 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:12.055944    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf35d600-7eac-42ff-b1f9-79ad561fea3c-istio-data\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp\" (UID: \"cf35d600-7eac-42ff-b1f9-79ad561fea3c\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp"
Oct 30 19:43:12.109560 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:12.096307    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf35d600-7eac-42ff-b1f9-79ad561fea3c-istio-token\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp\" (UID: \"cf35d600-7eac-42ff-b1f9-79ad561fea3c\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp"
Oct 30 19:43:12.109560 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:12.102891    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf35d600-7eac-42ff-b1f9-79ad561fea3c-istio-envoy\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp\" (UID: \"cf35d600-7eac-42ff-b1f9-79ad561fea3c\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp"
Oct 30 19:43:12.158630 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:12.158591    6996 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5db77fbd9f-2rj4k container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:43:12.158909 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:12.158884    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-2rj4k" podUID="8d0ef765-f9f8-4f8a-82ad-0a766a663782" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:43:12.283641 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:12.283602    6996 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5db77fbd9f-sbbjg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:43:12.284218 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:12.283894    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sbbjg" podUID="81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:43:12.337963 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:12.337927    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp"
Oct 30 19:43:13.112304 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:13.111789    6996 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5db77fbd9f-5jtlw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:43:13.112304 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:13.111876    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-5jtlw" podUID="abe1772a-c94e-4dd9-b023-176bab5081d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:43:13.220942 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:13.219992    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp"]
Oct 30 19:43:13.225339 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:43:13.224809    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf35d600_7eac_42ff_b1f9_79ad561fea3c.slice/crio-64fc659b35a5efe5a4097d5ad969dd34b8875b692baed9ab052ca851320943a2 WatchSource:0}: Error finding container 64fc659b35a5efe5a4097d5ad969dd34b8875b692baed9ab052ca851320943a2: Status 404 returned error can't find the container with id 64fc659b35a5efe5a4097d5ad969dd34b8875b692baed9ab052ca851320943a2
Oct 30 19:43:13.227934 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:13.227894    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:43:13.228154 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:13.228127    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:43:13.228303 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:13.228283    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:43:14.068991 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:14.068958    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp" event={"ID":"cf35d600-7eac-42ff-b1f9-79ad561fea3c","Type":"ContainerStarted","Data":"61e17e881b70d1f754c6a22b9c7fd84ea43951fd13fae2deb934ddf05866a1c0"}
Oct 30 19:43:14.069187 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:14.069015    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp" event={"ID":"cf35d600-7eac-42ff-b1f9-79ad561fea3c","Type":"ContainerStarted","Data":"64fc659b35a5efe5a4097d5ad969dd34b8875b692baed9ab052ca851320943a2"}
Oct 30 19:43:14.097492 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:14.097433    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp" podStartSLOduration=3.097418372 podStartE2EDuration="3.097418372s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:43:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:43:14.094236046 +0000 UTC m=+669.422354202" watchObservedRunningTime="2026-10-30 19:43:14.097418372 +0000 UTC m=+669.425536525"
Oct 30 19:43:14.357563 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:14.357035    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp"
Oct 30 19:43:14.361663 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:14.361630    6996 patch_prober.go:29] interesting pod/e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Oct 30 19:43:14.361849 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:14.361687    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp" podUID="cf35d600-7eac-42ff-b1f9-79ad561fea3c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Oct 30 19:43:15.348505 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:15.347581    6996 patch_prober.go:29] interesting pod/e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Oct 30 19:43:15.348505 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:15.347661    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp" podUID="cf35d600-7eac-42ff-b1f9-79ad561fea3c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Oct 30 19:43:16.361868 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:16.361832    6996 patch_prober.go:29] interesting pod/e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Oct 30 19:43:16.362270 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:16.361887    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp" podUID="cf35d600-7eac-42ff-b1f9-79ad561fea3c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Oct 30 19:43:17.355504 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:17.345869    6996 patch_prober.go:29] interesting pod/e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Oct 30 19:43:17.355504 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:17.345936    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp" podUID="cf35d600-7eac-42ff-b1f9-79ad561fea3c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Oct 30 19:43:18.352037 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:18.345899    6996 patch_prober.go:29] interesting pod/e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Oct 30 19:43:18.352037 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:18.345975    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp" podUID="cf35d600-7eac-42ff-b1f9-79ad561fea3c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Oct 30 19:43:19.339647 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:19.339608    6996 patch_prober.go:29] interesting pod/e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Oct 30 19:43:19.339916 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:19.339672    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp" podUID="cf35d600-7eac-42ff-b1f9-79ad561fea3c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Oct 30 19:43:20.344571 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:20.344538    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp"
Oct 30 19:43:20.345068 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:20.344902    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp"
Oct 30 19:43:20.345548 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:20.345527    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp"
Oct 30 19:43:27.671658 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:27.671624    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-54696766b9-rvzmg"]
Oct 30 19:43:30.324078 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:30.324039    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-8836a8/e2e-conflict-b-05cbc882-kserve-59789d99bd-6szds"]
Oct 30 19:43:30.328717 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:30.328692    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-8836a8/e2e-conflict-b-05cbc882-kserve-59789d99bd-6szds"
Oct 30 19:43:30.340656 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:30.340628    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-8836a8\"/\"default-dockercfg-vp47g\""
Oct 30 19:43:30.341029 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:30.341005    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-8836a8\"/\"e2e-conflict-b-05cbc882-kserve-self-signed-certs\""
Oct 30 19:43:30.341288 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:30.341231    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-8836a8\"/\"openshift-service-ca.crt\""
Oct 30 19:43:30.341529 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:30.341513    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-8836a8\"/\"kube-root-ca.crt\""
Oct 30 19:43:30.343773 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:30.343753    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-8836a8/e2e-conflict-b-05cbc882-kserve-59789d99bd-6szds"]
Oct 30 19:43:30.433017 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:30.432981    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/58231be8-6245-4753-8edf-73ce5b916784-dshm\") pod \"e2e-conflict-b-05cbc882-kserve-59789d99bd-6szds\" (UID: \"58231be8-6245-4753-8edf-73ce5b916784\") " pod="e2e-models-e2e-worker-w0-8836a8/e2e-conflict-b-05cbc882-kserve-59789d99bd-6szds"
Oct 30 19:43:30.433170 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:30.433033    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/58231be8-6245-4753-8edf-73ce5b916784-tmp-dir\") pod \"e2e-conflict-b-05cbc882-kserve-59789d99bd-6szds\" (UID: \"58231be8-6245-4753-8edf-73ce5b916784\") " pod="e2e-models-e2e-worker-w0-8836a8/e2e-conflict-b-05cbc882-kserve-59789d99bd-6szds"
Oct 30 19:43:30.433170 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:30.433146    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wz6cx\" (UniqueName: \"kubernetes.io/projected/58231be8-6245-4753-8edf-73ce5b916784-kube-api-access-wz6cx\") pod \"e2e-conflict-b-05cbc882-kserve-59789d99bd-6szds\" (UID: \"58231be8-6245-4753-8edf-73ce5b916784\") " pod="e2e-models-e2e-worker-w0-8836a8/e2e-conflict-b-05cbc882-kserve-59789d99bd-6szds"
Oct 30 19:43:30.433244 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:30.433198    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/58231be8-6245-4753-8edf-73ce5b916784-home\") pod \"e2e-conflict-b-05cbc882-kserve-59789d99bd-6szds\" (UID: \"58231be8-6245-4753-8edf-73ce5b916784\") " pod="e2e-models-e2e-worker-w0-8836a8/e2e-conflict-b-05cbc882-kserve-59789d99bd-6szds"
Oct 30 19:43:30.433284 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:30.433259    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/58231be8-6245-4753-8edf-73ce5b916784-tls-certs\") pod \"e2e-conflict-b-05cbc882-kserve-59789d99bd-6szds\" (UID: \"58231be8-6245-4753-8edf-73ce5b916784\") " pod="e2e-models-e2e-worker-w0-8836a8/e2e-conflict-b-05cbc882-kserve-59789d99bd-6szds"
Oct 30 19:43:30.433318 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:30.433308    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/58231be8-6245-4753-8edf-73ce5b916784-model-cache\") pod \"e2e-conflict-b-05cbc882-kserve-59789d99bd-6szds\" (UID: \"58231be8-6245-4753-8edf-73ce5b916784\") " pod="e2e-models-e2e-worker-w0-8836a8/e2e-conflict-b-05cbc882-kserve-59789d99bd-6szds"
Oct 30 19:43:30.534161 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:30.534122    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/58231be8-6245-4753-8edf-73ce5b916784-model-cache\") pod \"e2e-conflict-b-05cbc882-kserve-59789d99bd-6szds\" (UID: \"58231be8-6245-4753-8edf-73ce5b916784\") " pod="e2e-models-e2e-worker-w0-8836a8/e2e-conflict-b-05cbc882-kserve-59789d99bd-6szds"
Oct 30 19:43:30.534385 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:30.534188    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/58231be8-6245-4753-8edf-73ce5b916784-dshm\") pod \"e2e-conflict-b-05cbc882-kserve-59789d99bd-6szds\" (UID: \"58231be8-6245-4753-8edf-73ce5b916784\") " pod="e2e-models-e2e-worker-w0-8836a8/e2e-conflict-b-05cbc882-kserve-59789d99bd-6szds"
Oct 30 19:43:30.534385 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:30.534226    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/58231be8-6245-4753-8edf-73ce5b916784-tmp-dir\") pod \"e2e-conflict-b-05cbc882-kserve-59789d99bd-6szds\" (UID: \"58231be8-6245-4753-8edf-73ce5b916784\") " pod="e2e-models-e2e-worker-w0-8836a8/e2e-conflict-b-05cbc882-kserve-59789d99bd-6szds"
Oct 30 19:43:30.534385 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:30.534298    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wz6cx\" (UniqueName: \"kubernetes.io/projected/58231be8-6245-4753-8edf-73ce5b916784-kube-api-access-wz6cx\") pod \"e2e-conflict-b-05cbc882-kserve-59789d99bd-6szds\" (UID: \"58231be8-6245-4753-8edf-73ce5b916784\") " pod="e2e-models-e2e-worker-w0-8836a8/e2e-conflict-b-05cbc882-kserve-59789d99bd-6szds"
Oct 30 19:43:30.534385 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:30.534358    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/58231be8-6245-4753-8edf-73ce5b916784-home\") pod \"e2e-conflict-b-05cbc882-kserve-59789d99bd-6szds\" (UID: \"58231be8-6245-4753-8edf-73ce5b916784\") " pod="e2e-models-e2e-worker-w0-8836a8/e2e-conflict-b-05cbc882-kserve-59789d99bd-6szds"
Oct 30 19:43:30.534625 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:30.534403    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/58231be8-6245-4753-8edf-73ce5b916784-tls-certs\") pod \"e2e-conflict-b-05cbc882-kserve-59789d99bd-6szds\" (UID: \"58231be8-6245-4753-8edf-73ce5b916784\") " pod="e2e-models-e2e-worker-w0-8836a8/e2e-conflict-b-05cbc882-kserve-59789d99bd-6szds"
Oct 30 19:43:30.535102 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:30.535077    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/58231be8-6245-4753-8edf-73ce5b916784-tmp-dir\") pod \"e2e-conflict-b-05cbc882-kserve-59789d99bd-6szds\" (UID: \"58231be8-6245-4753-8edf-73ce5b916784\") " pod="e2e-models-e2e-worker-w0-8836a8/e2e-conflict-b-05cbc882-kserve-59789d99bd-6szds"
Oct 30 19:43:30.536055 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:30.535406    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/58231be8-6245-4753-8edf-73ce5b916784-model-cache\") pod \"e2e-conflict-b-05cbc882-kserve-59789d99bd-6szds\" (UID: \"58231be8-6245-4753-8edf-73ce5b916784\") " pod="e2e-models-e2e-worker-w0-8836a8/e2e-conflict-b-05cbc882-kserve-59789d99bd-6szds"
Oct 30 19:43:30.536055 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:30.535666    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/58231be8-6245-4753-8edf-73ce5b916784-home\") pod \"e2e-conflict-b-05cbc882-kserve-59789d99bd-6szds\" (UID: \"58231be8-6245-4753-8edf-73ce5b916784\") " pod="e2e-models-e2e-worker-w0-8836a8/e2e-conflict-b-05cbc882-kserve-59789d99bd-6szds"
Oct 30 19:43:30.537790 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:30.537766    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/58231be8-6245-4753-8edf-73ce5b916784-dshm\") pod \"e2e-conflict-b-05cbc882-kserve-59789d99bd-6szds\" (UID: \"58231be8-6245-4753-8edf-73ce5b916784\") " pod="e2e-models-e2e-worker-w0-8836a8/e2e-conflict-b-05cbc882-kserve-59789d99bd-6szds"
Oct 30 19:43:30.537891 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:30.537805    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/58231be8-6245-4753-8edf-73ce5b916784-tls-certs\") pod \"e2e-conflict-b-05cbc882-kserve-59789d99bd-6szds\" (UID: \"58231be8-6245-4753-8edf-73ce5b916784\") " pod="e2e-models-e2e-worker-w0-8836a8/e2e-conflict-b-05cbc882-kserve-59789d99bd-6szds"
Oct 30 19:43:30.547403 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:30.547375    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wz6cx\" (UniqueName: \"kubernetes.io/projected/58231be8-6245-4753-8edf-73ce5b916784-kube-api-access-wz6cx\") pod \"e2e-conflict-b-05cbc882-kserve-59789d99bd-6szds\" (UID: \"58231be8-6245-4753-8edf-73ce5b916784\") " pod="e2e-models-e2e-worker-w0-8836a8/e2e-conflict-b-05cbc882-kserve-59789d99bd-6szds"
Oct 30 19:43:30.656234 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:30.656146    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-8836a8/e2e-conflict-b-05cbc882-kserve-59789d99bd-6szds"
Oct 30 19:43:30.939672 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:30.939645    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-8836a8/e2e-conflict-b-05cbc882-kserve-59789d99bd-6szds"]
Oct 30 19:43:30.941542 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:43:30.941469    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod58231be8_6245_4753_8edf_73ce5b916784.slice/crio-e8d80ca2fa35f0c3d3a22d748fa161b6be47e6e0c8a123f4dbd8bd0c73973c10 WatchSource:0}: Error finding container e8d80ca2fa35f0c3d3a22d748fa161b6be47e6e0c8a123f4dbd8bd0c73973c10: Status 404 returned error can't find the container with id e8d80ca2fa35f0c3d3a22d748fa161b6be47e6e0c8a123f4dbd8bd0c73973c10
Oct 30 19:43:31.196183 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:43:31.196082    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 19:43:31.393367 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:31.393336    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-8836a8/e2e-conflict-b-05cbc882-kserve-59789d99bd-6szds" event={"ID":"58231be8-6245-4753-8edf-73ce5b916784","Type":"ContainerStarted","Data":"6057a34a65b4385138ce2d98e3a3b544c27c303593eb6beb9e23dd4202f857d9"}
Oct 30 19:43:31.393831 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:31.393375    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-8836a8/e2e-conflict-b-05cbc882-kserve-59789d99bd-6szds" event={"ID":"58231be8-6245-4753-8edf-73ce5b916784","Type":"ContainerStarted","Data":"e8d80ca2fa35f0c3d3a22d748fa161b6be47e6e0c8a123f4dbd8bd0c73973c10"}
Oct 30 19:43:31.447722 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:31.447612    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-8836a8/e2e-conflict-b-05cbc882-kserve-59789d99bd-6szds" podStartSLOduration=1.4475931260000001 podStartE2EDuration="1.447593126s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:43:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:43:31.446040306 +0000 UTC m=+686.774158472" watchObservedRunningTime="2026-10-30 19:43:31.447593126 +0000 UTC m=+686.775711281"
Oct 30 19:43:31.656633 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:31.656600    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-8836a8/e2e-conflict-b-05cbc882-kserve-59789d99bd-6szds"
Oct 30 19:43:32.672629 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:32.672575    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="models-as-a-service/payload-processing-external-model-54696766b9-rvzmg" podUID="4e5665dc-3c3a-49e4-83de-cd514a209c0b" containerName="payload-processing" containerID="cri-o://c5aa6e021717d768f91c3f90cd95eb6af951462e7bbba07ecebe1a0608992dc6" gracePeriod=25
Oct 30 19:43:32.944120 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:32.944096    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-54696766b9-rvzmg"
Oct 30 19:43:33.064582 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:33.064546    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4e5665dc-3c3a-49e4-83de-cd514a209c0b-provider-credentials\" (UniqueName: \"kubernetes.io/projected/4e5665dc-3c3a-49e4-83de-cd514a209c0b-provider-credentials\") pod \"4e5665dc-3c3a-49e4-83de-cd514a209c0b\" (UID: \"4e5665dc-3c3a-49e4-83de-cd514a209c0b\") "
Oct 30 19:43:33.064767 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:33.064608    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4e5665dc-3c3a-49e4-83de-cd514a209c0b-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/4e5665dc-3c3a-49e4-83de-cd514a209c0b-routing-overlay-volume\") pod \"4e5665dc-3c3a-49e4-83de-cd514a209c0b\" (UID: \"4e5665dc-3c3a-49e4-83de-cd514a209c0b\") "
Oct 30 19:43:33.064767 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:33.064735    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4e5665dc-3c3a-49e4-83de-cd514a209c0b-kube-api-access-sdzbf\" (UniqueName: \"kubernetes.io/projected/4e5665dc-3c3a-49e4-83de-cd514a209c0b-kube-api-access-sdzbf\") pod \"4e5665dc-3c3a-49e4-83de-cd514a209c0b\" (UID: \"4e5665dc-3c3a-49e4-83de-cd514a209c0b\") "
Oct 30 19:43:33.064911 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:33.064891    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4e5665dc-3c3a-49e4-83de-cd514a209c0b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4e5665dc-3c3a-49e4-83de-cd514a209c0b-plugins-config-volume\") pod \"4e5665dc-3c3a-49e4-83de-cd514a209c0b\" (UID: \"4e5665dc-3c3a-49e4-83de-cd514a209c0b\") "
Oct 30 19:43:33.065091 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:33.065050    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e5665dc-3c3a-49e4-83de-cd514a209c0b-routing-overlay-volume" pod "4e5665dc-3c3a-49e4-83de-cd514a209c0b" (UID: "4e5665dc-3c3a-49e4-83de-cd514a209c0b"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:43:33.065566 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:33.065417    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e5665dc-3c3a-49e4-83de-cd514a209c0b-plugins-config-volume" pod "4e5665dc-3c3a-49e4-83de-cd514a209c0b" (UID: "4e5665dc-3c3a-49e4-83de-cd514a209c0b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:43:33.065566 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:33.065540    6996 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4e5665dc-3c3a-49e4-83de-cd514a209c0b-plugins-config-volume\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:43:33.065566 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:33.065565    6996 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/4e5665dc-3c3a-49e4-83de-cd514a209c0b-routing-overlay-volume\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:43:33.067224 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:33.067193    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e5665dc-3c3a-49e4-83de-cd514a209c0b-kube-api-access-sdzbf" pod "4e5665dc-3c3a-49e4-83de-cd514a209c0b" (UID: "4e5665dc-3c3a-49e4-83de-cd514a209c0b"). InnerVolumeSpecName "kube-api-access-sdzbf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:33.067336 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:33.067315    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e5665dc-3c3a-49e4-83de-cd514a209c0b-provider-credentials" pod "4e5665dc-3c3a-49e4-83de-cd514a209c0b" (UID: "4e5665dc-3c3a-49e4-83de-cd514a209c0b"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:33.166605 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:33.166572    6996 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/4e5665dc-3c3a-49e4-83de-cd514a209c0b-provider-credentials\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:43:33.166605 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:33.166608    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sdzbf\" (UniqueName: \"kubernetes.io/projected/4e5665dc-3c3a-49e4-83de-cd514a209c0b-kube-api-access-sdzbf\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:43:33.415191 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:33.415159    6996 generic.go:356] "Generic (PLEG): container finished" podID="4e5665dc-3c3a-49e4-83de-cd514a209c0b" containerID="c5aa6e021717d768f91c3f90cd95eb6af951462e7bbba07ecebe1a0608992dc6" exitCode=0
Oct 30 19:43:33.415373 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:33.415219    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-54696766b9-rvzmg"
Oct 30 19:43:33.415373 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:33.415240    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-54696766b9-rvzmg" event={"ID":"4e5665dc-3c3a-49e4-83de-cd514a209c0b","Type":"ContainerDied","Data":"c5aa6e021717d768f91c3f90cd95eb6af951462e7bbba07ecebe1a0608992dc6"}
Oct 30 19:43:33.415373 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:33.415326    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-54696766b9-rvzmg" event={"ID":"4e5665dc-3c3a-49e4-83de-cd514a209c0b","Type":"ContainerDied","Data":"ef5af5d32f61cf8c749167337f460274c55b0106bcd298c4d6332ffb6bb1acc5"}
Oct 30 19:43:33.415373 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:33.415355    6996 scope.go:122] "RemoveContainer" containerID="c5aa6e021717d768f91c3f90cd95eb6af951462e7bbba07ecebe1a0608992dc6"
Oct 30 19:43:33.425564 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:33.425544    6996 scope.go:122] "RemoveContainer" containerID="c5aa6e021717d768f91c3f90cd95eb6af951462e7bbba07ecebe1a0608992dc6"
Oct 30 19:43:33.425851 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:43:33.425832    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5aa6e021717d768f91c3f90cd95eb6af951462e7bbba07ecebe1a0608992dc6\": container with ID starting with c5aa6e021717d768f91c3f90cd95eb6af951462e7bbba07ecebe1a0608992dc6 not found: ID does not exist" containerID="c5aa6e021717d768f91c3f90cd95eb6af951462e7bbba07ecebe1a0608992dc6"
Oct 30 19:43:33.425895 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:33.425859    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5aa6e021717d768f91c3f90cd95eb6af951462e7bbba07ecebe1a0608992dc6"} err="failed to get container status \"c5aa6e021717d768f91c3f90cd95eb6af951462e7bbba07ecebe1a0608992dc6\": rpc error: code = NotFound desc = could not find container \"c5aa6e021717d768f91c3f90cd95eb6af951462e7bbba07ecebe1a0608992dc6\": container with ID starting with c5aa6e021717d768f91c3f90cd95eb6af951462e7bbba07ecebe1a0608992dc6 not found: ID does not exist"
Oct 30 19:43:33.439005 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:33.438977    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-54696766b9-rvzmg"]
Oct 30 19:43:33.444220 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:33.444199    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["models-as-a-service/payload-processing-external-model-54696766b9-rvzmg"]
Oct 30 19:43:34.793095 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:34.793056    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4e5665dc-3c3a-49e4-83de-cd514a209c0b" path="/var/lib/kubelet/pods/4e5665dc-3c3a-49e4-83de-cd514a209c0b/volumes"
Oct 30 19:43:35.898984 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:35.898943    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 19:43:38.450243 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:38.450209    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["models-as-a-service/payload-processing-external-model-646458976c-x8jhm"]
Oct 30 19:43:38.450790 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:38.450774    6996 cpu_manager.go:414] "containerMap: removing container" podUID="4e5665dc-3c3a-49e4-83de-cd514a209c0b" containerName="payload-processing"
Oct 30 19:43:38.450836 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:38.450791    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4e5665dc-3c3a-49e4-83de-cd514a209c0b" containerName="payload-processing"
Oct 30 19:43:38.450917 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:38.450907    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="4e5665dc-3c3a-49e4-83de-cd514a209c0b" containerName="payload-processing"
Oct 30 19:43:38.455896 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:38.455880    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-646458976c-x8jhm"
Oct 30 19:43:38.464315 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:38.464291    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"payload-processing-external-model-dockercfg-8p64v\""
Oct 30 19:43:38.464315 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:38.464293    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"agc-cred-077d59be6edd\""
Oct 30 19:43:38.464547 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:38.464515    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"payload-processing-external-model-plugins\""
Oct 30 19:43:38.464604 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:38.464577    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"kube-root-ca.crt\""
Oct 30 19:43:38.464604 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:38.464581    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"routing-overlay\""
Oct 30 19:43:38.464709 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:38.464658    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"openshift-service-ca.crt\""
Oct 30 19:43:38.467359 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:38.467339    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-646458976c-x8jhm"]
Oct 30 19:43:38.534731 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:38.534692    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/1cedc683-f566-4e92-b59c-1efcc56a7738-routing-overlay-volume\") pod \"payload-processing-external-model-646458976c-x8jhm\" (UID: \"1cedc683-f566-4e92-b59c-1efcc56a7738\") " pod="models-as-a-service/payload-processing-external-model-646458976c-x8jhm"
Oct 30 19:43:38.534891 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:38.534799    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1cedc683-f566-4e92-b59c-1efcc56a7738-plugins-config-volume\") pod \"payload-processing-external-model-646458976c-x8jhm\" (UID: \"1cedc683-f566-4e92-b59c-1efcc56a7738\") " pod="models-as-a-service/payload-processing-external-model-646458976c-x8jhm"
Oct 30 19:43:38.534937 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:38.534899    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/1cedc683-f566-4e92-b59c-1efcc56a7738-provider-credentials\") pod \"payload-processing-external-model-646458976c-x8jhm\" (UID: \"1cedc683-f566-4e92-b59c-1efcc56a7738\") " pod="models-as-a-service/payload-processing-external-model-646458976c-x8jhm"
Oct 30 19:43:38.534979 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:38.534959    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2wkg\" (UniqueName: \"kubernetes.io/projected/1cedc683-f566-4e92-b59c-1efcc56a7738-kube-api-access-r2wkg\") pod \"payload-processing-external-model-646458976c-x8jhm\" (UID: \"1cedc683-f566-4e92-b59c-1efcc56a7738\") " pod="models-as-a-service/payload-processing-external-model-646458976c-x8jhm"
Oct 30 19:43:38.635897 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:38.635866    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/1cedc683-f566-4e92-b59c-1efcc56a7738-routing-overlay-volume\") pod \"payload-processing-external-model-646458976c-x8jhm\" (UID: \"1cedc683-f566-4e92-b59c-1efcc56a7738\") " pod="models-as-a-service/payload-processing-external-model-646458976c-x8jhm"
Oct 30 19:43:38.636056 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:38.635931    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1cedc683-f566-4e92-b59c-1efcc56a7738-plugins-config-volume\") pod \"payload-processing-external-model-646458976c-x8jhm\" (UID: \"1cedc683-f566-4e92-b59c-1efcc56a7738\") " pod="models-as-a-service/payload-processing-external-model-646458976c-x8jhm"
Oct 30 19:43:38.636056 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:38.636003    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/1cedc683-f566-4e92-b59c-1efcc56a7738-provider-credentials\") pod \"payload-processing-external-model-646458976c-x8jhm\" (UID: \"1cedc683-f566-4e92-b59c-1efcc56a7738\") " pod="models-as-a-service/payload-processing-external-model-646458976c-x8jhm"
Oct 30 19:43:38.636056 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:38.636018    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2wkg\" (UniqueName: \"kubernetes.io/projected/1cedc683-f566-4e92-b59c-1efcc56a7738-kube-api-access-r2wkg\") pod \"payload-processing-external-model-646458976c-x8jhm\" (UID: \"1cedc683-f566-4e92-b59c-1efcc56a7738\") " pod="models-as-a-service/payload-processing-external-model-646458976c-x8jhm"
Oct 30 19:43:38.636770 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:38.636725    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/1cedc683-f566-4e92-b59c-1efcc56a7738-routing-overlay-volume\") pod \"payload-processing-external-model-646458976c-x8jhm\" (UID: \"1cedc683-f566-4e92-b59c-1efcc56a7738\") " pod="models-as-a-service/payload-processing-external-model-646458976c-x8jhm"
Oct 30 19:43:38.636882 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:38.636846    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1cedc683-f566-4e92-b59c-1efcc56a7738-plugins-config-volume\") pod \"payload-processing-external-model-646458976c-x8jhm\" (UID: \"1cedc683-f566-4e92-b59c-1efcc56a7738\") " pod="models-as-a-service/payload-processing-external-model-646458976c-x8jhm"
Oct 30 19:43:38.638401 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:38.638378    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/1cedc683-f566-4e92-b59c-1efcc56a7738-provider-credentials\") pod \"payload-processing-external-model-646458976c-x8jhm\" (UID: \"1cedc683-f566-4e92-b59c-1efcc56a7738\") " pod="models-as-a-service/payload-processing-external-model-646458976c-x8jhm"
Oct 30 19:43:38.654177 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:38.654146    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2wkg\" (UniqueName: \"kubernetes.io/projected/1cedc683-f566-4e92-b59c-1efcc56a7738-kube-api-access-r2wkg\") pod \"payload-processing-external-model-646458976c-x8jhm\" (UID: \"1cedc683-f566-4e92-b59c-1efcc56a7738\") " pod="models-as-a-service/payload-processing-external-model-646458976c-x8jhm"
Oct 30 19:43:38.770350 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:38.770251    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-646458976c-x8jhm"
Oct 30 19:43:39.063385 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:39.063359    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-646458976c-x8jhm"]
Oct 30 19:43:39.065769 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:43:39.065739    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cedc683_f566_4e92_b59c_1efcc56a7738.slice/crio-0e25c67bdad3e1a819720adadee1e1e98a068ec1792f0baf9524676fb299067d WatchSource:0}: Error finding container 0e25c67bdad3e1a819720adadee1e1e98a068ec1792f0baf9524676fb299067d: Status 404 returned error can't find the container with id 0e25c67bdad3e1a819720adadee1e1e98a068ec1792f0baf9524676fb299067d
Oct 30 19:43:39.457529 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:39.457431    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-646458976c-x8jhm" event={"ID":"1cedc683-f566-4e92-b59c-1efcc56a7738","Type":"ContainerStarted","Data":"469a5be560f41bf6a4f42686e2fa418cb98eb661102055afdc45f7c8e010457d"}
Oct 30 19:43:39.457529 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:39.457515    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-646458976c-x8jhm" event={"ID":"1cedc683-f566-4e92-b59c-1efcc56a7738","Type":"ContainerStarted","Data":"0e25c67bdad3e1a819720adadee1e1e98a068ec1792f0baf9524676fb299067d"}
Oct 30 19:43:39.458171 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:39.457624    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="models-as-a-service/payload-processing-external-model-646458976c-x8jhm"
Oct 30 19:43:39.484920 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:39.484876    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="models-as-a-service/payload-processing-external-model-646458976c-x8jhm" podStartSLOduration=1.4848631669999999 podStartE2EDuration="1.484863167s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:43:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:43:39.482617772 +0000 UTC m=+694.810735925" watchObservedRunningTime="2026-10-30 19:43:39.484863167 +0000 UTC m=+694.812981321"
Oct 30 19:43:39.838362 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:39.838327    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b"]
Oct 30 19:43:39.842933 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:39.842907    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b"
Oct 30 19:43:39.848249 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:39.848224    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19fe4fba-09ae-4620-b4d2-0ef8c2463d71-dshm\") pod \"distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b\" (UID: \"19fe4fba-09ae-4620-b4d2-0ef8c2463d71\") " pod="e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b"
Oct 30 19:43:39.848651 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:39.848627    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77mb5\" (UniqueName: \"kubernetes.io/projected/19fe4fba-09ae-4620-b4d2-0ef8c2463d71-kube-api-access-77mb5\") pod \"distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b\" (UID: \"19fe4fba-09ae-4620-b4d2-0ef8c2463d71\") " pod="e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b"
Oct 30 19:43:39.848751 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:39.848702    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19fe4fba-09ae-4620-b4d2-0ef8c2463d71-tls-certs\") pod \"distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b\" (UID: \"19fe4fba-09ae-4620-b4d2-0ef8c2463d71\") " pod="e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b"
Oct 30 19:43:39.848816 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:39.848787    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19fe4fba-09ae-4620-b4d2-0ef8c2463d71-home\") pod \"distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b\" (UID: \"19fe4fba-09ae-4620-b4d2-0ef8c2463d71\") " pod="e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b"
Oct 30 19:43:39.848870 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:39.848831    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19fe4fba-09ae-4620-b4d2-0ef8c2463d71-tmp-dir\") pod \"distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b\" (UID: \"19fe4fba-09ae-4620-b4d2-0ef8c2463d71\") " pod="e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b"
Oct 30 19:43:39.848969 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:39.848949    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19fe4fba-09ae-4620-b4d2-0ef8c2463d71-model-cache\") pod \"distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b\" (UID: \"19fe4fba-09ae-4620-b4d2-0ef8c2463d71\") " pod="e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b"
Oct 30 19:43:39.851909 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:39.851890    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-c1fdd6\"/\"default-dockercfg-qq4v5\""
Oct 30 19:43:39.852022 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:39.851975    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-c1fdd6\"/\"distinct-2-w3-c1fdd6-kserve-self-signed-certs\""
Oct 30 19:43:39.852276 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:39.852257    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-c1fdd6\"/\"kube-root-ca.crt\""
Oct 30 19:43:39.852405 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:39.852393    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-c1fdd6\"/\"openshift-service-ca.crt\""
Oct 30 19:43:39.854844 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:39.854818    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b"]
Oct 30 19:43:39.950576 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:39.950537    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77mb5\" (UniqueName: \"kubernetes.io/projected/19fe4fba-09ae-4620-b4d2-0ef8c2463d71-kube-api-access-77mb5\") pod \"distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b\" (UID: \"19fe4fba-09ae-4620-b4d2-0ef8c2463d71\") " pod="e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b"
Oct 30 19:43:39.950817 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:39.950595    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19fe4fba-09ae-4620-b4d2-0ef8c2463d71-tls-certs\") pod \"distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b\" (UID: \"19fe4fba-09ae-4620-b4d2-0ef8c2463d71\") " pod="e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b"
Oct 30 19:43:39.950817 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:39.950638    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19fe4fba-09ae-4620-b4d2-0ef8c2463d71-home\") pod \"distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b\" (UID: \"19fe4fba-09ae-4620-b4d2-0ef8c2463d71\") " pod="e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b"
Oct 30 19:43:39.950817 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:39.950664    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19fe4fba-09ae-4620-b4d2-0ef8c2463d71-tmp-dir\") pod \"distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b\" (UID: \"19fe4fba-09ae-4620-b4d2-0ef8c2463d71\") " pod="e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b"
Oct 30 19:43:39.950817 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:39.950720    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19fe4fba-09ae-4620-b4d2-0ef8c2463d71-model-cache\") pod \"distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b\" (UID: \"19fe4fba-09ae-4620-b4d2-0ef8c2463d71\") " pod="e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b"
Oct 30 19:43:39.950817 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:39.950758    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19fe4fba-09ae-4620-b4d2-0ef8c2463d71-dshm\") pod \"distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b\" (UID: \"19fe4fba-09ae-4620-b4d2-0ef8c2463d71\") " pod="e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b"
Oct 30 19:43:39.951114 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:39.951047    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19fe4fba-09ae-4620-b4d2-0ef8c2463d71-home\") pod \"distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b\" (UID: \"19fe4fba-09ae-4620-b4d2-0ef8c2463d71\") " pod="e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b"
Oct 30 19:43:39.951170 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:39.951116    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19fe4fba-09ae-4620-b4d2-0ef8c2463d71-tmp-dir\") pod \"distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b\" (UID: \"19fe4fba-09ae-4620-b4d2-0ef8c2463d71\") " pod="e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b"
Oct 30 19:43:39.951170 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:39.951156    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19fe4fba-09ae-4620-b4d2-0ef8c2463d71-model-cache\") pod \"distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b\" (UID: \"19fe4fba-09ae-4620-b4d2-0ef8c2463d71\") " pod="e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b"
Oct 30 19:43:39.952946 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:39.952924    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19fe4fba-09ae-4620-b4d2-0ef8c2463d71-dshm\") pod \"distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b\" (UID: \"19fe4fba-09ae-4620-b4d2-0ef8c2463d71\") " pod="e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b"
Oct 30 19:43:39.953236 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:39.953215    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19fe4fba-09ae-4620-b4d2-0ef8c2463d71-tls-certs\") pod \"distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b\" (UID: \"19fe4fba-09ae-4620-b4d2-0ef8c2463d71\") " pod="e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b"
Oct 30 19:43:39.963035 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:39.963005    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-77mb5\" (UniqueName: \"kubernetes.io/projected/19fe4fba-09ae-4620-b4d2-0ef8c2463d71-kube-api-access-77mb5\") pod \"distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b\" (UID: \"19fe4fba-09ae-4620-b4d2-0ef8c2463d71\") " pod="e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b"
Oct 30 19:43:40.157807 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:40.157734    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b"
Oct 30 19:43:40.443818 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:40.443784    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b"]
Oct 30 19:43:40.445296 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:43:40.445264    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod19fe4fba_09ae_4620_b4d2_0ef8c2463d71.slice/crio-fa233be42d378d8c68f922d4cfd16c7b0945bff4699e72dbdce82a4e077327e5 WatchSource:0}: Error finding container fa233be42d378d8c68f922d4cfd16c7b0945bff4699e72dbdce82a4e077327e5: Status 404 returned error can't find the container with id fa233be42d378d8c68f922d4cfd16c7b0945bff4699e72dbdce82a4e077327e5
Oct 30 19:43:40.470997 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:40.470958    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b" event={"ID":"19fe4fba-09ae-4620-b4d2-0ef8c2463d71","Type":"ContainerStarted","Data":"fa233be42d378d8c68f922d4cfd16c7b0945bff4699e72dbdce82a4e077327e5"}
Oct 30 19:43:40.656495 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:40.656449    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-8836a8/e2e-conflict-b-05cbc882-kserve-59789d99bd-6szds"
Oct 30 19:43:40.669415 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:40.669393    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-8836a8/e2e-conflict-b-05cbc882-kserve-59789d99bd-6szds"
Oct 30 19:43:41.481787 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:41.481725    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b" event={"ID":"19fe4fba-09ae-4620-b4d2-0ef8c2463d71","Type":"ContainerStarted","Data":"8b1c7515b4a1b8e30a0d3ffb924e6c9741ab6f63e2beb13342b9cc7ca91763a3"}
Oct 30 19:43:41.496109 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:41.496073    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-8836a8/e2e-conflict-b-05cbc882-kserve-59789d99bd-6szds"
Oct 30 19:43:41.512449 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:41.512386    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b" podStartSLOduration=2.512372856 podStartE2EDuration="2.512372856s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:43:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:43:41.508992707 +0000 UTC m=+696.837110862" watchObservedRunningTime="2026-10-30 19:43:41.512372856 +0000 UTC m=+696.840491009"
Oct 30 19:43:42.158587 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:42.158548    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b"
Oct 30 19:43:50.178503 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:50.160625    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b"
Oct 30 19:43:50.537779 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:50.537682    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="models-as-a-service/payload-processing-external-model-646458976c-x8jhm"
Oct 30 19:43:51.030823 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:51.030791    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b"
Oct 30 19:43:51.726815 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:51.726782    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b"
Oct 30 19:43:53.850530 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:53.850494    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-8836a8/e2e-conflict-b-05cbc882-kserve-59789d99bd-6szds"]
Oct 30 19:43:56.564254 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:56.564213    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg"]
Oct 30 19:43:56.568677 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:56.568654    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg"
Oct 30 19:43:56.590749 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:56.590710    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg"]
Oct 30 19:43:56.621817 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:56.621784    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw"]
Oct 30 19:43:56.626919 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:56.626897    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw"
Oct 30 19:43:56.644035 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:56.644005    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw"]
Oct 30 19:43:56.669928 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:56.669897    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d1b1d3b-0290-4c57-9a3e-1afab986cb98-istio-envoy\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg\" (UID: \"0d1b1d3b-0290-4c57-9a3e-1afab986cb98\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg"
Oct 30 19:43:56.670081 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:56.669946    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8z2kh\" (UniqueName: \"kubernetes.io/projected/0d1b1d3b-0290-4c57-9a3e-1afab986cb98-kube-api-access-8z2kh\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg\" (UID: \"0d1b1d3b-0290-4c57-9a3e-1afab986cb98\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg"
Oct 30 19:43:56.670081 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:56.670045    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d1b1d3b-0290-4c57-9a3e-1afab986cb98-credential-socket\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg\" (UID: \"0d1b1d3b-0290-4c57-9a3e-1afab986cb98\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg"
Oct 30 19:43:56.670081 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:56.670061    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d1b1d3b-0290-4c57-9a3e-1afab986cb98-istiod-ca-cert\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg\" (UID: \"0d1b1d3b-0290-4c57-9a3e-1afab986cb98\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg"
Oct 30 19:43:56.670267 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:56.670146    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d1b1d3b-0290-4c57-9a3e-1afab986cb98-istio-data\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg\" (UID: \"0d1b1d3b-0290-4c57-9a3e-1afab986cb98\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg"
Oct 30 19:43:56.670267 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:56.670173    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d1b1d3b-0290-4c57-9a3e-1afab986cb98-istio-podinfo\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg\" (UID: \"0d1b1d3b-0290-4c57-9a3e-1afab986cb98\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg"
Oct 30 19:43:56.670267 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:56.670263    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d1b1d3b-0290-4c57-9a3e-1afab986cb98-istio-token\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg\" (UID: \"0d1b1d3b-0290-4c57-9a3e-1afab986cb98\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg"
Oct 30 19:43:56.670425 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:56.670310    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d1b1d3b-0290-4c57-9a3e-1afab986cb98-workload-socket\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg\" (UID: \"0d1b1d3b-0290-4c57-9a3e-1afab986cb98\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg"
Oct 30 19:43:56.670500 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:56.670432    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d1b1d3b-0290-4c57-9a3e-1afab986cb98-workload-certs\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg\" (UID: \"0d1b1d3b-0290-4c57-9a3e-1afab986cb98\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg"
Oct 30 19:43:56.772076 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:56.772047    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8-istio-data\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw\" (UID: \"ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw"
Oct 30 19:43:56.772216 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:56.772086    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8z2kh\" (UniqueName: \"kubernetes.io/projected/0d1b1d3b-0290-4c57-9a3e-1afab986cb98-kube-api-access-8z2kh\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg\" (UID: \"0d1b1d3b-0290-4c57-9a3e-1afab986cb98\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg"
Oct 30 19:43:56.772259 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:56.772217    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d1b1d3b-0290-4c57-9a3e-1afab986cb98-credential-socket\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg\" (UID: \"0d1b1d3b-0290-4c57-9a3e-1afab986cb98\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg"
Oct 30 19:43:56.772259 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:56.772248    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d1b1d3b-0290-4c57-9a3e-1afab986cb98-istiod-ca-cert\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg\" (UID: \"0d1b1d3b-0290-4c57-9a3e-1afab986cb98\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg"
Oct 30 19:43:56.772324 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:56.772311    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d1b1d3b-0290-4c57-9a3e-1afab986cb98-istio-data\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg\" (UID: \"0d1b1d3b-0290-4c57-9a3e-1afab986cb98\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg"
Oct 30 19:43:56.772408 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:56.772393    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d1b1d3b-0290-4c57-9a3e-1afab986cb98-istio-podinfo\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg\" (UID: \"0d1b1d3b-0290-4c57-9a3e-1afab986cb98\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg"
Oct 30 19:43:56.772451 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:56.772426    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qmrc\" (UniqueName: \"kubernetes.io/projected/ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8-kube-api-access-7qmrc\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw\" (UID: \"ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw"
Oct 30 19:43:56.772520 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:56.772464    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8-workload-socket\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw\" (UID: \"ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw"
Oct 30 19:43:56.772617 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:56.772595    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d1b1d3b-0290-4c57-9a3e-1afab986cb98-credential-socket\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg\" (UID: \"0d1b1d3b-0290-4c57-9a3e-1afab986cb98\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg"
Oct 30 19:43:56.772660 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:56.772598    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8-istio-token\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw\" (UID: \"ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw"
Oct 30 19:43:56.772660 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:56.772645    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d1b1d3b-0290-4c57-9a3e-1afab986cb98-istio-data\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg\" (UID: \"0d1b1d3b-0290-4c57-9a3e-1afab986cb98\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg"
Oct 30 19:43:56.772742 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:56.772664    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d1b1d3b-0290-4c57-9a3e-1afab986cb98-istio-token\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg\" (UID: \"0d1b1d3b-0290-4c57-9a3e-1afab986cb98\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg"
Oct 30 19:43:56.772742 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:56.772696    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8-istio-podinfo\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw\" (UID: \"ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw"
Oct 30 19:43:56.772839 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:56.772814    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8-istiod-ca-cert\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw\" (UID: \"ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw"
Oct 30 19:43:56.772892 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:56.772866    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8-istio-envoy\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw\" (UID: \"ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw"
Oct 30 19:43:56.772954 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:56.772901    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8-workload-certs\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw\" (UID: \"ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw"
Oct 30 19:43:56.772954 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:56.772908    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d1b1d3b-0290-4c57-9a3e-1afab986cb98-istiod-ca-cert\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg\" (UID: \"0d1b1d3b-0290-4c57-9a3e-1afab986cb98\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg"
Oct 30 19:43:56.773059 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:56.773007    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d1b1d3b-0290-4c57-9a3e-1afab986cb98-workload-socket\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg\" (UID: \"0d1b1d3b-0290-4c57-9a3e-1afab986cb98\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg"
Oct 30 19:43:56.773114 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:56.773074    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d1b1d3b-0290-4c57-9a3e-1afab986cb98-workload-certs\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg\" (UID: \"0d1b1d3b-0290-4c57-9a3e-1afab986cb98\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg"
Oct 30 19:43:56.773114 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:56.773105    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d1b1d3b-0290-4c57-9a3e-1afab986cb98-istio-envoy\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg\" (UID: \"0d1b1d3b-0290-4c57-9a3e-1afab986cb98\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg"
Oct 30 19:43:56.773228 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:56.773132    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8-credential-socket\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw\" (UID: \"ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw"
Oct 30 19:43:56.773367 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:56.773345    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d1b1d3b-0290-4c57-9a3e-1afab986cb98-workload-socket\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg\" (UID: \"0d1b1d3b-0290-4c57-9a3e-1afab986cb98\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg"
Oct 30 19:43:56.773452 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:56.773425    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d1b1d3b-0290-4c57-9a3e-1afab986cb98-workload-certs\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg\" (UID: \"0d1b1d3b-0290-4c57-9a3e-1afab986cb98\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg"
Oct 30 19:43:56.775071 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:56.775050    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d1b1d3b-0290-4c57-9a3e-1afab986cb98-istio-podinfo\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg\" (UID: \"0d1b1d3b-0290-4c57-9a3e-1afab986cb98\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg"
Oct 30 19:43:56.775133 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:56.775073    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d1b1d3b-0290-4c57-9a3e-1afab986cb98-istio-envoy\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg\" (UID: \"0d1b1d3b-0290-4c57-9a3e-1afab986cb98\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg"
Oct 30 19:43:56.785611 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:56.785586    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d1b1d3b-0290-4c57-9a3e-1afab986cb98-istio-token\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg\" (UID: \"0d1b1d3b-0290-4c57-9a3e-1afab986cb98\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg"
Oct 30 19:43:56.786631 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:56.786608    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8z2kh\" (UniqueName: \"kubernetes.io/projected/0d1b1d3b-0290-4c57-9a3e-1afab986cb98-kube-api-access-8z2kh\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg\" (UID: \"0d1b1d3b-0290-4c57-9a3e-1afab986cb98\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg"
Oct 30 19:43:56.874030 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:56.873996    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8-istio-token\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw\" (UID: \"ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw"
Oct 30 19:43:56.874188 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:56.874040    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8-istio-podinfo\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw\" (UID: \"ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw"
Oct 30 19:43:56.874188 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:56.874079    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8-istiod-ca-cert\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw\" (UID: \"ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw"
Oct 30 19:43:56.874188 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:56.874105    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8-istio-envoy\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw\" (UID: \"ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw"
Oct 30 19:43:56.874188 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:56.874129    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8-workload-certs\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw\" (UID: \"ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw"
Oct 30 19:43:56.874417 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:56.874227    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8-credential-socket\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw\" (UID: \"ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw"
Oct 30 19:43:56.874417 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:56.874274    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8-istio-data\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw\" (UID: \"ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw"
Oct 30 19:43:56.874561 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:56.874414    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qmrc\" (UniqueName: \"kubernetes.io/projected/ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8-kube-api-access-7qmrc\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw\" (UID: \"ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw"
Oct 30 19:43:56.874561 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:56.874450    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8-workload-socket\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw\" (UID: \"ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw"
Oct 30 19:43:56.874670 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:56.874622    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8-workload-certs\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw\" (UID: \"ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw"
Oct 30 19:43:56.874832 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:56.874811    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8-credential-socket\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw\" (UID: \"ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw"
Oct 30 19:43:56.874899 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:56.874826    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8-workload-socket\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw\" (UID: \"ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw"
Oct 30 19:43:56.874899 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:56.874862    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8-istio-data\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw\" (UID: \"ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw"
Oct 30 19:43:56.874974 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:56.874863    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8-istiod-ca-cert\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw\" (UID: \"ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw"
Oct 30 19:43:56.876314 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:56.876297    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8-istio-envoy\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw\" (UID: \"ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw"
Oct 30 19:43:56.876912 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:56.876893    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8-istio-podinfo\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw\" (UID: \"ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw"
Oct 30 19:43:56.885299 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:56.885282    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg"
Oct 30 19:43:56.887370 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:56.887351    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8-istio-token\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw\" (UID: \"ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw"
Oct 30 19:43:56.892458 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:56.892437    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qmrc\" (UniqueName: \"kubernetes.io/projected/ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8-kube-api-access-7qmrc\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw\" (UID: \"ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw"
Oct 30 19:43:56.948413 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:56.945550    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw"
Oct 30 19:43:57.195988 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:57.195957    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg"]
Oct 30 19:43:57.197226 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:43:57.197202    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d1b1d3b_0290_4c57_9a3e_1afab986cb98.slice/crio-22c7aba96449a04433a0c667aeed8e394af2591dedfe4f57db48447c25e91577 WatchSource:0}: Error finding container 22c7aba96449a04433a0c667aeed8e394af2591dedfe4f57db48447c25e91577: Status 404 returned error can't find the container with id 22c7aba96449a04433a0c667aeed8e394af2591dedfe4f57db48447c25e91577
Oct 30 19:43:57.199742 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:57.199708    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:43:57.199820 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:57.199772    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:43:57.199870 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:57.199820    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:43:57.287487 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:57.287452    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw"]
Oct 30 19:43:57.303520 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:43:57.294822    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce1ed4ae_eab8_49a6_a68f_b7fe756a10a8.slice/crio-5aa91cc14c6a0a28266ecec121a0d55937f1a86dbcc8797c8df7a19024323bb9 WatchSource:0}: Error finding container 5aa91cc14c6a0a28266ecec121a0d55937f1a86dbcc8797c8df7a19024323bb9: Status 404 returned error can't find the container with id 5aa91cc14c6a0a28266ecec121a0d55937f1a86dbcc8797c8df7a19024323bb9
Oct 30 19:43:57.303520 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:57.299882    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:43:57.303520 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:57.300004    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:43:57.303520 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:57.300047    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:43:57.652105 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:57.652064    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw" event={"ID":"ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8","Type":"ContainerStarted","Data":"3794d879be1a019eab9cfaff47c6d5bb19470a96503b33abed996c7ab8f6c536"}
Oct 30 19:43:57.652580 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:57.652135    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw" event={"ID":"ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8","Type":"ContainerStarted","Data":"5aa91cc14c6a0a28266ecec121a0d55937f1a86dbcc8797c8df7a19024323bb9"}
Oct 30 19:43:57.654169 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:57.654137    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg" event={"ID":"0d1b1d3b-0290-4c57-9a3e-1afab986cb98","Type":"ContainerStarted","Data":"6fb733d4f7a8a52004c449204dace712960898006ac3adadf4084d24858885c5"}
Oct 30 19:43:57.654282 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:57.654193    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg" event={"ID":"0d1b1d3b-0290-4c57-9a3e-1afab986cb98","Type":"ContainerStarted","Data":"22c7aba96449a04433a0c667aeed8e394af2591dedfe4f57db48447c25e91577"}
Oct 30 19:43:57.685985 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:57.685933    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw" podStartSLOduration=1.685918493 podStartE2EDuration="1.685918493s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:43:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:43:57.682604533 +0000 UTC m=+713.010722687" watchObservedRunningTime="2026-10-30 19:43:57.685918493 +0000 UTC m=+713.014036647"
Oct 30 19:43:57.710559 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:57.710511    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg" podStartSLOduration=1.7104975759999999 podStartE2EDuration="1.710497576s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:43:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:43:57.708653844 +0000 UTC m=+713.036771982" watchObservedRunningTime="2026-10-30 19:43:57.710497576 +0000 UTC m=+713.038615727"
Oct 30 19:43:57.886406 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:57.886369    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg"
Oct 30 19:43:57.968561 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:57.967557    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw"
Oct 30 19:43:58.889947 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:58.889903    6996 patch_prober.go:29] interesting pod/e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Oct 30 19:43:58.890389 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:58.889973    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg" podUID="0d1b1d3b-0290-4c57-9a3e-1afab986cb98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Oct 30 19:43:58.986546 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:58.965576    6996 patch_prober.go:29] interesting pod/e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 30 19:43:58.986546 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:58.965652    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw" podUID="ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 30 19:43:59.602111 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:59.600167    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 19:43:59.887887 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:59.887782    6996 patch_prober.go:29] interesting pod/e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Oct 30 19:43:59.887887 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:59.887866    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg" podUID="0d1b1d3b-0290-4c57-9a3e-1afab986cb98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Oct 30 19:43:59.959904 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:59.959858    6996 patch_prober.go:29] interesting pod/e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 30 19:43:59.960512 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:43:59.960456    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw" podUID="ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 30 19:44:00.895526 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:00.891978    6996 patch_prober.go:29] interesting pod/e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Oct 30 19:44:00.895526 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:00.892054    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg" podUID="0d1b1d3b-0290-4c57-9a3e-1afab986cb98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Oct 30 19:44:00.959504 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:00.947682    6996 patch_prober.go:29] interesting pod/e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 30 19:44:00.959504 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:00.947758    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw" podUID="ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 30 19:44:01.437505 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:44:01.436666    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 19:44:01.918507 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:01.917945    6996 patch_prober.go:29] interesting pod/e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Oct 30 19:44:01.918507 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:01.918003    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg" podUID="0d1b1d3b-0290-4c57-9a3e-1afab986cb98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Oct 30 19:44:01.971505 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:01.967690    6996 patch_prober.go:29] interesting pod/e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 30 19:44:01.971505 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:01.967756    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw" podUID="ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 30 19:44:02.888505 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:02.887865    6996 patch_prober.go:29] interesting pod/e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Oct 30 19:44:02.888505 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:02.887924    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg" podUID="0d1b1d3b-0290-4c57-9a3e-1afab986cb98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Oct 30 19:44:02.960504 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:02.959733    6996 patch_prober.go:29] interesting pod/e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 30 19:44:02.960504 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:02.959795    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw" podUID="ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 30 19:44:03.609958 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:03.609905    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["models-as-a-service/payload-processing-external-model-54b4664b47-zpskh"]
Oct 30 19:44:03.622063 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:03.622024    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-54b4664b47-zpskh"
Oct 30 19:44:03.630319 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:03.630294    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"agc-cred-93ded87a4c64\""
Oct 30 19:44:03.632232 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:03.632202    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-54b4664b47-zpskh"]
Oct 30 19:44:03.636305 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:03.636285    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"agc-cred-ec074d41ddb3\""
Oct 30 19:44:03.749351 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:03.749315    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b06d7926-a4e2-4f55-aa9a-f8684c73bac6-plugins-config-volume\") pod \"payload-processing-external-model-54b4664b47-zpskh\" (UID: \"b06d7926-a4e2-4f55-aa9a-f8684c73bac6\") " pod="models-as-a-service/payload-processing-external-model-54b4664b47-zpskh"
Oct 30 19:44:03.749545 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:03.749509    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/b06d7926-a4e2-4f55-aa9a-f8684c73bac6-routing-overlay-volume\") pod \"payload-processing-external-model-54b4664b47-zpskh\" (UID: \"b06d7926-a4e2-4f55-aa9a-f8684c73bac6\") " pod="models-as-a-service/payload-processing-external-model-54b4664b47-zpskh"
Oct 30 19:44:03.749609 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:03.749560    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/b06d7926-a4e2-4f55-aa9a-f8684c73bac6-provider-credentials\") pod \"payload-processing-external-model-54b4664b47-zpskh\" (UID: \"b06d7926-a4e2-4f55-aa9a-f8684c73bac6\") " pod="models-as-a-service/payload-processing-external-model-54b4664b47-zpskh"
Oct 30 19:44:03.749689 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:03.749674    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77htp\" (UniqueName: \"kubernetes.io/projected/b06d7926-a4e2-4f55-aa9a-f8684c73bac6-kube-api-access-77htp\") pod \"payload-processing-external-model-54b4664b47-zpskh\" (UID: \"b06d7926-a4e2-4f55-aa9a-f8684c73bac6\") " pod="models-as-a-service/payload-processing-external-model-54b4664b47-zpskh"
Oct 30 19:44:03.850674 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:03.850642    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b06d7926-a4e2-4f55-aa9a-f8684c73bac6-plugins-config-volume\") pod \"payload-processing-external-model-54b4664b47-zpskh\" (UID: \"b06d7926-a4e2-4f55-aa9a-f8684c73bac6\") " pod="models-as-a-service/payload-processing-external-model-54b4664b47-zpskh"
Oct 30 19:44:03.850849 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:03.850818    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/b06d7926-a4e2-4f55-aa9a-f8684c73bac6-routing-overlay-volume\") pod \"payload-processing-external-model-54b4664b47-zpskh\" (UID: \"b06d7926-a4e2-4f55-aa9a-f8684c73bac6\") " pod="models-as-a-service/payload-processing-external-model-54b4664b47-zpskh"
Oct 30 19:44:03.850926 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:03.850879    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/b06d7926-a4e2-4f55-aa9a-f8684c73bac6-provider-credentials\") pod \"payload-processing-external-model-54b4664b47-zpskh\" (UID: \"b06d7926-a4e2-4f55-aa9a-f8684c73bac6\") " pod="models-as-a-service/payload-processing-external-model-54b4664b47-zpskh"
Oct 30 19:44:03.851008 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:03.850989    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77htp\" (UniqueName: \"kubernetes.io/projected/b06d7926-a4e2-4f55-aa9a-f8684c73bac6-kube-api-access-77htp\") pod \"payload-processing-external-model-54b4664b47-zpskh\" (UID: \"b06d7926-a4e2-4f55-aa9a-f8684c73bac6\") " pod="models-as-a-service/payload-processing-external-model-54b4664b47-zpskh"
Oct 30 19:44:03.851574 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:03.851546    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b06d7926-a4e2-4f55-aa9a-f8684c73bac6-plugins-config-volume\") pod \"payload-processing-external-model-54b4664b47-zpskh\" (UID: \"b06d7926-a4e2-4f55-aa9a-f8684c73bac6\") " pod="models-as-a-service/payload-processing-external-model-54b4664b47-zpskh"
Oct 30 19:44:03.851750 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:03.851728    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/b06d7926-a4e2-4f55-aa9a-f8684c73bac6-routing-overlay-volume\") pod \"payload-processing-external-model-54b4664b47-zpskh\" (UID: \"b06d7926-a4e2-4f55-aa9a-f8684c73bac6\") " pod="models-as-a-service/payload-processing-external-model-54b4664b47-zpskh"
Oct 30 19:44:03.853973 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:03.853947    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/b06d7926-a4e2-4f55-aa9a-f8684c73bac6-provider-credentials\") pod \"payload-processing-external-model-54b4664b47-zpskh\" (UID: \"b06d7926-a4e2-4f55-aa9a-f8684c73bac6\") " pod="models-as-a-service/payload-processing-external-model-54b4664b47-zpskh"
Oct 30 19:44:03.863374 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:03.863317    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-77htp\" (UniqueName: \"kubernetes.io/projected/b06d7926-a4e2-4f55-aa9a-f8684c73bac6-kube-api-access-77htp\") pod \"payload-processing-external-model-54b4664b47-zpskh\" (UID: \"b06d7926-a4e2-4f55-aa9a-f8684c73bac6\") " pod="models-as-a-service/payload-processing-external-model-54b4664b47-zpskh"
Oct 30 19:44:03.885966 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:03.885940    6996 patch_prober.go:29] interesting pod/e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Oct 30 19:44:03.886082 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:03.885994    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg" podUID="0d1b1d3b-0290-4c57-9a3e-1afab986cb98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Oct 30 19:44:03.942119 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:03.942091    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-54b4664b47-zpskh"
Oct 30 19:44:03.946804 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:03.946775    6996 patch_prober.go:29] interesting pod/e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 30 19:44:03.947016 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:03.946982    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw" podUID="ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 30 19:44:04.236338 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:04.236312    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-54b4664b47-zpskh"]
Oct 30 19:44:04.239267 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:44:04.239231    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb06d7926_a4e2_4f55_aa9a_f8684c73bac6.slice/crio-c244ff1be998fa482d05c5d19bf6c6ca1547b59025c08a41d8b2b96340369116 WatchSource:0}: Error finding container c244ff1be998fa482d05c5d19bf6c6ca1547b59025c08a41d8b2b96340369116: Status 404 returned error can't find the container with id c244ff1be998fa482d05c5d19bf6c6ca1547b59025c08a41d8b2b96340369116
Oct 30 19:44:04.846038 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:04.845983    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-54b4664b47-zpskh" event={"ID":"b06d7926-a4e2-4f55-aa9a-f8684c73bac6","Type":"ContainerStarted","Data":"52431ece8a590f5ed7a141d956bbf289c123b6c1ba40f509db97eb7999896525"}
Oct 30 19:44:04.846242 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:04.846053    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-54b4664b47-zpskh" event={"ID":"b06d7926-a4e2-4f55-aa9a-f8684c73bac6","Type":"ContainerStarted","Data":"c244ff1be998fa482d05c5d19bf6c6ca1547b59025c08a41d8b2b96340369116"}
Oct 30 19:44:04.846242 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:04.846151    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="models-as-a-service/payload-processing-external-model-54b4664b47-zpskh"
Oct 30 19:44:04.879838 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:04.879712    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="models-as-a-service/payload-processing-external-model-54b4664b47-zpskh" podStartSLOduration=1.8796940869999998 podStartE2EDuration="1.879694087s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:44:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:44:04.879086513 +0000 UTC m=+720.207204671" watchObservedRunningTime="2026-10-30 19:44:04.879694087 +0000 UTC m=+720.207812243"
Oct 30 19:44:04.955594 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:04.955564    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg"
Oct 30 19:44:04.956072 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:04.955916    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg"
Oct 30 19:44:04.957906 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:04.957881    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg"
Oct 30 19:44:05.986656 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:05.986599    6996 patch_prober.go:29] interesting pod/e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:44:05.987167 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:05.986664    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw" podUID="ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:44:06.108570 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:06.108536    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw"
Oct 30 19:44:06.109017 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:06.108945    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw"
Oct 30 19:44:06.110076 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:06.110057    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw"
Oct 30 19:44:08.872034 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:08.871978    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-8836a8/e2e-conflict-b-05cbc882-kserve-59789d99bd-6szds" podUID="58231be8-6245-4753-8edf-73ce5b916784" containerName="main" containerID="cri-o://6057a34a65b4385138ce2d98e3a3b544c27c303593eb6beb9e23dd4202f857d9" gracePeriod=45
Oct 30 19:44:09.155322 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:09.155292    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-8836a8/e2e-conflict-b-05cbc882-kserve-59789d99bd-6szds"
Oct 30 19:44:09.224129 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:09.224087    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/58231be8-6245-4753-8edf-73ce5b916784-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/58231be8-6245-4753-8edf-73ce5b916784-tmp-dir\") pod \"58231be8-6245-4753-8edf-73ce5b916784\" (UID: \"58231be8-6245-4753-8edf-73ce5b916784\") "
Oct 30 19:44:09.224295 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:09.224241    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/58231be8-6245-4753-8edf-73ce5b916784-kube-api-access-wz6cx\" (UniqueName: \"kubernetes.io/projected/58231be8-6245-4753-8edf-73ce5b916784-kube-api-access-wz6cx\") pod \"58231be8-6245-4753-8edf-73ce5b916784\" (UID: \"58231be8-6245-4753-8edf-73ce5b916784\") "
Oct 30 19:44:09.224295 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:09.224284    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/58231be8-6245-4753-8edf-73ce5b916784-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/58231be8-6245-4753-8edf-73ce5b916784-model-cache\") pod \"58231be8-6245-4753-8edf-73ce5b916784\" (UID: \"58231be8-6245-4753-8edf-73ce5b916784\") "
Oct 30 19:44:09.224380 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:09.224333    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/58231be8-6245-4753-8edf-73ce5b916784-tls-certs\" (UniqueName: \"kubernetes.io/secret/58231be8-6245-4753-8edf-73ce5b916784-tls-certs\") pod \"58231be8-6245-4753-8edf-73ce5b916784\" (UID: \"58231be8-6245-4753-8edf-73ce5b916784\") "
Oct 30 19:44:09.224380 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:09.224335    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58231be8-6245-4753-8edf-73ce5b916784-tmp-dir" pod "58231be8-6245-4753-8edf-73ce5b916784" (UID: "58231be8-6245-4753-8edf-73ce5b916784"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:44:09.224380 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:09.224365    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/58231be8-6245-4753-8edf-73ce5b916784-dshm\" (UniqueName: \"kubernetes.io/empty-dir/58231be8-6245-4753-8edf-73ce5b916784-dshm\") pod \"58231be8-6245-4753-8edf-73ce5b916784\" (UID: \"58231be8-6245-4753-8edf-73ce5b916784\") "
Oct 30 19:44:09.224380 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:09.224380    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/58231be8-6245-4753-8edf-73ce5b916784-home\" (UniqueName: \"kubernetes.io/empty-dir/58231be8-6245-4753-8edf-73ce5b916784-home\") pod \"58231be8-6245-4753-8edf-73ce5b916784\" (UID: \"58231be8-6245-4753-8edf-73ce5b916784\") "
Oct 30 19:44:09.224583 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:09.224562    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58231be8-6245-4753-8edf-73ce5b916784-model-cache" pod "58231be8-6245-4753-8edf-73ce5b916784" (UID: "58231be8-6245-4753-8edf-73ce5b916784"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:44:09.224749 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:09.224727    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58231be8-6245-4753-8edf-73ce5b916784-home" pod "58231be8-6245-4753-8edf-73ce5b916784" (UID: "58231be8-6245-4753-8edf-73ce5b916784"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:44:09.224910 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:09.224894    6996 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/58231be8-6245-4753-8edf-73ce5b916784-model-cache\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:44:09.224979 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:09.224911    6996 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/58231be8-6245-4753-8edf-73ce5b916784-home\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:44:09.224979 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:09.224919    6996 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/58231be8-6245-4753-8edf-73ce5b916784-tmp-dir\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:44:09.226616 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:09.226588    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58231be8-6245-4753-8edf-73ce5b916784-tls-certs" pod "58231be8-6245-4753-8edf-73ce5b916784" (UID: "58231be8-6245-4753-8edf-73ce5b916784"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:44:09.226738 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:09.226629    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58231be8-6245-4753-8edf-73ce5b916784-dshm" pod "58231be8-6245-4753-8edf-73ce5b916784" (UID: "58231be8-6245-4753-8edf-73ce5b916784"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:44:09.226738 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:09.226705    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58231be8-6245-4753-8edf-73ce5b916784-kube-api-access-wz6cx" pod "58231be8-6245-4753-8edf-73ce5b916784" (UID: "58231be8-6245-4753-8edf-73ce5b916784"). InnerVolumeSpecName "kube-api-access-wz6cx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:44:09.326430 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:09.326396    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wz6cx\" (UniqueName: \"kubernetes.io/projected/58231be8-6245-4753-8edf-73ce5b916784-kube-api-access-wz6cx\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:44:09.326430 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:09.326422    6996 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/58231be8-6245-4753-8edf-73ce5b916784-tls-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:44:09.326430 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:09.326431    6996 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/58231be8-6245-4753-8edf-73ce5b916784-dshm\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:44:09.885285 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:09.885251    6996 generic.go:356] "Generic (PLEG): container finished" podID="58231be8-6245-4753-8edf-73ce5b916784" containerID="6057a34a65b4385138ce2d98e3a3b544c27c303593eb6beb9e23dd4202f857d9" exitCode=0
Oct 30 19:44:09.885760 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:09.885297    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-8836a8/e2e-conflict-b-05cbc882-kserve-59789d99bd-6szds"
Oct 30 19:44:09.885760 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:09.885337    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-8836a8/e2e-conflict-b-05cbc882-kserve-59789d99bd-6szds" event={"ID":"58231be8-6245-4753-8edf-73ce5b916784","Type":"ContainerDied","Data":"6057a34a65b4385138ce2d98e3a3b544c27c303593eb6beb9e23dd4202f857d9"}
Oct 30 19:44:09.885760 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:09.885378    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-8836a8/e2e-conflict-b-05cbc882-kserve-59789d99bd-6szds" event={"ID":"58231be8-6245-4753-8edf-73ce5b916784","Type":"ContainerDied","Data":"e8d80ca2fa35f0c3d3a22d748fa161b6be47e6e0c8a123f4dbd8bd0c73973c10"}
Oct 30 19:44:09.885760 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:09.885404    6996 scope.go:122] "RemoveContainer" containerID="6057a34a65b4385138ce2d98e3a3b544c27c303593eb6beb9e23dd4202f857d9"
Oct 30 19:44:09.896196 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:09.896180    6996 scope.go:122] "RemoveContainer" containerID="6057a34a65b4385138ce2d98e3a3b544c27c303593eb6beb9e23dd4202f857d9"
Oct 30 19:44:09.896455 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:44:09.896435    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6057a34a65b4385138ce2d98e3a3b544c27c303593eb6beb9e23dd4202f857d9\": container with ID starting with 6057a34a65b4385138ce2d98e3a3b544c27c303593eb6beb9e23dd4202f857d9 not found: ID does not exist" containerID="6057a34a65b4385138ce2d98e3a3b544c27c303593eb6beb9e23dd4202f857d9"
Oct 30 19:44:09.896532 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:09.896463    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6057a34a65b4385138ce2d98e3a3b544c27c303593eb6beb9e23dd4202f857d9"} err="failed to get container status \"6057a34a65b4385138ce2d98e3a3b544c27c303593eb6beb9e23dd4202f857d9\": rpc error: code = NotFound desc = could not find container \"6057a34a65b4385138ce2d98e3a3b544c27c303593eb6beb9e23dd4202f857d9\": container with ID starting with 6057a34a65b4385138ce2d98e3a3b544c27c303593eb6beb9e23dd4202f857d9 not found: ID does not exist"
Oct 30 19:44:09.916550 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:09.912228    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-8836a8/e2e-conflict-b-05cbc882-kserve-59789d99bd-6szds"]
Oct 30 19:44:09.919908 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:09.919889    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-8836a8/e2e-conflict-b-05cbc882-kserve-59789d99bd-6szds"]
Oct 30 19:44:10.792144 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:10.792111    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="58231be8-6245-4753-8edf-73ce5b916784" path="/var/lib/kubelet/pods/58231be8-6245-4753-8edf-73ce5b916784/volumes"
Oct 30 19:44:15.869873 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:15.869675    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="models-as-a-service/payload-processing-external-model-54b4664b47-zpskh"
Oct 30 19:44:16.586619 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:16.469162    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-646458976c-x8jhm"]
Oct 30 19:44:16.735616 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:16.720217    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5"]
Oct 30 19:44:16.742413 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:16.739209    6996 cpu_manager.go:414] "containerMap: removing container" podUID="58231be8-6245-4753-8edf-73ce5b916784" containerName="main"
Oct 30 19:44:16.742413 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:16.739247    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="58231be8-6245-4753-8edf-73ce5b916784" containerName="main"
Oct 30 19:44:16.742413 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:16.739414    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="58231be8-6245-4753-8edf-73ce5b916784" containerName="main"
Oct 30 19:44:16.853779 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:16.853674    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5"
Oct 30 19:44:16.926896 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:16.925965    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5"]
Oct 30 19:44:16.964670 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:16.964621    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s" podUID="ae905dfb-f30d-480a-899b-ea676abc1d8d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:44:17.002506 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:16.995393    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zk67\" (UniqueName: \"kubernetes.io/projected/7938d051-21e1-4dfc-8d60-a3162c8398c1-kube-api-access-6zk67\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5\" (UID: \"7938d051-21e1-4dfc-8d60-a3162c8398c1\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5"
Oct 30 19:44:17.002506 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:16.995537    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7938d051-21e1-4dfc-8d60-a3162c8398c1-istio-podinfo\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5\" (UID: \"7938d051-21e1-4dfc-8d60-a3162c8398c1\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5"
Oct 30 19:44:17.002506 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:16.995583    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7938d051-21e1-4dfc-8d60-a3162c8398c1-istio-data\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5\" (UID: \"7938d051-21e1-4dfc-8d60-a3162c8398c1\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5"
Oct 30 19:44:17.002506 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:16.995624    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7938d051-21e1-4dfc-8d60-a3162c8398c1-workload-certs\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5\" (UID: \"7938d051-21e1-4dfc-8d60-a3162c8398c1\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5"
Oct 30 19:44:17.002506 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:16.995705    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7938d051-21e1-4dfc-8d60-a3162c8398c1-credential-socket\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5\" (UID: \"7938d051-21e1-4dfc-8d60-a3162c8398c1\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5"
Oct 30 19:44:17.002506 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:16.995783    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7938d051-21e1-4dfc-8d60-a3162c8398c1-workload-socket\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5\" (UID: \"7938d051-21e1-4dfc-8d60-a3162c8398c1\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5"
Oct 30 19:44:17.002506 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:16.995811    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7938d051-21e1-4dfc-8d60-a3162c8398c1-istio-envoy\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5\" (UID: \"7938d051-21e1-4dfc-8d60-a3162c8398c1\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5"
Oct 30 19:44:17.002506 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:16.995884    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7938d051-21e1-4dfc-8d60-a3162c8398c1-istiod-ca-cert\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5\" (UID: \"7938d051-21e1-4dfc-8d60-a3162c8398c1\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5"
Oct 30 19:44:17.002506 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:16.995932    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7938d051-21e1-4dfc-8d60-a3162c8398c1-istio-token\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5\" (UID: \"7938d051-21e1-4dfc-8d60-a3162c8398c1\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5"
Oct 30 19:44:17.077506 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:17.074567    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr" podUID="0e470010-625d-443f-81ec-9bf965988212" containerName="main" probeResult="failure" output="Get \"https://10.134.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:44:17.087851 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:17.087807    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b" podUID="19fe4fba-09ae-4620-b4d2-0ef8c2463d71" containerName="main" probeResult="failure" output="Get \"https://10.134.0.81:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:44:17.098135 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:17.098095    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zk67\" (UniqueName: \"kubernetes.io/projected/7938d051-21e1-4dfc-8d60-a3162c8398c1-kube-api-access-6zk67\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5\" (UID: \"7938d051-21e1-4dfc-8d60-a3162c8398c1\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5"
Oct 30 19:44:17.113509 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:17.113399    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7938d051-21e1-4dfc-8d60-a3162c8398c1-istio-podinfo\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5\" (UID: \"7938d051-21e1-4dfc-8d60-a3162c8398c1\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5"
Oct 30 19:44:17.114903 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:17.114878    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7938d051-21e1-4dfc-8d60-a3162c8398c1-istio-data\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5\" (UID: \"7938d051-21e1-4dfc-8d60-a3162c8398c1\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5"
Oct 30 19:44:17.115112 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:17.115091    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7938d051-21e1-4dfc-8d60-a3162c8398c1-workload-certs\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5\" (UID: \"7938d051-21e1-4dfc-8d60-a3162c8398c1\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5"
Oct 30 19:44:17.115250 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:17.115235    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7938d051-21e1-4dfc-8d60-a3162c8398c1-credential-socket\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5\" (UID: \"7938d051-21e1-4dfc-8d60-a3162c8398c1\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5"
Oct 30 19:44:17.115413 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:17.115398    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7938d051-21e1-4dfc-8d60-a3162c8398c1-workload-socket\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5\" (UID: \"7938d051-21e1-4dfc-8d60-a3162c8398c1\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5"
Oct 30 19:44:17.115547 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:17.115529    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7938d051-21e1-4dfc-8d60-a3162c8398c1-istio-envoy\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5\" (UID: \"7938d051-21e1-4dfc-8d60-a3162c8398c1\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5"
Oct 30 19:44:17.115722 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:17.115704    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7938d051-21e1-4dfc-8d60-a3162c8398c1-istiod-ca-cert\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5\" (UID: \"7938d051-21e1-4dfc-8d60-a3162c8398c1\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5"
Oct 30 19:44:17.117906 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:17.117884    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7938d051-21e1-4dfc-8d60-a3162c8398c1-istio-token\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5\" (UID: \"7938d051-21e1-4dfc-8d60-a3162c8398c1\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5"
Oct 30 19:44:17.118556 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:17.116362    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7938d051-21e1-4dfc-8d60-a3162c8398c1-istio-data\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5\" (UID: \"7938d051-21e1-4dfc-8d60-a3162c8398c1\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5"
Oct 30 19:44:17.118754 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:17.116610    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7938d051-21e1-4dfc-8d60-a3162c8398c1-workload-certs\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5\" (UID: \"7938d051-21e1-4dfc-8d60-a3162c8398c1\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5"
Oct 30 19:44:17.118862 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:17.116883    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7938d051-21e1-4dfc-8d60-a3162c8398c1-workload-socket\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5\" (UID: \"7938d051-21e1-4dfc-8d60-a3162c8398c1\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5"
Oct 30 19:44:17.122571 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:17.117810    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7938d051-21e1-4dfc-8d60-a3162c8398c1-istiod-ca-cert\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5\" (UID: \"7938d051-21e1-4dfc-8d60-a3162c8398c1\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5"
Oct 30 19:44:17.122726 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:17.116136    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7938d051-21e1-4dfc-8d60-a3162c8398c1-credential-socket\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5\" (UID: \"7938d051-21e1-4dfc-8d60-a3162c8398c1\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5"
Oct 30 19:44:17.150577 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:17.150544    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zk67\" (UniqueName: \"kubernetes.io/projected/7938d051-21e1-4dfc-8d60-a3162c8398c1-kube-api-access-6zk67\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5\" (UID: \"7938d051-21e1-4dfc-8d60-a3162c8398c1\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5"
Oct 30 19:44:17.173915 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:17.173873    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7938d051-21e1-4dfc-8d60-a3162c8398c1-istio-envoy\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5\" (UID: \"7938d051-21e1-4dfc-8d60-a3162c8398c1\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5"
Oct 30 19:44:17.187506 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:17.180308    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7938d051-21e1-4dfc-8d60-a3162c8398c1-istio-token\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5\" (UID: \"7938d051-21e1-4dfc-8d60-a3162c8398c1\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5"
Oct 30 19:44:17.187506 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:17.185606    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7938d051-21e1-4dfc-8d60-a3162c8398c1-istio-podinfo\") pod \"e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5\" (UID: \"7938d051-21e1-4dfc-8d60-a3162c8398c1\") " pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5"
Oct 30 19:44:17.280763 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:17.280725    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5"
Oct 30 19:44:18.039557 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:18.035751    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s" podUID="ae905dfb-f30d-480a-899b-ea676abc1d8d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.73:8000/ready\": context deadline exceeded"
Oct 30 19:44:18.129816 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:18.065006    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s"
Oct 30 19:44:18.186530 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:18.150837    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b" podUID="19fe4fba-09ae-4620-b4d2-0ef8c2463d71" containerName="main" probeResult="failure" output="Get \"https://10.134.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:44:18.605555 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:18.268354    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:44:18.605555 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:18.553657    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:44:18.605555 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:18.553726    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:44:18.605555 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:18.457013    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5"]
Oct 30 19:44:18.605555 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:18.554345    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b"
Oct 30 19:44:19.110637 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:19.110591    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s" podUID="ae905dfb-f30d-480a-899b-ea676abc1d8d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.73:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:44:19.540595 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:19.538154    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5" event={"ID":"7938d051-21e1-4dfc-8d60-a3162c8398c1","Type":"ContainerStarted","Data":"7bc6bdf44639b6996bcf54ce042e812237d3d642a8c8945423165d0ed5e530c4"}
Oct 30 19:44:19.551175 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:19.550005    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b" podUID="19fe4fba-09ae-4620-b4d2-0ef8c2463d71" containerName="main" probeResult="failure" output="Get \"https://10.134.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:44:19.590937 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:19.589846    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s"
Oct 30 19:44:20.255744 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:20.255712    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b"
Oct 30 19:44:20.580281 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:20.580241    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5" event={"ID":"7938d051-21e1-4dfc-8d60-a3162c8398c1","Type":"ContainerStarted","Data":"01ea146cc38a96596da921936b295813275892534276527fd5e244488861f496"}
Oct 30 19:44:21.283160 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:21.283128    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5"
Oct 30 19:44:21.284311 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:21.284289    6996 patch_prober.go:29] interesting pod/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Oct 30 19:44:21.284398 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:21.284343    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5" podUID="7938d051-21e1-4dfc-8d60-a3162c8398c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Oct 30 19:44:21.587537 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:21.587497    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="models-as-a-service/payload-processing-external-model-646458976c-x8jhm" podUID="1cedc683-f566-4e92-b59c-1efcc56a7738" containerName="payload-processing" containerID="cri-o://469a5be560f41bf6a4f42686e2fa418cb98eb661102055afdc45f7c8e010457d" gracePeriod=25
Oct 30 19:44:21.893537 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:21.893513    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-646458976c-x8jhm"
Oct 30 19:44:21.917319 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:21.917260    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5" podStartSLOduration=5.917244824 podStartE2EDuration="5.917244824s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:44:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:44:20.615784455 +0000 UTC m=+735.943902610" watchObservedRunningTime="2026-10-30 19:44:21.917244824 +0000 UTC m=+737.245362979"
Oct 30 19:44:21.972610 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:21.972580    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1cedc683-f566-4e92-b59c-1efcc56a7738-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/1cedc683-f566-4e92-b59c-1efcc56a7738-routing-overlay-volume\") pod \"1cedc683-f566-4e92-b59c-1efcc56a7738\" (UID: \"1cedc683-f566-4e92-b59c-1efcc56a7738\") "
Oct 30 19:44:21.972768 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:21.972653    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1cedc683-f566-4e92-b59c-1efcc56a7738-kube-api-access-r2wkg\" (UniqueName: \"kubernetes.io/projected/1cedc683-f566-4e92-b59c-1efcc56a7738-kube-api-access-r2wkg\") pod \"1cedc683-f566-4e92-b59c-1efcc56a7738\" (UID: \"1cedc683-f566-4e92-b59c-1efcc56a7738\") "
Oct 30 19:44:21.972768 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:21.972711    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1cedc683-f566-4e92-b59c-1efcc56a7738-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1cedc683-f566-4e92-b59c-1efcc56a7738-plugins-config-volume\") pod \"1cedc683-f566-4e92-b59c-1efcc56a7738\" (UID: \"1cedc683-f566-4e92-b59c-1efcc56a7738\") "
Oct 30 19:44:21.972768 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:21.972766    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1cedc683-f566-4e92-b59c-1efcc56a7738-provider-credentials\" (UniqueName: \"kubernetes.io/projected/1cedc683-f566-4e92-b59c-1efcc56a7738-provider-credentials\") pod \"1cedc683-f566-4e92-b59c-1efcc56a7738\" (UID: \"1cedc683-f566-4e92-b59c-1efcc56a7738\") "
Oct 30 19:44:21.973026 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:21.972997    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1cedc683-f566-4e92-b59c-1efcc56a7738-routing-overlay-volume" pod "1cedc683-f566-4e92-b59c-1efcc56a7738" (UID: "1cedc683-f566-4e92-b59c-1efcc56a7738"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:44:21.973132 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:21.973112    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1cedc683-f566-4e92-b59c-1efcc56a7738-plugins-config-volume" pod "1cedc683-f566-4e92-b59c-1efcc56a7738" (UID: "1cedc683-f566-4e92-b59c-1efcc56a7738"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:44:21.973265 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:21.973249    6996 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/1cedc683-f566-4e92-b59c-1efcc56a7738-routing-overlay-volume\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:44:21.973324 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:21.973267    6996 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1cedc683-f566-4e92-b59c-1efcc56a7738-plugins-config-volume\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:44:21.974775 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:21.974753    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cedc683-f566-4e92-b59c-1efcc56a7738-kube-api-access-r2wkg" pod "1cedc683-f566-4e92-b59c-1efcc56a7738" (UID: "1cedc683-f566-4e92-b59c-1efcc56a7738"). InnerVolumeSpecName "kube-api-access-r2wkg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:44:21.974879 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:21.974852    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cedc683-f566-4e92-b59c-1efcc56a7738-provider-credentials" pod "1cedc683-f566-4e92-b59c-1efcc56a7738" (UID: "1cedc683-f566-4e92-b59c-1efcc56a7738"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:44:22.028259 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.028220    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw"]
Oct 30 19:44:22.028896 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.028876    6996 cpu_manager.go:414] "containerMap: removing container" podUID="1cedc683-f566-4e92-b59c-1efcc56a7738" containerName="payload-processing"
Oct 30 19:44:22.029004 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.028897    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1cedc683-f566-4e92-b59c-1efcc56a7738" containerName="payload-processing"
Oct 30 19:44:22.029129 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.029110    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="1cedc683-f566-4e92-b59c-1efcc56a7738" containerName="payload-processing"
Oct 30 19:44:22.033099 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.033077    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw"
Oct 30 19:44:22.070809 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.070773    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw"]
Oct 30 19:44:22.074005 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.073971    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f-credential-socket\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw\" (UID: \"3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw"
Oct 30 19:44:22.075116 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.075009    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f-workload-certs\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw\" (UID: \"3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw"
Oct 30 19:44:22.075320 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.075300    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f-workload-socket\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw\" (UID: \"3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw"
Oct 30 19:44:22.075617 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.075595    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f-istio-podinfo\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw\" (UID: \"3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw"
Oct 30 19:44:22.075809 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.075785    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzds4\" (UniqueName: \"kubernetes.io/projected/3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f-kube-api-access-gzds4\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw\" (UID: \"3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw"
Oct 30 19:44:22.075927 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.075830    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f-istio-token\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw\" (UID: \"3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw"
Oct 30 19:44:22.075995 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.075967    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f-istiod-ca-cert\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw\" (UID: \"3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw"
Oct 30 19:44:22.076060 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.076015    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f-istio-envoy\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw\" (UID: \"3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw"
Oct 30 19:44:22.076060 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.076041    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f-istio-data\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw\" (UID: \"3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw"
Oct 30 19:44:22.076168 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.076154    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r2wkg\" (UniqueName: \"kubernetes.io/projected/1cedc683-f566-4e92-b59c-1efcc56a7738-kube-api-access-r2wkg\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:44:22.076231 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.076172    6996 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/1cedc683-f566-4e92-b59c-1efcc56a7738-provider-credentials\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:44:22.076289 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.076271    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q"]
Oct 30 19:44:22.081337 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.081315    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q"
Oct 30 19:44:22.094531 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.094457    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q"]
Oct 30 19:44:22.176968 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.176860    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4dc9cd9-2754-4e38-af83-046f8536a53b-istio-token\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q\" (UID: \"e4dc9cd9-2754-4e38-af83-046f8536a53b\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q"
Oct 30 19:44:22.176968 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.176911    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f-istiod-ca-cert\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw\" (UID: \"3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw"
Oct 30 19:44:22.177209 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.176985    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f-istio-envoy\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw\" (UID: \"3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw"
Oct 30 19:44:22.177209 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.177021    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f-istio-data\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw\" (UID: \"3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw"
Oct 30 19:44:22.177209 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.177060    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4dc9cd9-2754-4e38-af83-046f8536a53b-credential-socket\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q\" (UID: \"e4dc9cd9-2754-4e38-af83-046f8536a53b\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q"
Oct 30 19:44:22.177209 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.177085    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4dc9cd9-2754-4e38-af83-046f8536a53b-workload-socket\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q\" (UID: \"e4dc9cd9-2754-4e38-af83-046f8536a53b\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q"
Oct 30 19:44:22.177209 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.177151    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4dc9cd9-2754-4e38-af83-046f8536a53b-istiod-ca-cert\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q\" (UID: \"e4dc9cd9-2754-4e38-af83-046f8536a53b\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q"
Oct 30 19:44:22.177442 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.177237    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4dc9cd9-2754-4e38-af83-046f8536a53b-istio-envoy\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q\" (UID: \"e4dc9cd9-2754-4e38-af83-046f8536a53b\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q"
Oct 30 19:44:22.177442 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.177283    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f-credential-socket\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw\" (UID: \"3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw"
Oct 30 19:44:22.177442 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.177368    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f-workload-certs\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw\" (UID: \"3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw"
Oct 30 19:44:22.177442 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.177396    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4dc9cd9-2754-4e38-af83-046f8536a53b-istio-podinfo\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q\" (UID: \"e4dc9cd9-2754-4e38-af83-046f8536a53b\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q"
Oct 30 19:44:22.177442 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.177424    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f-istio-data\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw\" (UID: \"3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw"
Oct 30 19:44:22.177442 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.177432    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f-workload-socket\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw\" (UID: \"3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw"
Oct 30 19:44:22.177771 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.177547    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f-istio-podinfo\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw\" (UID: \"3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw"
Oct 30 19:44:22.177771 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.177582    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzds4\" (UniqueName: \"kubernetes.io/projected/3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f-kube-api-access-gzds4\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw\" (UID: \"3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw"
Oct 30 19:44:22.177771 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.177591    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f-credential-socket\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw\" (UID: \"3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw"
Oct 30 19:44:22.177771 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.177633    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f-istio-token\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw\" (UID: \"3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw"
Oct 30 19:44:22.177771 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.177687    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4dc9cd9-2754-4e38-af83-046f8536a53b-workload-certs\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q\" (UID: \"e4dc9cd9-2754-4e38-af83-046f8536a53b\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q"
Oct 30 19:44:22.177771 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.177704    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4dc9cd9-2754-4e38-af83-046f8536a53b-istio-data\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q\" (UID: \"e4dc9cd9-2754-4e38-af83-046f8536a53b\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q"
Oct 30 19:44:22.177771 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.177735    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f-istiod-ca-cert\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw\" (UID: \"3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw"
Oct 30 19:44:22.177771 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.177763    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75mqr\" (UniqueName: \"kubernetes.io/projected/e4dc9cd9-2754-4e38-af83-046f8536a53b-kube-api-access-75mqr\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q\" (UID: \"e4dc9cd9-2754-4e38-af83-046f8536a53b\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q"
Oct 30 19:44:22.177771 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.177767    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f-workload-socket\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw\" (UID: \"3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw"
Oct 30 19:44:22.178259 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.178015    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f-workload-certs\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw\" (UID: \"3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw"
Oct 30 19:44:22.179456 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.179435    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f-istio-envoy\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw\" (UID: \"3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw"
Oct 30 19:44:22.180253 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.180234    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f-istio-podinfo\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw\" (UID: \"3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw"
Oct 30 19:44:22.193263 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.193238    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzds4\" (UniqueName: \"kubernetes.io/projected/3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f-kube-api-access-gzds4\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw\" (UID: \"3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw"
Oct 30 19:44:22.195416 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.195398    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f-istio-token\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw\" (UID: \"3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw"
Oct 30 19:44:22.279434 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.279386    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4dc9cd9-2754-4e38-af83-046f8536a53b-istio-podinfo\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q\" (UID: \"e4dc9cd9-2754-4e38-af83-046f8536a53b\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q"
Oct 30 19:44:22.279635 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.279539    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4dc9cd9-2754-4e38-af83-046f8536a53b-workload-certs\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q\" (UID: \"e4dc9cd9-2754-4e38-af83-046f8536a53b\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q"
Oct 30 19:44:22.279635 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.279556    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4dc9cd9-2754-4e38-af83-046f8536a53b-istio-data\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q\" (UID: \"e4dc9cd9-2754-4e38-af83-046f8536a53b\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q"
Oct 30 19:44:22.279635 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.279604    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75mqr\" (UniqueName: \"kubernetes.io/projected/e4dc9cd9-2754-4e38-af83-046f8536a53b-kube-api-access-75mqr\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q\" (UID: \"e4dc9cd9-2754-4e38-af83-046f8536a53b\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q"
Oct 30 19:44:22.279771 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.279643    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4dc9cd9-2754-4e38-af83-046f8536a53b-istio-token\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q\" (UID: \"e4dc9cd9-2754-4e38-af83-046f8536a53b\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q"
Oct 30 19:44:22.279771 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.279728    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4dc9cd9-2754-4e38-af83-046f8536a53b-credential-socket\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q\" (UID: \"e4dc9cd9-2754-4e38-af83-046f8536a53b\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q"
Oct 30 19:44:22.279771 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.279761    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4dc9cd9-2754-4e38-af83-046f8536a53b-workload-socket\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q\" (UID: \"e4dc9cd9-2754-4e38-af83-046f8536a53b\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q"
Oct 30 19:44:22.279924 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.279790    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4dc9cd9-2754-4e38-af83-046f8536a53b-istiod-ca-cert\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q\" (UID: \"e4dc9cd9-2754-4e38-af83-046f8536a53b\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q"
Oct 30 19:44:22.279982 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.279919    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4dc9cd9-2754-4e38-af83-046f8536a53b-istio-envoy\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q\" (UID: \"e4dc9cd9-2754-4e38-af83-046f8536a53b\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q"
Oct 30 19:44:22.280053 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.280031    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4dc9cd9-2754-4e38-af83-046f8536a53b-istio-data\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q\" (UID: \"e4dc9cd9-2754-4e38-af83-046f8536a53b\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q"
Oct 30 19:44:22.280114 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.280055    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4dc9cd9-2754-4e38-af83-046f8536a53b-workload-certs\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q\" (UID: \"e4dc9cd9-2754-4e38-af83-046f8536a53b\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q"
Oct 30 19:44:22.280114 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.280102    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4dc9cd9-2754-4e38-af83-046f8536a53b-credential-socket\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q\" (UID: \"e4dc9cd9-2754-4e38-af83-046f8536a53b\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q"
Oct 30 19:44:22.280227 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.280184    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4dc9cd9-2754-4e38-af83-046f8536a53b-workload-socket\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q\" (UID: \"e4dc9cd9-2754-4e38-af83-046f8536a53b\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q"
Oct 30 19:44:22.280403 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.280383    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4dc9cd9-2754-4e38-af83-046f8536a53b-istiod-ca-cert\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q\" (UID: \"e4dc9cd9-2754-4e38-af83-046f8536a53b\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q"
Oct 30 19:44:22.281996 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.281975    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4dc9cd9-2754-4e38-af83-046f8536a53b-istio-podinfo\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q\" (UID: \"e4dc9cd9-2754-4e38-af83-046f8536a53b\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q"
Oct 30 19:44:22.282083 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.281993    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4dc9cd9-2754-4e38-af83-046f8536a53b-istio-envoy\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q\" (UID: \"e4dc9cd9-2754-4e38-af83-046f8536a53b\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q"
Oct 30 19:44:22.283198 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.283179    6996 patch_prober.go:29] interesting pod/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Oct 30 19:44:22.283496 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.283218    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5" podUID="7938d051-21e1-4dfc-8d60-a3162c8398c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Oct 30 19:44:22.293289 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.293270    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4dc9cd9-2754-4e38-af83-046f8536a53b-istio-token\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q\" (UID: \"e4dc9cd9-2754-4e38-af83-046f8536a53b\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q"
Oct 30 19:44:22.293668 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.293651    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-75mqr\" (UniqueName: \"kubernetes.io/projected/e4dc9cd9-2754-4e38-af83-046f8536a53b-kube-api-access-75mqr\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q\" (UID: \"e4dc9cd9-2754-4e38-af83-046f8536a53b\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q"
Oct 30 19:44:22.382757 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.382715    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw"
Oct 30 19:44:22.397213 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.397194    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q"
Oct 30 19:44:22.594300 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.594268    6996 generic.go:356] "Generic (PLEG): container finished" podID="1cedc683-f566-4e92-b59c-1efcc56a7738" containerID="469a5be560f41bf6a4f42686e2fa418cb98eb661102055afdc45f7c8e010457d" exitCode=0
Oct 30 19:44:22.594512 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.594322    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-646458976c-x8jhm"
Oct 30 19:44:22.594512 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.594348    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-646458976c-x8jhm" event={"ID":"1cedc683-f566-4e92-b59c-1efcc56a7738","Type":"ContainerDied","Data":"469a5be560f41bf6a4f42686e2fa418cb98eb661102055afdc45f7c8e010457d"}
Oct 30 19:44:22.594512 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.594397    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-646458976c-x8jhm" event={"ID":"1cedc683-f566-4e92-b59c-1efcc56a7738","Type":"ContainerDied","Data":"0e25c67bdad3e1a819720adadee1e1e98a068ec1792f0baf9524676fb299067d"}
Oct 30 19:44:22.594512 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.594418    6996 scope.go:122] "RemoveContainer" containerID="469a5be560f41bf6a4f42686e2fa418cb98eb661102055afdc45f7c8e010457d"
Oct 30 19:44:22.604601 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.604585    6996 scope.go:122] "RemoveContainer" containerID="469a5be560f41bf6a4f42686e2fa418cb98eb661102055afdc45f7c8e010457d"
Oct 30 19:44:22.604848 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:44:22.604831    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"469a5be560f41bf6a4f42686e2fa418cb98eb661102055afdc45f7c8e010457d\": container with ID starting with 469a5be560f41bf6a4f42686e2fa418cb98eb661102055afdc45f7c8e010457d not found: ID does not exist" containerID="469a5be560f41bf6a4f42686e2fa418cb98eb661102055afdc45f7c8e010457d"
Oct 30 19:44:22.604891 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.604857    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"469a5be560f41bf6a4f42686e2fa418cb98eb661102055afdc45f7c8e010457d"} err="failed to get container status \"469a5be560f41bf6a4f42686e2fa418cb98eb661102055afdc45f7c8e010457d\": rpc error: code = NotFound desc = could not find container \"469a5be560f41bf6a4f42686e2fa418cb98eb661102055afdc45f7c8e010457d\": container with ID starting with 469a5be560f41bf6a4f42686e2fa418cb98eb661102055afdc45f7c8e010457d not found: ID does not exist"
Oct 30 19:44:22.619895 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.619871    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-646458976c-x8jhm"]
Oct 30 19:44:22.625611 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.625589    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["models-as-a-service/payload-processing-external-model-646458976c-x8jhm"]
Oct 30 19:44:22.694902 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.694876    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw"]
Oct 30 19:44:22.696904 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:44:22.696865    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e2461ca_03d1_49f8_9c16_a97d1bcc4b4f.slice/crio-3240e6d1c695c8d25b00ec0269e1f17622b8048fe254028f9282ee2543ed9c86 WatchSource:0}: Error finding container 3240e6d1c695c8d25b00ec0269e1f17622b8048fe254028f9282ee2543ed9c86: Status 404 returned error can't find the container with id 3240e6d1c695c8d25b00ec0269e1f17622b8048fe254028f9282ee2543ed9c86
Oct 30 19:44:22.699694 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.699658    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:44:22.699782 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.699761    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:44:22.699818 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.699808    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:44:22.726157 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.726134    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q"]
Oct 30 19:44:22.763799 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.763763    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:44:22.763924 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.763830    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:44:22.763924 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.763881    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:44:22.800960 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:22.800923    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1cedc683-f566-4e92-b59c-1efcc56a7738" path="/var/lib/kubelet/pods/1cedc683-f566-4e92-b59c-1efcc56a7738/volumes"
Oct 30 19:44:23.282775 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:23.282727    6996 patch_prober.go:29] interesting pod/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Oct 30 19:44:23.282980 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:23.282830    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5" podUID="7938d051-21e1-4dfc-8d60-a3162c8398c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Oct 30 19:44:23.744586 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:23.744530    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q" event={"ID":"e4dc9cd9-2754-4e38-af83-046f8536a53b","Type":"ContainerStarted","Data":"4e76f852fdd7b74ff9a30e4a940491d1c806c247b7b6eb613489659148d1e143"}
Oct 30 19:44:23.771079 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:23.771027    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q" event={"ID":"e4dc9cd9-2754-4e38-af83-046f8536a53b","Type":"ContainerStarted","Data":"fadeab9ec2914c51273ab05c0c67e3ff3307a6fea2b3a156bf6a6d7d6da95fd8"}
Oct 30 19:44:23.823549 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:23.821902    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q" podStartSLOduration=1.821884479 podStartE2EDuration="1.821884479s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:44:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:44:23.7909846 +0000 UTC m=+739.119102755" watchObservedRunningTime="2026-10-30 19:44:23.821884479 +0000 UTC m=+739.150002632"
Oct 30 19:44:23.825536 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:23.825375    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw" event={"ID":"3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f","Type":"ContainerStarted","Data":"ed9b12658f56200202df9922ffeb7c897e533122721cc3f7d9ca7b6ee503e8cc"}
Oct 30 19:44:23.825536 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:23.825441    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw" event={"ID":"3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f","Type":"ContainerStarted","Data":"3240e6d1c695c8d25b00ec0269e1f17622b8048fe254028f9282ee2543ed9c86"}
Oct 30 19:44:23.890277 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:23.890208    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw" podStartSLOduration=2.89018711 podStartE2EDuration="2.89018711s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:44:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:44:23.862441708 +0000 UTC m=+739.190559873" watchObservedRunningTime="2026-10-30 19:44:23.89018711 +0000 UTC m=+739.218305265"
Oct 30 19:44:24.285505 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:24.284513    6996 patch_prober.go:29] interesting pod/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Oct 30 19:44:24.285505 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:24.284589    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5" podUID="7938d051-21e1-4dfc-8d60-a3162c8398c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Oct 30 19:44:24.413506 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:24.412698    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw"
Oct 30 19:44:24.413506 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:24.412776    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q"
Oct 30 19:44:24.532503 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:24.524889    6996 patch_prober.go:29] interesting pod/e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body=
Oct 30 19:44:24.532503 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:24.524956    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw" podUID="3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused"
Oct 30 19:44:24.532503 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:24.525244    6996 patch_prober.go:29] interesting pod/e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused" start-of-body=
Oct 30 19:44:24.532503 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:24.525272    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q" podUID="e4dc9cd9-2754-4e38-af83-046f8536a53b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused"
Oct 30 19:44:25.290504 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:25.290129    6996 patch_prober.go:29] interesting pod/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Oct 30 19:44:25.290504 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:25.290211    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5" podUID="7938d051-21e1-4dfc-8d60-a3162c8398c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Oct 30 19:44:25.405504 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:25.396635    6996 patch_prober.go:29] interesting pod/e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body=
Oct 30 19:44:25.405504 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:25.396710    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw" podUID="3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused"
Oct 30 19:44:25.405504 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:25.404875    6996 patch_prober.go:29] interesting pod/e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused" start-of-body=
Oct 30 19:44:25.405504 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:25.404940    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q" podUID="e4dc9cd9-2754-4e38-af83-046f8536a53b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused"
Oct 30 19:44:26.286504 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:26.286164    6996 patch_prober.go:29] interesting pod/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Oct 30 19:44:26.286504 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:26.286238    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5" podUID="7938d051-21e1-4dfc-8d60-a3162c8398c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Oct 30 19:44:26.386507 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:26.384672    6996 patch_prober.go:29] interesting pod/e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body=
Oct 30 19:44:26.386507 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:26.384739    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw" podUID="3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused"
Oct 30 19:44:26.397771 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:26.397652    6996 patch_prober.go:29] interesting pod/e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused" start-of-body=
Oct 30 19:44:26.397771 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:26.397700    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q" podUID="e4dc9cd9-2754-4e38-af83-046f8536a53b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused"
Oct 30 19:44:27.292081 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:27.292035    6996 patch_prober.go:29] interesting pod/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Oct 30 19:44:27.292349 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:27.292319    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5" podUID="7938d051-21e1-4dfc-8d60-a3162c8398c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Oct 30 19:44:27.386744 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:27.386708    6996 patch_prober.go:29] interesting pod/e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body=
Oct 30 19:44:27.387570 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:27.387539    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw" podUID="3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused"
Oct 30 19:44:27.397820 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:27.397740    6996 patch_prober.go:29] interesting pod/e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused" start-of-body=
Oct 30 19:44:27.397995 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:27.397816    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q" podUID="e4dc9cd9-2754-4e38-af83-046f8536a53b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused"
Oct 30 19:44:28.293092 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:28.293055    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5"
Oct 30 19:44:28.293792 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:28.293761    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5"
Oct 30 19:44:28.294874 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:28.294857    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5"
Oct 30 19:44:28.404521 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:28.401317    6996 patch_prober.go:29] interesting pod/e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body=
Oct 30 19:44:28.404521 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:28.401375    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw" podUID="3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused"
Oct 30 19:44:28.404521 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:28.402614    6996 patch_prober.go:29] interesting pod/e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused" start-of-body=
Oct 30 19:44:28.404521 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:28.402656    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q" podUID="e4dc9cd9-2754-4e38-af83-046f8536a53b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused"
Oct 30 19:44:29.383741 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:29.383698    6996 patch_prober.go:29] interesting pod/e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body=
Oct 30 19:44:29.383925 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:29.383770    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw" podUID="3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused"
Oct 30 19:44:29.397809 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:29.397779    6996 patch_prober.go:29] interesting pod/e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused" start-of-body=
Oct 30 19:44:29.397952 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:29.397827    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q" podUID="e4dc9cd9-2754-4e38-af83-046f8536a53b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused"
Oct 30 19:44:30.399341 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:30.399309    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw"
Oct 30 19:44:30.399991 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:30.399692    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw"
Oct 30 19:44:30.400839 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:30.400812    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw"
Oct 30 19:44:30.405530 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:30.405506    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q"
Oct 30 19:44:30.405898 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:30.405871    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q"
Oct 30 19:44:30.406656 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:30.406632    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q"
Oct 30 19:44:31.191768 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:44:31.191731    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 19:44:31.480908 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:31.480792    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 19:44:49.906946 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:49.906903    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-2a3e1b/distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq"]
Oct 30 19:44:49.912161 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:49.912132    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq"
Oct 30 19:44:49.919160 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:49.919128    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-2a3e1b\"/\"distinct-w4-2a3e1b-kserve-self-signed-certs\""
Oct 30 19:44:49.925190 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:49.925163    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-2a3e1b/distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq"]
Oct 30 19:44:50.034040 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:50.033999    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89009638-003d-4a5d-ab13-1f8c8f64d3b7-model-cache\") pod \"distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq\" (UID: \"89009638-003d-4a5d-ab13-1f8c8f64d3b7\") " pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq"
Oct 30 19:44:50.034040 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:50.034041    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89009638-003d-4a5d-ab13-1f8c8f64d3b7-home\") pod \"distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq\" (UID: \"89009638-003d-4a5d-ab13-1f8c8f64d3b7\") " pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq"
Oct 30 19:44:50.034271 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:50.034171    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89009638-003d-4a5d-ab13-1f8c8f64d3b7-tls-certs\") pod \"distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq\" (UID: \"89009638-003d-4a5d-ab13-1f8c8f64d3b7\") " pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq"
Oct 30 19:44:50.034271 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:50.034218    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89009638-003d-4a5d-ab13-1f8c8f64d3b7-dshm\") pod \"distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq\" (UID: \"89009638-003d-4a5d-ab13-1f8c8f64d3b7\") " pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq"
Oct 30 19:44:50.034338 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:50.034304    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6kwl\" (UniqueName: \"kubernetes.io/projected/89009638-003d-4a5d-ab13-1f8c8f64d3b7-kube-api-access-x6kwl\") pod \"distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq\" (UID: \"89009638-003d-4a5d-ab13-1f8c8f64d3b7\") " pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq"
Oct 30 19:44:50.034375 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:50.034361    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89009638-003d-4a5d-ab13-1f8c8f64d3b7-tmp-dir\") pod \"distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq\" (UID: \"89009638-003d-4a5d-ab13-1f8c8f64d3b7\") " pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq"
Oct 30 19:44:50.136181 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:50.136138    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6kwl\" (UniqueName: \"kubernetes.io/projected/89009638-003d-4a5d-ab13-1f8c8f64d3b7-kube-api-access-x6kwl\") pod \"distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq\" (UID: \"89009638-003d-4a5d-ab13-1f8c8f64d3b7\") " pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq"
Oct 30 19:44:50.136400 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:50.136235    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89009638-003d-4a5d-ab13-1f8c8f64d3b7-tmp-dir\") pod \"distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq\" (UID: \"89009638-003d-4a5d-ab13-1f8c8f64d3b7\") " pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq"
Oct 30 19:44:50.136400 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:50.136385    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89009638-003d-4a5d-ab13-1f8c8f64d3b7-model-cache\") pod \"distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq\" (UID: \"89009638-003d-4a5d-ab13-1f8c8f64d3b7\") " pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq"
Oct 30 19:44:50.136564 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:50.136415    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89009638-003d-4a5d-ab13-1f8c8f64d3b7-home\") pod \"distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq\" (UID: \"89009638-003d-4a5d-ab13-1f8c8f64d3b7\") " pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq"
Oct 30 19:44:50.136939 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:50.136913    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89009638-003d-4a5d-ab13-1f8c8f64d3b7-tmp-dir\") pod \"distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq\" (UID: \"89009638-003d-4a5d-ab13-1f8c8f64d3b7\") " pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq"
Oct 30 19:44:50.137008 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:50.136951    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89009638-003d-4a5d-ab13-1f8c8f64d3b7-model-cache\") pod \"distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq\" (UID: \"89009638-003d-4a5d-ab13-1f8c8f64d3b7\") " pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq"
Oct 30 19:44:50.137008 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:50.136988    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89009638-003d-4a5d-ab13-1f8c8f64d3b7-tls-certs\") pod \"distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq\" (UID: \"89009638-003d-4a5d-ab13-1f8c8f64d3b7\") " pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq"
Oct 30 19:44:50.137115 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:50.137064    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89009638-003d-4a5d-ab13-1f8c8f64d3b7-dshm\") pod \"distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq\" (UID: \"89009638-003d-4a5d-ab13-1f8c8f64d3b7\") " pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq"
Oct 30 19:44:50.137333 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:50.137309    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89009638-003d-4a5d-ab13-1f8c8f64d3b7-home\") pod \"distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq\" (UID: \"89009638-003d-4a5d-ab13-1f8c8f64d3b7\") " pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq"
Oct 30 19:44:50.139925 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:50.139903    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89009638-003d-4a5d-ab13-1f8c8f64d3b7-tls-certs\") pod \"distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq\" (UID: \"89009638-003d-4a5d-ab13-1f8c8f64d3b7\") " pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq"
Oct 30 19:44:50.145744 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:50.143746    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89009638-003d-4a5d-ab13-1f8c8f64d3b7-dshm\") pod \"distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq\" (UID: \"89009638-003d-4a5d-ab13-1f8c8f64d3b7\") " pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq"
Oct 30 19:44:50.151195 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:50.151166    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6kwl\" (UniqueName: \"kubernetes.io/projected/89009638-003d-4a5d-ab13-1f8c8f64d3b7-kube-api-access-x6kwl\") pod \"distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq\" (UID: \"89009638-003d-4a5d-ab13-1f8c8f64d3b7\") " pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq"
Oct 30 19:44:50.230825 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:50.230736    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq"
Oct 30 19:44:50.515555 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:50.515520    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-2a3e1b/distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq"]
Oct 30 19:44:50.517551 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:44:50.517522    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod89009638_003d_4a5d_ab13_1f8c8f64d3b7.slice/crio-e74597a4c9e6ac335f1115fb4600bee2261c72e63c30becba614802a9daa3fc6 WatchSource:0}: Error finding container e74597a4c9e6ac335f1115fb4600bee2261c72e63c30becba614802a9daa3fc6: Status 404 returned error can't find the container with id e74597a4c9e6ac335f1115fb4600bee2261c72e63c30becba614802a9daa3fc6
Oct 30 19:44:51.206458 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:51.206380    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq" event={"ID":"89009638-003d-4a5d-ab13-1f8c8f64d3b7","Type":"ContainerStarted","Data":"cd497be8ff51e29fca5d267706805e3208fb8324980217626924a5fed9a7bd35"}
Oct 30 19:44:51.206458 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:51.206425    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq" event={"ID":"89009638-003d-4a5d-ab13-1f8c8f64d3b7","Type":"ContainerStarted","Data":"e74597a4c9e6ac335f1115fb4600bee2261c72e63c30becba614802a9daa3fc6"}
Oct 30 19:44:51.227633 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:51.227581    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq" podStartSLOduration=2.227563567 podStartE2EDuration="2.227563567s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:44:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:44:51.225944496 +0000 UTC m=+766.554062651" watchObservedRunningTime="2026-10-30 19:44:51.227563567 +0000 UTC m=+766.555681721"
Oct 30 19:44:51.231394 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:51.231366    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq"
Oct 30 19:44:52.065955 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:52.065921    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 19:44:55.406503 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:55.402785    6996 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s" podUID="ae905dfb-f30d-480a-899b-ea676abc1d8d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.73:8000/health\": context deadline exceeded"
Oct 30 19:44:55.710568 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:55.708385    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b" podUID="19fe4fba-09ae-4620-b4d2-0ef8c2463d71" containerName="main" probeResult="failure" output="Get \"https://10.134.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:44:56.153514 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:56.145589    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s" podUID="ae905dfb-f30d-480a-899b-ea676abc1d8d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.73:8000/ready\": context deadline exceeded"
Oct 30 19:44:56.153514 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:44:56.146358    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr" podUID="0e470010-625d-443f-81ec-9bf965988212" containerName="main" probeResult="failure" output="Get \"https://10.134.0.72:8000/ready\": context deadline exceeded"
Oct 30 19:45:00.109781 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:00.109746    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["models-as-a-service/payload-processing-external-model-646458976c-4t9np"]
Oct 30 19:45:00.141988 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:00.137364    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-646458976c-4t9np"]
Oct 30 19:45:00.141988 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:00.137514    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-646458976c-4t9np"
Oct 30 19:45:00.235500 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:00.235431    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq"
Oct 30 19:45:00.288511 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:00.284245    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/2d279bab-751e-4f75-8f72-f887e5b2fc6e-routing-overlay-volume\") pod \"payload-processing-external-model-646458976c-4t9np\" (UID: \"2d279bab-751e-4f75-8f72-f887e5b2fc6e\") " pod="models-as-a-service/payload-processing-external-model-646458976c-4t9np"
Oct 30 19:45:00.288511 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:00.284452    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d279bab-751e-4f75-8f72-f887e5b2fc6e-plugins-config-volume\") pod \"payload-processing-external-model-646458976c-4t9np\" (UID: \"2d279bab-751e-4f75-8f72-f887e5b2fc6e\") " pod="models-as-a-service/payload-processing-external-model-646458976c-4t9np"
Oct 30 19:45:00.288511 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:00.284550    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/2d279bab-751e-4f75-8f72-f887e5b2fc6e-provider-credentials\") pod \"payload-processing-external-model-646458976c-4t9np\" (UID: \"2d279bab-751e-4f75-8f72-f887e5b2fc6e\") " pod="models-as-a-service/payload-processing-external-model-646458976c-4t9np"
Oct 30 19:45:00.288511 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:00.284611    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxjq7\" (UniqueName: \"kubernetes.io/projected/2d279bab-751e-4f75-8f72-f887e5b2fc6e-kube-api-access-qxjq7\") pod \"payload-processing-external-model-646458976c-4t9np\" (UID: \"2d279bab-751e-4f75-8f72-f887e5b2fc6e\") " pod="models-as-a-service/payload-processing-external-model-646458976c-4t9np"
Oct 30 19:45:00.293473 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:00.293443    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq"
Oct 30 19:45:00.385695 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:00.385601    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d279bab-751e-4f75-8f72-f887e5b2fc6e-plugins-config-volume\") pod \"payload-processing-external-model-646458976c-4t9np\" (UID: \"2d279bab-751e-4f75-8f72-f887e5b2fc6e\") " pod="models-as-a-service/payload-processing-external-model-646458976c-4t9np"
Oct 30 19:45:00.385988 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:00.385965    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/2d279bab-751e-4f75-8f72-f887e5b2fc6e-provider-credentials\") pod \"payload-processing-external-model-646458976c-4t9np\" (UID: \"2d279bab-751e-4f75-8f72-f887e5b2fc6e\") " pod="models-as-a-service/payload-processing-external-model-646458976c-4t9np"
Oct 30 19:45:00.386198 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:00.386164    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxjq7\" (UniqueName: \"kubernetes.io/projected/2d279bab-751e-4f75-8f72-f887e5b2fc6e-kube-api-access-qxjq7\") pod \"payload-processing-external-model-646458976c-4t9np\" (UID: \"2d279bab-751e-4f75-8f72-f887e5b2fc6e\") " pod="models-as-a-service/payload-processing-external-model-646458976c-4t9np"
Oct 30 19:45:00.386438 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:00.386417    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/2d279bab-751e-4f75-8f72-f887e5b2fc6e-routing-overlay-volume\") pod \"payload-processing-external-model-646458976c-4t9np\" (UID: \"2d279bab-751e-4f75-8f72-f887e5b2fc6e\") " pod="models-as-a-service/payload-processing-external-model-646458976c-4t9np"
Oct 30 19:45:00.386583 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:00.386558    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d279bab-751e-4f75-8f72-f887e5b2fc6e-plugins-config-volume\") pod \"payload-processing-external-model-646458976c-4t9np\" (UID: \"2d279bab-751e-4f75-8f72-f887e5b2fc6e\") " pod="models-as-a-service/payload-processing-external-model-646458976c-4t9np"
Oct 30 19:45:00.387623 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:00.387596    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/2d279bab-751e-4f75-8f72-f887e5b2fc6e-routing-overlay-volume\") pod \"payload-processing-external-model-646458976c-4t9np\" (UID: \"2d279bab-751e-4f75-8f72-f887e5b2fc6e\") " pod="models-as-a-service/payload-processing-external-model-646458976c-4t9np"
Oct 30 19:45:00.394402 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:00.394373    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/2d279bab-751e-4f75-8f72-f887e5b2fc6e-provider-credentials\") pod \"payload-processing-external-model-646458976c-4t9np\" (UID: \"2d279bab-751e-4f75-8f72-f887e5b2fc6e\") " pod="models-as-a-service/payload-processing-external-model-646458976c-4t9np"
Oct 30 19:45:00.409162 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:00.409131    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxjq7\" (UniqueName: \"kubernetes.io/projected/2d279bab-751e-4f75-8f72-f887e5b2fc6e-kube-api-access-qxjq7\") pod \"payload-processing-external-model-646458976c-4t9np\" (UID: \"2d279bab-751e-4f75-8f72-f887e5b2fc6e\") " pod="models-as-a-service/payload-processing-external-model-646458976c-4t9np"
Oct 30 19:45:00.597265 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:00.596709    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-646458976c-4t9np"
Oct 30 19:45:00.914829 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:00.914800    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-646458976c-4t9np"]
Oct 30 19:45:00.916879 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:45:00.916851    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d279bab_751e_4f75_8f72_f887e5b2fc6e.slice/crio-394b4971067437e4612fd242222e13ee44b562ed351979bcdb704cc5e3c4975f WatchSource:0}: Error finding container 394b4971067437e4612fd242222e13ee44b562ed351979bcdb704cc5e3c4975f: Status 404 returned error can't find the container with id 394b4971067437e4612fd242222e13ee44b562ed351979bcdb704cc5e3c4975f
Oct 30 19:45:01.004902 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:01.004867    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-646458976c-4t9np" event={"ID":"2d279bab-751e-4f75-8f72-f887e5b2fc6e","Type":"ContainerStarted","Data":"e36c02eed5d745804cc033ad1a8573032a466df53b465936ee3e6c689b8d4d04"}
Oct 30 19:45:01.005177 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:01.005144    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-646458976c-4t9np" event={"ID":"2d279bab-751e-4f75-8f72-f887e5b2fc6e","Type":"ContainerStarted","Data":"394b4971067437e4612fd242222e13ee44b562ed351979bcdb704cc5e3c4975f"}
Oct 30 19:45:01.005322 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:01.005310    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="models-as-a-service/payload-processing-external-model-646458976c-4t9np"
Oct 30 19:45:01.028831 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:01.028799    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq"
Oct 30 19:45:01.029613 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:01.029572    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="models-as-a-service/payload-processing-external-model-646458976c-4t9np" podStartSLOduration=1.029555964 podStartE2EDuration="1.029555964s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:45:01.028715937 +0000 UTC m=+776.356834087" watchObservedRunningTime="2026-10-30 19:45:01.029555964 +0000 UTC m=+776.357674120"
Oct 30 19:45:01.196155 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:45:01.196073    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 19:45:08.634157 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:08.634113    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-2a3e1b/distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk"]
Oct 30 19:45:08.641293 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:08.641267    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk"
Oct 30 19:45:08.646100 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:08.646058    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-2a3e1b\"/\"distinct-2-w4-2a3e1b-kserve-self-signed-certs\""
Oct 30 19:45:08.651095 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:08.651065    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-2a3e1b/distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk"]
Oct 30 19:45:08.808419 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:08.808379    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1857d2d-3b40-4e63-be33-be4ba476f926-home\") pod \"distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk\" (UID: \"a1857d2d-3b40-4e63-be33-be4ba476f926\") " pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk"
Oct 30 19:45:08.810868 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:08.810827    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1857d2d-3b40-4e63-be33-be4ba476f926-tls-certs\") pod \"distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk\" (UID: \"a1857d2d-3b40-4e63-be33-be4ba476f926\") " pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk"
Oct 30 19:45:08.811170 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:08.811144    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1857d2d-3b40-4e63-be33-be4ba476f926-tmp-dir\") pod \"distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk\" (UID: \"a1857d2d-3b40-4e63-be33-be4ba476f926\") " pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk"
Oct 30 19:45:08.815536 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:08.811471    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1857d2d-3b40-4e63-be33-be4ba476f926-dshm\") pod \"distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk\" (UID: \"a1857d2d-3b40-4e63-be33-be4ba476f926\") " pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk"
Oct 30 19:45:08.815781 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:08.815760    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1857d2d-3b40-4e63-be33-be4ba476f926-model-cache\") pod \"distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk\" (UID: \"a1857d2d-3b40-4e63-be33-be4ba476f926\") " pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk"
Oct 30 19:45:08.816003 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:08.815983    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4xlr\" (UniqueName: \"kubernetes.io/projected/a1857d2d-3b40-4e63-be33-be4ba476f926-kube-api-access-p4xlr\") pod \"distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk\" (UID: \"a1857d2d-3b40-4e63-be33-be4ba476f926\") " pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk"
Oct 30 19:45:08.917991 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:08.917899    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1857d2d-3b40-4e63-be33-be4ba476f926-home\") pod \"distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk\" (UID: \"a1857d2d-3b40-4e63-be33-be4ba476f926\") " pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk"
Oct 30 19:45:08.918160 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:08.918012    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1857d2d-3b40-4e63-be33-be4ba476f926-tls-certs\") pod \"distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk\" (UID: \"a1857d2d-3b40-4e63-be33-be4ba476f926\") " pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk"
Oct 30 19:45:08.918160 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:08.918049    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1857d2d-3b40-4e63-be33-be4ba476f926-tmp-dir\") pod \"distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk\" (UID: \"a1857d2d-3b40-4e63-be33-be4ba476f926\") " pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk"
Oct 30 19:45:08.918255 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:08.918188    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1857d2d-3b40-4e63-be33-be4ba476f926-dshm\") pod \"distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk\" (UID: \"a1857d2d-3b40-4e63-be33-be4ba476f926\") " pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk"
Oct 30 19:45:08.918255 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:08.918216    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1857d2d-3b40-4e63-be33-be4ba476f926-model-cache\") pod \"distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk\" (UID: \"a1857d2d-3b40-4e63-be33-be4ba476f926\") " pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk"
Oct 30 19:45:08.918358 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:08.918284    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4xlr\" (UniqueName: \"kubernetes.io/projected/a1857d2d-3b40-4e63-be33-be4ba476f926-kube-api-access-p4xlr\") pod \"distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk\" (UID: \"a1857d2d-3b40-4e63-be33-be4ba476f926\") " pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk"
Oct 30 19:45:08.918358 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:08.918330    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1857d2d-3b40-4e63-be33-be4ba476f926-home\") pod \"distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk\" (UID: \"a1857d2d-3b40-4e63-be33-be4ba476f926\") " pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk"
Oct 30 19:45:08.918550 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:08.918443    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1857d2d-3b40-4e63-be33-be4ba476f926-tmp-dir\") pod \"distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk\" (UID: \"a1857d2d-3b40-4e63-be33-be4ba476f926\") " pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk"
Oct 30 19:45:08.918661 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:08.918634    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1857d2d-3b40-4e63-be33-be4ba476f926-model-cache\") pod \"distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk\" (UID: \"a1857d2d-3b40-4e63-be33-be4ba476f926\") " pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk"
Oct 30 19:45:08.920582 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:08.920554    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1857d2d-3b40-4e63-be33-be4ba476f926-dshm\") pod \"distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk\" (UID: \"a1857d2d-3b40-4e63-be33-be4ba476f926\") " pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk"
Oct 30 19:45:08.920840 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:08.920820    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1857d2d-3b40-4e63-be33-be4ba476f926-tls-certs\") pod \"distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk\" (UID: \"a1857d2d-3b40-4e63-be33-be4ba476f926\") " pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk"
Oct 30 19:45:08.931912 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:08.931882    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4xlr\" (UniqueName: \"kubernetes.io/projected/a1857d2d-3b40-4e63-be33-be4ba476f926-kube-api-access-p4xlr\") pod \"distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk\" (UID: \"a1857d2d-3b40-4e63-be33-be4ba476f926\") " pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk"
Oct 30 19:45:08.956735 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:08.956694    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk"
Oct 30 19:45:09.244511 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:09.244459    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-2a3e1b/distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk"]
Oct 30 19:45:09.246072 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:45:09.246048    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda1857d2d_3b40_4e63_be33_be4ba476f926.slice/crio-92aae822cebf644c7224bef16724d9985654afb4dd677b98965c5170e5ef20a1 WatchSource:0}: Error finding container 92aae822cebf644c7224bef16724d9985654afb4dd677b98965c5170e5ef20a1: Status 404 returned error can't find the container with id 92aae822cebf644c7224bef16724d9985654afb4dd677b98965c5170e5ef20a1
Oct 30 19:45:10.070384 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:10.070349    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk" event={"ID":"a1857d2d-3b40-4e63-be33-be4ba476f926","Type":"ContainerStarted","Data":"333b252c6267e56044991c7b47299163c325d1b3ea1e70604a319e9615a1ce52"}
Oct 30 19:45:10.070384 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:10.070393    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk" event={"ID":"a1857d2d-3b40-4e63-be33-be4ba476f926","Type":"ContainerStarted","Data":"92aae822cebf644c7224bef16724d9985654afb4dd677b98965c5170e5ef20a1"}
Oct 30 19:45:10.094185 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:10.094135    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk" podStartSLOduration=2.094118048 podStartE2EDuration="2.094118048s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:45:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:45:10.090279254 +0000 UTC m=+785.418397408" watchObservedRunningTime="2026-10-30 19:45:10.094118048 +0000 UTC m=+785.422236202"
Oct 30 19:45:11.007508 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:10.990694    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk"
Oct 30 19:45:12.316678 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:12.316627    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s" podUID="ae905dfb-f30d-480a-899b-ea676abc1d8d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.73:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:45:12.582662 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:12.582557    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b" podUID="19fe4fba-09ae-4620-b4d2-0ef8c2463d71" containerName="main" probeResult="failure" output="Get \"https://10.134.0.81:8000/ready\": context deadline exceeded"
Oct 30 19:45:13.022562 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:13.021763    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr" podUID="0e470010-625d-443f-81ec-9bf965988212" containerName="main" probeResult="failure" output="Get \"https://10.134.0.72:8000/ready\": context deadline exceeded"
Oct 30 19:45:13.032726 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:13.032693    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="models-as-a-service/payload-processing-external-model-646458976c-4t9np"
Oct 30 19:45:13.089666 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:13.089617    6996 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5db77fbd9f-5jtlw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:45:13.090002 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:13.089970    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-5jtlw" podUID="abe1772a-c94e-4dd9-b023-176bab5081d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:45:13.397638 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:13.375427    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s" podUID="ae905dfb-f30d-480a-899b-ea676abc1d8d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:45:13.452649 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:13.437777    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s"
Oct 30 19:45:13.452649 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:13.440139    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-54b4664b47-zpskh"]
Oct 30 19:45:13.588501 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:13.588329    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b" podUID="19fe4fba-09ae-4620-b4d2-0ef8c2463d71" containerName="main" probeResult="failure" output="Get \"https://10.134.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:45:13.614514 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:13.590145    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b"
Oct 30 19:45:14.013753 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:14.013707    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq" podUID="89009638-003d-4a5d-ab13-1f8c8f64d3b7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.88:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:45:14.035905 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:14.035860    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr" podUID="0e470010-625d-443f-81ec-9bf965988212" containerName="main" probeResult="failure" output="Get \"https://10.134.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:45:14.038053 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:14.038026    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr"
Oct 30 19:45:14.539505 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:14.409631    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s" podUID="ae905dfb-f30d-480a-899b-ea676abc1d8d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.73:8000/ready\": context deadline exceeded"
Oct 30 19:45:14.539505 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:14.518601    6996 patch_prober.go:29] interesting pod/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:45:14.539505 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:14.518659    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5" podUID="7938d051-21e1-4dfc-8d60-a3162c8398c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:45:14.567239 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:14.567203    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-646458976c-4t9np"]
Oct 30 19:45:14.596752 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:14.596710    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b" podUID="19fe4fba-09ae-4620-b4d2-0ef8c2463d71" containerName="main" probeResult="failure" output="Get \"https://10.134.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:45:15.099507 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:15.097542    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq" podUID="89009638-003d-4a5d-ab13-1f8c8f64d3b7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.88:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:45:15.135628 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:15.122553    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq"
Oct 30 19:45:15.182520 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:15.170145    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr" podUID="0e470010-625d-443f-81ec-9bf965988212" containerName="main" probeResult="failure" output="Get \"https://10.134.0.72:8000/ready\": context deadline exceeded"
Oct 30 19:45:15.316785 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:15.316734    6996 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s" podUID="ae905dfb-f30d-480a-899b-ea676abc1d8d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.73:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:45:15.430074 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:15.429965    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s" podUID="ae905dfb-f30d-480a-899b-ea676abc1d8d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:45:15.571762 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:15.571721    6996 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5db77fbd9f-sjbzh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:45:15.593505 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:15.572383    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sjbzh" podUID="b79c69dc-9d5b-4b3b-802e-ac414149df33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:45:15.623387 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:15.623168    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b" podUID="19fe4fba-09ae-4620-b4d2-0ef8c2463d71" containerName="main" probeResult="failure" output="Get \"https://10.134.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:45:16.099685 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:16.099639    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq" podUID="89009638-003d-4a5d-ab13-1f8c8f64d3b7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.88:8000/ready\": context deadline exceeded"
Oct 30 19:45:16.345506 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:16.345229    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr" podUID="0e470010-625d-443f-81ec-9bf965988212" containerName="main" probeResult="failure" output="Get \"https://10.134.0.72:8000/ready\": context deadline exceeded"
Oct 30 19:45:16.464300 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:16.464194    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s" podUID="ae905dfb-f30d-480a-899b-ea676abc1d8d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:45:16.628049 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:16.627995    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b" podUID="19fe4fba-09ae-4620-b4d2-0ef8c2463d71" containerName="main" probeResult="failure" output="Get \"https://10.134.0.81:8000/ready\": context deadline exceeded"
Oct 30 19:45:16.916225 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:16.916183    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq"
Oct 30 19:45:17.373237 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:17.366752    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s"
Oct 30 19:45:17.373237 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:17.370303    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr" podUID="0e470010-625d-443f-81ec-9bf965988212" containerName="main" probeResult="failure" output="Get \"https://10.134.0.72:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:45:17.373237 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:17.370881    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b"
Oct 30 19:45:17.397387 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:17.397354    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr"
Oct 30 19:45:18.440469 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:18.440410    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="models-as-a-service/payload-processing-external-model-54b4664b47-zpskh" podUID="b06d7926-a4e2-4f55-aa9a-f8684c73bac6" containerName="payload-processing" containerID="cri-o://52431ece8a590f5ed7a141d956bbf289c123b6c1ba40f509db97eb7999896525" gracePeriod=25
Oct 30 19:45:18.815666 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:18.815640    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-54b4664b47-zpskh"
Oct 30 19:45:18.920517 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:18.920457    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b06d7926-a4e2-4f55-aa9a-f8684c73bac6-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/b06d7926-a4e2-4f55-aa9a-f8684c73bac6-routing-overlay-volume\") pod \"b06d7926-a4e2-4f55-aa9a-f8684c73bac6\" (UID: \"b06d7926-a4e2-4f55-aa9a-f8684c73bac6\") "
Oct 30 19:45:18.920688 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:18.920597    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b06d7926-a4e2-4f55-aa9a-f8684c73bac6-kube-api-access-77htp\" (UniqueName: \"kubernetes.io/projected/b06d7926-a4e2-4f55-aa9a-f8684c73bac6-kube-api-access-77htp\") pod \"b06d7926-a4e2-4f55-aa9a-f8684c73bac6\" (UID: \"b06d7926-a4e2-4f55-aa9a-f8684c73bac6\") "
Oct 30 19:45:18.920732 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:18.920702    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b06d7926-a4e2-4f55-aa9a-f8684c73bac6-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b06d7926-a4e2-4f55-aa9a-f8684c73bac6-plugins-config-volume\") pod \"b06d7926-a4e2-4f55-aa9a-f8684c73bac6\" (UID: \"b06d7926-a4e2-4f55-aa9a-f8684c73bac6\") "
Oct 30 19:45:18.920771 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:18.920749    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b06d7926-a4e2-4f55-aa9a-f8684c73bac6-provider-credentials\" (UniqueName: \"kubernetes.io/projected/b06d7926-a4e2-4f55-aa9a-f8684c73bac6-provider-credentials\") pod \"b06d7926-a4e2-4f55-aa9a-f8684c73bac6\" (UID: \"b06d7926-a4e2-4f55-aa9a-f8684c73bac6\") "
Oct 30 19:45:18.920853 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:18.920825    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b06d7926-a4e2-4f55-aa9a-f8684c73bac6-routing-overlay-volume" pod "b06d7926-a4e2-4f55-aa9a-f8684c73bac6" (UID: "b06d7926-a4e2-4f55-aa9a-f8684c73bac6"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:45:18.921153 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:18.921121    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b06d7926-a4e2-4f55-aa9a-f8684c73bac6-plugins-config-volume" pod "b06d7926-a4e2-4f55-aa9a-f8684c73bac6" (UID: "b06d7926-a4e2-4f55-aa9a-f8684c73bac6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:45:18.921371 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:18.921350    6996 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b06d7926-a4e2-4f55-aa9a-f8684c73bac6-plugins-config-volume\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:18.921519 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:18.921373    6996 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/b06d7926-a4e2-4f55-aa9a-f8684c73bac6-routing-overlay-volume\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:18.922917 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:18.922887    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b06d7926-a4e2-4f55-aa9a-f8684c73bac6-provider-credentials" pod "b06d7926-a4e2-4f55-aa9a-f8684c73bac6" (UID: "b06d7926-a4e2-4f55-aa9a-f8684c73bac6"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:45:18.923031 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:18.922932    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b06d7926-a4e2-4f55-aa9a-f8684c73bac6-kube-api-access-77htp" pod "b06d7926-a4e2-4f55-aa9a-f8684c73bac6" (UID: "b06d7926-a4e2-4f55-aa9a-f8684c73bac6"). InnerVolumeSpecName "kube-api-access-77htp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:45:18.957040 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:18.957008    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk"
Oct 30 19:45:18.969900 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:18.969824    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk"
Oct 30 19:45:19.022594 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:19.022551    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-77htp\" (UniqueName: \"kubernetes.io/projected/b06d7926-a4e2-4f55-aa9a-f8684c73bac6-kube-api-access-77htp\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:19.022594 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:19.022588    6996 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/b06d7926-a4e2-4f55-aa9a-f8684c73bac6-provider-credentials\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:19.499221 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:19.499184    6996 generic.go:356] "Generic (PLEG): container finished" podID="b06d7926-a4e2-4f55-aa9a-f8684c73bac6" containerID="52431ece8a590f5ed7a141d956bbf289c123b6c1ba40f509db97eb7999896525" exitCode=0
Oct 30 19:45:19.499680 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:19.499232    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-54b4664b47-zpskh"
Oct 30 19:45:19.499680 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:19.499266    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-54b4664b47-zpskh" event={"ID":"b06d7926-a4e2-4f55-aa9a-f8684c73bac6","Type":"ContainerDied","Data":"52431ece8a590f5ed7a141d956bbf289c123b6c1ba40f509db97eb7999896525"}
Oct 30 19:45:19.499680 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:19.499321    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-54b4664b47-zpskh" event={"ID":"b06d7926-a4e2-4f55-aa9a-f8684c73bac6","Type":"ContainerDied","Data":"c244ff1be998fa482d05c5d19bf6c6ca1547b59025c08a41d8b2b96340369116"}
Oct 30 19:45:19.499680 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:19.499347    6996 scope.go:122] "RemoveContainer" containerID="52431ece8a590f5ed7a141d956bbf289c123b6c1ba40f509db97eb7999896525"
Oct 30 19:45:19.512331 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:19.512308    6996 scope.go:122] "RemoveContainer" containerID="52431ece8a590f5ed7a141d956bbf289c123b6c1ba40f509db97eb7999896525"
Oct 30 19:45:19.513427 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:45:19.513394    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52431ece8a590f5ed7a141d956bbf289c123b6c1ba40f509db97eb7999896525\": container with ID starting with 52431ece8a590f5ed7a141d956bbf289c123b6c1ba40f509db97eb7999896525 not found: ID does not exist" containerID="52431ece8a590f5ed7a141d956bbf289c123b6c1ba40f509db97eb7999896525"
Oct 30 19:45:19.513560 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:19.513436    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52431ece8a590f5ed7a141d956bbf289c123b6c1ba40f509db97eb7999896525"} err="failed to get container status \"52431ece8a590f5ed7a141d956bbf289c123b6c1ba40f509db97eb7999896525\": rpc error: code = NotFound desc = could not find container \"52431ece8a590f5ed7a141d956bbf289c123b6c1ba40f509db97eb7999896525\": container with ID starting with 52431ece8a590f5ed7a141d956bbf289c123b6c1ba40f509db97eb7999896525 not found: ID does not exist"
Oct 30 19:45:19.513652 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:19.513623    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk"
Oct 30 19:45:19.526851 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:19.526824    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-54b4664b47-zpskh"]
Oct 30 19:45:19.531635 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:19.531610    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["models-as-a-service/payload-processing-external-model-54b4664b47-zpskh"]
Oct 30 19:45:19.571673 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:19.571621    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="models-as-a-service/payload-processing-external-model-646458976c-4t9np" podUID="2d279bab-751e-4f75-8f72-f887e5b2fc6e" containerName="payload-processing" containerID="cri-o://e36c02eed5d745804cc033ad1a8573032a466df53b465936ee3e6c689b8d4d04" gracePeriod=25
Oct 30 19:45:19.881950 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:19.881919    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-646458976c-4t9np"
Oct 30 19:45:19.934648 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:19.934615    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2d279bab-751e-4f75-8f72-f887e5b2fc6e-kube-api-access-qxjq7\" (UniqueName: \"kubernetes.io/projected/2d279bab-751e-4f75-8f72-f887e5b2fc6e-kube-api-access-qxjq7\") pod \"2d279bab-751e-4f75-8f72-f887e5b2fc6e\" (UID: \"2d279bab-751e-4f75-8f72-f887e5b2fc6e\") "
Oct 30 19:45:19.934835 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:19.934814    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2d279bab-751e-4f75-8f72-f887e5b2fc6e-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/2d279bab-751e-4f75-8f72-f887e5b2fc6e-routing-overlay-volume\") pod \"2d279bab-751e-4f75-8f72-f887e5b2fc6e\" (UID: \"2d279bab-751e-4f75-8f72-f887e5b2fc6e\") "
Oct 30 19:45:19.934887 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:19.934861    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2d279bab-751e-4f75-8f72-f887e5b2fc6e-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d279bab-751e-4f75-8f72-f887e5b2fc6e-plugins-config-volume\") pod \"2d279bab-751e-4f75-8f72-f887e5b2fc6e\" (UID: \"2d279bab-751e-4f75-8f72-f887e5b2fc6e\") "
Oct 30 19:45:19.935056 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:19.935037    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2d279bab-751e-4f75-8f72-f887e5b2fc6e-provider-credentials\" (UniqueName: \"kubernetes.io/projected/2d279bab-751e-4f75-8f72-f887e5b2fc6e-provider-credentials\") pod \"2d279bab-751e-4f75-8f72-f887e5b2fc6e\" (UID: \"2d279bab-751e-4f75-8f72-f887e5b2fc6e\") "
Oct 30 19:45:19.935273 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:19.935243    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d279bab-751e-4f75-8f72-f887e5b2fc6e-routing-overlay-volume" pod "2d279bab-751e-4f75-8f72-f887e5b2fc6e" (UID: "2d279bab-751e-4f75-8f72-f887e5b2fc6e"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:45:19.935368 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:19.935316    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d279bab-751e-4f75-8f72-f887e5b2fc6e-plugins-config-volume" pod "2d279bab-751e-4f75-8f72-f887e5b2fc6e" (UID: "2d279bab-751e-4f75-8f72-f887e5b2fc6e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:45:19.935748 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:19.935714    6996 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/2d279bab-751e-4f75-8f72-f887e5b2fc6e-routing-overlay-volume\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:19.935748 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:19.935744    6996 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d279bab-751e-4f75-8f72-f887e5b2fc6e-plugins-config-volume\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:19.937017 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:19.936991    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d279bab-751e-4f75-8f72-f887e5b2fc6e-kube-api-access-qxjq7" pod "2d279bab-751e-4f75-8f72-f887e5b2fc6e" (UID: "2d279bab-751e-4f75-8f72-f887e5b2fc6e"). InnerVolumeSpecName "kube-api-access-qxjq7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:45:19.937113 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:19.937072    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d279bab-751e-4f75-8f72-f887e5b2fc6e-provider-credentials" pod "2d279bab-751e-4f75-8f72-f887e5b2fc6e" (UID: "2d279bab-751e-4f75-8f72-f887e5b2fc6e"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:45:20.037118 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:20.037027    6996 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/2d279bab-751e-4f75-8f72-f887e5b2fc6e-provider-credentials\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:20.037118 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:20.037064    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qxjq7\" (UniqueName: \"kubernetes.io/projected/2d279bab-751e-4f75-8f72-f887e5b2fc6e-kube-api-access-qxjq7\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:20.510072 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:20.510034    6996 generic.go:356] "Generic (PLEG): container finished" podID="2d279bab-751e-4f75-8f72-f887e5b2fc6e" containerID="e36c02eed5d745804cc033ad1a8573032a466df53b465936ee3e6c689b8d4d04" exitCode=0
Oct 30 19:45:20.510645 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:20.510088    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-646458976c-4t9np"
Oct 30 19:45:20.510645 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:20.510114    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-646458976c-4t9np" event={"ID":"2d279bab-751e-4f75-8f72-f887e5b2fc6e","Type":"ContainerDied","Data":"e36c02eed5d745804cc033ad1a8573032a466df53b465936ee3e6c689b8d4d04"}
Oct 30 19:45:20.510645 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:20.510176    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-646458976c-4t9np" event={"ID":"2d279bab-751e-4f75-8f72-f887e5b2fc6e","Type":"ContainerDied","Data":"394b4971067437e4612fd242222e13ee44b562ed351979bcdb704cc5e3c4975f"}
Oct 30 19:45:20.510645 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:20.510209    6996 scope.go:122] "RemoveContainer" containerID="e36c02eed5d745804cc033ad1a8573032a466df53b465936ee3e6c689b8d4d04"
Oct 30 19:45:20.521870 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:20.521850    6996 scope.go:122] "RemoveContainer" containerID="e36c02eed5d745804cc033ad1a8573032a466df53b465936ee3e6c689b8d4d04"
Oct 30 19:45:20.522166 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:45:20.522144    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e36c02eed5d745804cc033ad1a8573032a466df53b465936ee3e6c689b8d4d04\": container with ID starting with e36c02eed5d745804cc033ad1a8573032a466df53b465936ee3e6c689b8d4d04 not found: ID does not exist" containerID="e36c02eed5d745804cc033ad1a8573032a466df53b465936ee3e6c689b8d4d04"
Oct 30 19:45:20.522223 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:20.522175    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e36c02eed5d745804cc033ad1a8573032a466df53b465936ee3e6c689b8d4d04"} err="failed to get container status \"e36c02eed5d745804cc033ad1a8573032a466df53b465936ee3e6c689b8d4d04\": rpc error: code = NotFound desc = could not find container \"e36c02eed5d745804cc033ad1a8573032a466df53b465936ee3e6c689b8d4d04\": container with ID starting with e36c02eed5d745804cc033ad1a8573032a466df53b465936ee3e6c689b8d4d04 not found: ID does not exist"
Oct 30 19:45:20.534870 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:20.534842    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-646458976c-4t9np"]
Oct 30 19:45:20.549571 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:20.549543    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["models-as-a-service/payload-processing-external-model-646458976c-4t9np"]
Oct 30 19:45:20.792829 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:20.792751    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2d279bab-751e-4f75-8f72-f887e5b2fc6e" path="/var/lib/kubelet/pods/2d279bab-751e-4f75-8f72-f887e5b2fc6e/volumes"
Oct 30 19:45:20.793206 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:20.793192    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b06d7926-a4e2-4f55-aa9a-f8684c73bac6" path="/var/lib/kubelet/pods/b06d7926-a4e2-4f55-aa9a-f8684c73bac6/volumes"
Oct 30 19:45:30.190429 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:30.190383    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-07292b-openshift-default-744696b8d9-pljg8"]
Oct 30 19:45:30.191156 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:30.191136    6996 cpu_manager.go:414] "containerMap: removing container" podUID="b06d7926-a4e2-4f55-aa9a-f8684c73bac6" containerName="payload-processing"
Oct 30 19:45:30.191227 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:30.191157    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b06d7926-a4e2-4f55-aa9a-f8684c73bac6" containerName="payload-processing"
Oct 30 19:45:30.191227 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:30.191168    6996 cpu_manager.go:414] "containerMap: removing container" podUID="2d279bab-751e-4f75-8f72-f887e5b2fc6e" containerName="payload-processing"
Oct 30 19:45:30.191227 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:30.191173    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2d279bab-751e-4f75-8f72-f887e5b2fc6e" containerName="payload-processing"
Oct 30 19:45:30.191385 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:30.191273    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="b06d7926-a4e2-4f55-aa9a-f8684c73bac6" containerName="payload-processing"
Oct 30 19:45:30.191385 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:30.191282    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="2d279bab-751e-4f75-8f72-f887e5b2fc6e" containerName="payload-processing"
Oct 30 19:45:30.196861 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:30.196834    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-07292b-openshift-default-744696b8d9-pljg8"
Oct 30 19:45:30.202521 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:30.202496    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-07292b-openshift-default-dockercfg-xb6p6\""
Oct 30 19:45:30.209815 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:30.209788    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-07292b-openshift-default-744696b8d9-pljg8"]
Oct 30 19:45:30.283750 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:30.283708    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8797217-8ec7-46db-8f6a-4a11f77164f6-istio-data\") pod \"e2e-api-a-07292b-openshift-default-744696b8d9-pljg8\" (UID: \"a8797217-8ec7-46db-8f6a-4a11f77164f6\") " pod="openshift-ingress/e2e-api-a-07292b-openshift-default-744696b8d9-pljg8"
Oct 30 19:45:30.283934 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:30.283761    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8797217-8ec7-46db-8f6a-4a11f77164f6-workload-certs\") pod \"e2e-api-a-07292b-openshift-default-744696b8d9-pljg8\" (UID: \"a8797217-8ec7-46db-8f6a-4a11f77164f6\") " pod="openshift-ingress/e2e-api-a-07292b-openshift-default-744696b8d9-pljg8"
Oct 30 19:45:30.283934 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:30.283879    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8797217-8ec7-46db-8f6a-4a11f77164f6-istio-token\") pod \"e2e-api-a-07292b-openshift-default-744696b8d9-pljg8\" (UID: \"a8797217-8ec7-46db-8f6a-4a11f77164f6\") " pod="openshift-ingress/e2e-api-a-07292b-openshift-default-744696b8d9-pljg8"
Oct 30 19:45:30.283934 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:30.283923    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8797217-8ec7-46db-8f6a-4a11f77164f6-istiod-ca-cert\") pod \"e2e-api-a-07292b-openshift-default-744696b8d9-pljg8\" (UID: \"a8797217-8ec7-46db-8f6a-4a11f77164f6\") " pod="openshift-ingress/e2e-api-a-07292b-openshift-default-744696b8d9-pljg8"
Oct 30 19:45:30.284050 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:30.284020    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8797217-8ec7-46db-8f6a-4a11f77164f6-istio-podinfo\") pod \"e2e-api-a-07292b-openshift-default-744696b8d9-pljg8\" (UID: \"a8797217-8ec7-46db-8f6a-4a11f77164f6\") " pod="openshift-ingress/e2e-api-a-07292b-openshift-default-744696b8d9-pljg8"
Oct 30 19:45:30.284098 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:30.284047    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8797217-8ec7-46db-8f6a-4a11f77164f6-workload-socket\") pod \"e2e-api-a-07292b-openshift-default-744696b8d9-pljg8\" (UID: \"a8797217-8ec7-46db-8f6a-4a11f77164f6\") " pod="openshift-ingress/e2e-api-a-07292b-openshift-default-744696b8d9-pljg8"
Oct 30 19:45:30.284139 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:30.284095    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8797217-8ec7-46db-8f6a-4a11f77164f6-istio-envoy\") pod \"e2e-api-a-07292b-openshift-default-744696b8d9-pljg8\" (UID: \"a8797217-8ec7-46db-8f6a-4a11f77164f6\") " pod="openshift-ingress/e2e-api-a-07292b-openshift-default-744696b8d9-pljg8"
Oct 30 19:45:30.284139 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:30.284127    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8797217-8ec7-46db-8f6a-4a11f77164f6-credential-socket\") pod \"e2e-api-a-07292b-openshift-default-744696b8d9-pljg8\" (UID: \"a8797217-8ec7-46db-8f6a-4a11f77164f6\") " pod="openshift-ingress/e2e-api-a-07292b-openshift-default-744696b8d9-pljg8"
Oct 30 19:45:30.284205 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:30.284153    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9j9qc\" (UniqueName: \"kubernetes.io/projected/a8797217-8ec7-46db-8f6a-4a11f77164f6-kube-api-access-9j9qc\") pod \"e2e-api-a-07292b-openshift-default-744696b8d9-pljg8\" (UID: \"a8797217-8ec7-46db-8f6a-4a11f77164f6\") " pod="openshift-ingress/e2e-api-a-07292b-openshift-default-744696b8d9-pljg8"
Oct 30 19:45:30.388664 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:30.388608    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 19:45:30.391335 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:30.391177    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8797217-8ec7-46db-8f6a-4a11f77164f6-istio-data\") pod \"e2e-api-a-07292b-openshift-default-744696b8d9-pljg8\" (UID: \"a8797217-8ec7-46db-8f6a-4a11f77164f6\") " pod="openshift-ingress/e2e-api-a-07292b-openshift-default-744696b8d9-pljg8"
Oct 30 19:45:30.399459 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:30.399161    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8797217-8ec7-46db-8f6a-4a11f77164f6-workload-certs\") pod \"e2e-api-a-07292b-openshift-default-744696b8d9-pljg8\" (UID: \"a8797217-8ec7-46db-8f6a-4a11f77164f6\") " pod="openshift-ingress/e2e-api-a-07292b-openshift-default-744696b8d9-pljg8"
Oct 30 19:45:30.399459 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:30.399282    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8797217-8ec7-46db-8f6a-4a11f77164f6-istio-token\") pod \"e2e-api-a-07292b-openshift-default-744696b8d9-pljg8\" (UID: \"a8797217-8ec7-46db-8f6a-4a11f77164f6\") " pod="openshift-ingress/e2e-api-a-07292b-openshift-default-744696b8d9-pljg8"
Oct 30 19:45:30.399459 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:30.399344    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8797217-8ec7-46db-8f6a-4a11f77164f6-istiod-ca-cert\") pod \"e2e-api-a-07292b-openshift-default-744696b8d9-pljg8\" (UID: \"a8797217-8ec7-46db-8f6a-4a11f77164f6\") " pod="openshift-ingress/e2e-api-a-07292b-openshift-default-744696b8d9-pljg8"
Oct 30 19:45:30.402368 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:30.399876    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8797217-8ec7-46db-8f6a-4a11f77164f6-istio-podinfo\") pod \"e2e-api-a-07292b-openshift-default-744696b8d9-pljg8\" (UID: \"a8797217-8ec7-46db-8f6a-4a11f77164f6\") " pod="openshift-ingress/e2e-api-a-07292b-openshift-default-744696b8d9-pljg8"
Oct 30 19:45:30.402368 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:30.399924    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8797217-8ec7-46db-8f6a-4a11f77164f6-workload-socket\") pod \"e2e-api-a-07292b-openshift-default-744696b8d9-pljg8\" (UID: \"a8797217-8ec7-46db-8f6a-4a11f77164f6\") " pod="openshift-ingress/e2e-api-a-07292b-openshift-default-744696b8d9-pljg8"
Oct 30 19:45:30.402368 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:30.399968    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8797217-8ec7-46db-8f6a-4a11f77164f6-istio-envoy\") pod \"e2e-api-a-07292b-openshift-default-744696b8d9-pljg8\" (UID: \"a8797217-8ec7-46db-8f6a-4a11f77164f6\") " pod="openshift-ingress/e2e-api-a-07292b-openshift-default-744696b8d9-pljg8"
Oct 30 19:45:30.402368 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:30.400009    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8797217-8ec7-46db-8f6a-4a11f77164f6-credential-socket\") pod \"e2e-api-a-07292b-openshift-default-744696b8d9-pljg8\" (UID: \"a8797217-8ec7-46db-8f6a-4a11f77164f6\") " pod="openshift-ingress/e2e-api-a-07292b-openshift-default-744696b8d9-pljg8"
Oct 30 19:45:30.402368 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:30.400041    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9j9qc\" (UniqueName: \"kubernetes.io/projected/a8797217-8ec7-46db-8f6a-4a11f77164f6-kube-api-access-9j9qc\") pod \"e2e-api-a-07292b-openshift-default-744696b8d9-pljg8\" (UID: \"a8797217-8ec7-46db-8f6a-4a11f77164f6\") " pod="openshift-ingress/e2e-api-a-07292b-openshift-default-744696b8d9-pljg8"
Oct 30 19:45:30.402368 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:30.401050    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8797217-8ec7-46db-8f6a-4a11f77164f6-workload-certs\") pod \"e2e-api-a-07292b-openshift-default-744696b8d9-pljg8\" (UID: \"a8797217-8ec7-46db-8f6a-4a11f77164f6\") " pod="openshift-ingress/e2e-api-a-07292b-openshift-default-744696b8d9-pljg8"
Oct 30 19:45:30.402368 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:30.401944    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8797217-8ec7-46db-8f6a-4a11f77164f6-istiod-ca-cert\") pod \"e2e-api-a-07292b-openshift-default-744696b8d9-pljg8\" (UID: \"a8797217-8ec7-46db-8f6a-4a11f77164f6\") " pod="openshift-ingress/e2e-api-a-07292b-openshift-default-744696b8d9-pljg8"
Oct 30 19:45:30.403613 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:30.403585    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8797217-8ec7-46db-8f6a-4a11f77164f6-workload-socket\") pod \"e2e-api-a-07292b-openshift-default-744696b8d9-pljg8\" (UID: \"a8797217-8ec7-46db-8f6a-4a11f77164f6\") " pod="openshift-ingress/e2e-api-a-07292b-openshift-default-744696b8d9-pljg8"
Oct 30 19:45:30.404313 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:30.404285    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8797217-8ec7-46db-8f6a-4a11f77164f6-credential-socket\") pod \"e2e-api-a-07292b-openshift-default-744696b8d9-pljg8\" (UID: \"a8797217-8ec7-46db-8f6a-4a11f77164f6\") " pod="openshift-ingress/e2e-api-a-07292b-openshift-default-744696b8d9-pljg8"
Oct 30 19:45:30.404664 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:30.404641    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8797217-8ec7-46db-8f6a-4a11f77164f6-istio-data\") pod \"e2e-api-a-07292b-openshift-default-744696b8d9-pljg8\" (UID: \"a8797217-8ec7-46db-8f6a-4a11f77164f6\") " pod="openshift-ingress/e2e-api-a-07292b-openshift-default-744696b8d9-pljg8"
Oct 30 19:45:30.406864 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:30.406838    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8797217-8ec7-46db-8f6a-4a11f77164f6-istio-envoy\") pod \"e2e-api-a-07292b-openshift-default-744696b8d9-pljg8\" (UID: \"a8797217-8ec7-46db-8f6a-4a11f77164f6\") " pod="openshift-ingress/e2e-api-a-07292b-openshift-default-744696b8d9-pljg8"
Oct 30 19:45:30.421726 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:30.421680    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8797217-8ec7-46db-8f6a-4a11f77164f6-istio-podinfo\") pod \"e2e-api-a-07292b-openshift-default-744696b8d9-pljg8\" (UID: \"a8797217-8ec7-46db-8f6a-4a11f77164f6\") " pod="openshift-ingress/e2e-api-a-07292b-openshift-default-744696b8d9-pljg8"
Oct 30 19:45:30.430719 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:30.430688    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9j9qc\" (UniqueName: \"kubernetes.io/projected/a8797217-8ec7-46db-8f6a-4a11f77164f6-kube-api-access-9j9qc\") pod \"e2e-api-a-07292b-openshift-default-744696b8d9-pljg8\" (UID: \"a8797217-8ec7-46db-8f6a-4a11f77164f6\") " pod="openshift-ingress/e2e-api-a-07292b-openshift-default-744696b8d9-pljg8"
Oct 30 19:45:30.433818 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:30.433781    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8797217-8ec7-46db-8f6a-4a11f77164f6-istio-token\") pod \"e2e-api-a-07292b-openshift-default-744696b8d9-pljg8\" (UID: \"a8797217-8ec7-46db-8f6a-4a11f77164f6\") " pod="openshift-ingress/e2e-api-a-07292b-openshift-default-744696b8d9-pljg8"
Oct 30 19:45:30.513517 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:30.513423    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-07292b-openshift-default-744696b8d9-pljg8"
Oct 30 19:45:30.822973 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:30.822936    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-07292b-openshift-default-744696b8d9-pljg8"]
Oct 30 19:45:30.824189 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:45:30.824158    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8797217_8ec7_46db_8f6a_4a11f77164f6.slice/crio-8c11250a6c07ec43a705c9bb032cb32c9ea249fd7099b2df563d55fc44594c71 WatchSource:0}: Error finding container 8c11250a6c07ec43a705c9bb032cb32c9ea249fd7099b2df563d55fc44594c71: Status 404 returned error can't find the container with id 8c11250a6c07ec43a705c9bb032cb32c9ea249fd7099b2df563d55fc44594c71
Oct 30 19:45:30.826416 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:30.826379    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:45:30.826515 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:30.826446    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:45:30.826515 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:30.826507    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:45:31.204856 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:45:31.204756    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 19:45:31.666503 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:31.665935    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-07292b-openshift-default-744696b8d9-pljg8" event={"ID":"a8797217-8ec7-46db-8f6a-4a11f77164f6","Type":"ContainerStarted","Data":"5b83384e092dbde5ebf88716e5b3c3c1297a9079b809893e40eb2874ceb3db5b"}
Oct 30 19:45:31.666503 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:31.666023    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-07292b-openshift-default-744696b8d9-pljg8" event={"ID":"a8797217-8ec7-46db-8f6a-4a11f77164f6","Type":"ContainerStarted","Data":"8c11250a6c07ec43a705c9bb032cb32c9ea249fd7099b2df563d55fc44594c71"}
Oct 30 19:45:31.753501 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:31.753085    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-07292b-openshift-default-744696b8d9-pljg8" podStartSLOduration=1.753065063 podStartE2EDuration="1.753065063s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:45:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:45:31.750200998 +0000 UTC m=+807.078319153" watchObservedRunningTime="2026-10-30 19:45:31.753065063 +0000 UTC m=+807.081183218"
Oct 30 19:45:32.515505 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:32.500804    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk" podUID="a1857d2d-3b40-4e63-be33-be4ba476f926" containerName="main" probeResult="failure" output="Get \"https://10.134.0.90:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:45:32.520160 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:32.520122    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-07292b-openshift-default-744696b8d9-pljg8"
Oct 30 19:45:32.537942 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:32.537897    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-07292b-openshift-default-744696b8d9-pljg8"
Oct 30 19:45:32.728510 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:32.725312    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-07292b-openshift-default-744696b8d9-pljg8"
Oct 30 19:45:32.736824 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:32.736789    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-07292b-openshift-default-744696b8d9-pljg8"
Oct 30 19:45:32.924632 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:32.924578    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq" podUID="89009638-003d-4a5d-ab13-1f8c8f64d3b7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.88:8000/ready\": context deadline exceeded"
Oct 30 19:45:33.199685 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:33.199583    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr" podUID="0e470010-625d-443f-81ec-9bf965988212" containerName="main" probeResult="failure" output="Get \"https://10.134.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:45:33.502565 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:33.501418    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk" podUID="a1857d2d-3b40-4e63-be33-be4ba476f926" containerName="main" probeResult="failure" output="Get \"https://10.134.0.90:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:45:33.537566 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:33.533264    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s" podUID="ae905dfb-f30d-480a-899b-ea676abc1d8d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.73:8000/ready\": context deadline exceeded"
Oct 30 19:45:33.588140 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:33.588099    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk"
Oct 30 19:45:33.752511 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:33.720396    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b" podUID="19fe4fba-09ae-4620-b4d2-0ef8c2463d71" containerName="main" probeResult="failure" output="Get \"https://10.134.0.81:8000/ready\": context deadline exceeded"
Oct 30 19:45:34.147446 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:34.147391    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq" podUID="89009638-003d-4a5d-ab13-1f8c8f64d3b7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.88:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:45:34.149590 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:34.149564    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq"
Oct 30 19:45:34.221636 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:34.212976    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr" podUID="0e470010-625d-443f-81ec-9bf965988212" containerName="main" probeResult="failure" output="Get \"https://10.134.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:45:34.327223 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:34.326777    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr"
Oct 30 19:45:34.578572 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:34.567608    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s" podUID="ae905dfb-f30d-480a-899b-ea676abc1d8d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:45:34.578572 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:34.568495    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s"
Oct 30 19:45:34.588227 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:34.588179    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk" podUID="a1857d2d-3b40-4e63-be33-be4ba476f926" containerName="main" probeResult="failure" output="Get \"https://10.134.0.90:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:45:34.759506 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:34.725138    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b" podUID="19fe4fba-09ae-4620-b4d2-0ef8c2463d71" containerName="main" probeResult="failure" output="Get \"https://10.134.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:45:34.760099 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:34.760066    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b"
Oct 30 19:45:35.242585 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:35.242536    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq" podUID="89009638-003d-4a5d-ab13-1f8c8f64d3b7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.88:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:45:35.244672 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:35.244634    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr" podUID="0e470010-625d-443f-81ec-9bf965988212" containerName="main" probeResult="failure" output="Get \"https://10.134.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:45:35.330513 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:35.329600    6996 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s" podUID="ae905dfb-f30d-480a-899b-ea676abc1d8d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.73:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:45:35.590504 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:35.572398    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s" podUID="ae905dfb-f30d-480a-899b-ea676abc1d8d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:45:35.618658 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:35.618609    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk" podUID="a1857d2d-3b40-4e63-be33-be4ba476f926" containerName="main" probeResult="failure" output="Get \"https://10.134.0.90:8000/ready\": context deadline exceeded"
Oct 30 19:45:35.791649 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:35.791606    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b" podUID="19fe4fba-09ae-4620-b4d2-0ef8c2463d71" containerName="main" probeResult="failure" output="Get \"https://10.134.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:45:36.012072 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:36.011621    6996 patch_prober.go:29] interesting pod/e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:45:36.012072 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:36.011692    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg" podUID="0d1b1d3b-0290-4c57-9a3e-1afab986cb98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:45:36.513510 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:36.512007    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq" podUID="89009638-003d-4a5d-ab13-1f8c8f64d3b7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.88:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:45:36.560510 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:36.541747    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr" podUID="0e470010-625d-443f-81ec-9bf965988212" containerName="main" probeResult="failure" output="Get \"https://10.134.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:45:36.610507 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:36.602577    6996 patch_prober.go:29] interesting pod/e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:45:36.665585 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:36.665517    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp" podUID="cf35d600-7eac-42ff-b1f9-79ad561fea3c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:45:36.721357 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:36.574985    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s" podUID="ae905dfb-f30d-480a-899b-ea676abc1d8d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:45:36.722015 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:36.629499    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk" podUID="a1857d2d-3b40-4e63-be33-be4ba476f926" containerName="main" probeResult="failure" output="Get \"https://10.134.0.90:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:45:36.872394 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:36.872351    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b" podUID="19fe4fba-09ae-4620-b4d2-0ef8c2463d71" containerName="main" probeResult="failure" output="Get \"https://10.134.0.81:8000/ready\": context deadline exceeded"
Oct 30 19:45:37.196614 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:37.196513    6996 patch_prober.go:29] interesting pod/e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:45:37.196900 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:37.196869    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw" podUID="ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:45:37.515771 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:37.515608    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq" podUID="89009638-003d-4a5d-ab13-1f8c8f64d3b7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.88:8000/ready\": context deadline exceeded"
Oct 30 19:45:37.561968 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:37.561918    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr" podUID="0e470010-625d-443f-81ec-9bf965988212" containerName="main" probeResult="failure" output="Get \"https://10.134.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:45:37.722764 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:37.722706    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk" podUID="a1857d2d-3b40-4e63-be33-be4ba476f926" containerName="main" probeResult="failure" output="Get \"https://10.134.0.90:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:45:37.724899 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:37.724869    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s" podUID="ae905dfb-f30d-480a-899b-ea676abc1d8d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.73:8000/ready\": context deadline exceeded"
Oct 30 19:45:38.056141 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:38.056096    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b" podUID="19fe4fba-09ae-4620-b4d2-0ef8c2463d71" containerName="main" probeResult="failure" output="Get \"https://10.134.0.81:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:45:38.297871 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:38.286430    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b"
Oct 30 19:45:38.318681 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:38.316181    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq"
Oct 30 19:45:38.318681 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:38.316713    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr"
Oct 30 19:45:38.319986 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:38.319147    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk"
Oct 30 19:45:38.344444 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:38.344411    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s"
Oct 30 19:45:38.535689 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:38.535656    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-07292b-74459b4d58-g8xfm"]
Oct 30 19:45:38.546775 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:38.546742    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-07292b-6c54c6b8c5-m67zk"]
Oct 30 19:45:38.549190 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:38.549148    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-07292b-74459b4d58-g8xfm"
Oct 30 19:45:38.554038 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:38.554006    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-07292b\""
Oct 30 19:45:38.554621 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:38.554598    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-07292b-dockercfg-n5kd4\""
Oct 30 19:45:38.554770 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:38.554753    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-07292b-74459b4d58-g8xfm"]
Oct 30 19:45:38.554890 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:38.554782    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-07292b-6c54c6b8c5-m67zk"
Oct 30 19:45:38.583773 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:38.583725    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-07292b-6c54c6b8c5-m67zk"]
Oct 30 19:45:38.715409 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:38.714768    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aa2c7806-fc8c-4c51-8108-c14d31c23be8-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-07292b-6c54c6b8c5-m67zk\" (UID: \"aa2c7806-fc8c-4c51-8108-c14d31c23be8\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-07292b-6c54c6b8c5-m67zk"
Oct 30 19:45:38.717032 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:38.717002    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea310863-7424-45c4-a916-8b824e270c40-plugins-config-volume\") pod \"payload-processing-e2e-api-a-07292b-74459b4d58-g8xfm\" (UID: \"ea310863-7424-45c4-a916-8b824e270c40\") " pod="openshift-ingress/payload-processing-e2e-api-a-07292b-74459b4d58-g8xfm"
Oct 30 19:45:38.717301 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:38.717282    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljrpf\" (UniqueName: \"kubernetes.io/projected/aa2c7806-fc8c-4c51-8108-c14d31c23be8-kube-api-access-ljrpf\") pod \"payload-pre-processing-e2e-api-a-07292b-6c54c6b8c5-m67zk\" (UID: \"aa2c7806-fc8c-4c51-8108-c14d31c23be8\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-07292b-6c54c6b8c5-m67zk"
Oct 30 19:45:38.717633 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:38.717610    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkdkp\" (UniqueName: \"kubernetes.io/projected/ea310863-7424-45c4-a916-8b824e270c40-kube-api-access-fkdkp\") pod \"payload-processing-e2e-api-a-07292b-74459b4d58-g8xfm\" (UID: \"ea310863-7424-45c4-a916-8b824e270c40\") " pod="openshift-ingress/payload-processing-e2e-api-a-07292b-74459b4d58-g8xfm"
Oct 30 19:45:38.819288 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:38.819248    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea310863-7424-45c4-a916-8b824e270c40-plugins-config-volume\") pod \"payload-processing-e2e-api-a-07292b-74459b4d58-g8xfm\" (UID: \"ea310863-7424-45c4-a916-8b824e270c40\") " pod="openshift-ingress/payload-processing-e2e-api-a-07292b-74459b4d58-g8xfm"
Oct 30 19:45:38.819950 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:38.819303    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljrpf\" (UniqueName: \"kubernetes.io/projected/aa2c7806-fc8c-4c51-8108-c14d31c23be8-kube-api-access-ljrpf\") pod \"payload-pre-processing-e2e-api-a-07292b-6c54c6b8c5-m67zk\" (UID: \"aa2c7806-fc8c-4c51-8108-c14d31c23be8\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-07292b-6c54c6b8c5-m67zk"
Oct 30 19:45:38.819950 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:38.819611    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkdkp\" (UniqueName: \"kubernetes.io/projected/ea310863-7424-45c4-a916-8b824e270c40-kube-api-access-fkdkp\") pod \"payload-processing-e2e-api-a-07292b-74459b4d58-g8xfm\" (UID: \"ea310863-7424-45c4-a916-8b824e270c40\") " pod="openshift-ingress/payload-processing-e2e-api-a-07292b-74459b4d58-g8xfm"
Oct 30 19:45:38.819950 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:38.819761    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aa2c7806-fc8c-4c51-8108-c14d31c23be8-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-07292b-6c54c6b8c5-m67zk\" (UID: \"aa2c7806-fc8c-4c51-8108-c14d31c23be8\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-07292b-6c54c6b8c5-m67zk"
Oct 30 19:45:38.820816 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:38.820779    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea310863-7424-45c4-a916-8b824e270c40-plugins-config-volume\") pod \"payload-processing-e2e-api-a-07292b-74459b4d58-g8xfm\" (UID: \"ea310863-7424-45c4-a916-8b824e270c40\") " pod="openshift-ingress/payload-processing-e2e-api-a-07292b-74459b4d58-g8xfm"
Oct 30 19:45:38.820977 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:38.820951    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aa2c7806-fc8c-4c51-8108-c14d31c23be8-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-07292b-6c54c6b8c5-m67zk\" (UID: \"aa2c7806-fc8c-4c51-8108-c14d31c23be8\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-07292b-6c54c6b8c5-m67zk"
Oct 30 19:45:38.838355 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:38.836549    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkdkp\" (UniqueName: \"kubernetes.io/projected/ea310863-7424-45c4-a916-8b824e270c40-kube-api-access-fkdkp\") pod \"payload-processing-e2e-api-a-07292b-74459b4d58-g8xfm\" (UID: \"ea310863-7424-45c4-a916-8b824e270c40\") " pod="openshift-ingress/payload-processing-e2e-api-a-07292b-74459b4d58-g8xfm"
Oct 30 19:45:38.838355 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:38.836655    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljrpf\" (UniqueName: \"kubernetes.io/projected/aa2c7806-fc8c-4c51-8108-c14d31c23be8-kube-api-access-ljrpf\") pod \"payload-pre-processing-e2e-api-a-07292b-6c54c6b8c5-m67zk\" (UID: \"aa2c7806-fc8c-4c51-8108-c14d31c23be8\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-07292b-6c54c6b8c5-m67zk"
Oct 30 19:45:38.879652 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:38.879615    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-07292b-74459b4d58-g8xfm"
Oct 30 19:45:38.892956 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:38.892927    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-07292b-6c54c6b8c5-m67zk"
Oct 30 19:45:39.178549 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:39.178516    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-07292b-74459b4d58-g8xfm"]
Oct 30 19:45:39.179896 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:45:39.179868    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea310863_7424_45c4_a916_8b824e270c40.slice/crio-8c1abbe9eb7f8e80afe30712ad2f1c9fd673f04914eb1b92b7bc41d2f7c637c5 WatchSource:0}: Error finding container 8c1abbe9eb7f8e80afe30712ad2f1c9fd673f04914eb1b92b7bc41d2f7c637c5: Status 404 returned error can't find the container with id 8c1abbe9eb7f8e80afe30712ad2f1c9fd673f04914eb1b92b7bc41d2f7c637c5
Oct 30 19:45:39.218305 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:39.218279    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-07292b-6c54c6b8c5-m67zk"]
Oct 30 19:45:39.220981 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:45:39.220954    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa2c7806_fc8c_4c51_8108_c14d31c23be8.slice/crio-94ab9bef0173608cb2126f002f8f49247721d49b68ef451bcaf15fe40b8fb036 WatchSource:0}: Error finding container 94ab9bef0173608cb2126f002f8f49247721d49b68ef451bcaf15fe40b8fb036: Status 404 returned error can't find the container with id 94ab9bef0173608cb2126f002f8f49247721d49b68ef451bcaf15fe40b8fb036
Oct 30 19:45:40.020802 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:40.020761    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-07292b-6c54c6b8c5-m67zk" event={"ID":"aa2c7806-fc8c-4c51-8108-c14d31c23be8","Type":"ContainerStarted","Data":"038ff78af4900cbe4f698e8eaeaf93c02cce384476db6db082634cf5f7eed454"}
Oct 30 19:45:40.021314 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:40.020831    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-07292b-6c54c6b8c5-m67zk" event={"ID":"aa2c7806-fc8c-4c51-8108-c14d31c23be8","Type":"ContainerStarted","Data":"94ab9bef0173608cb2126f002f8f49247721d49b68ef451bcaf15fe40b8fb036"}
Oct 30 19:45:40.021314 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:40.020985    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-07292b-6c54c6b8c5-m67zk"
Oct 30 19:45:40.022913 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:40.022880    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-07292b-74459b4d58-g8xfm" event={"ID":"ea310863-7424-45c4-a916-8b824e270c40","Type":"ContainerStarted","Data":"4abbcbe25c12aad6df334d0654dc60fa2111c41fc20b1868f79e3474685b051c"}
Oct 30 19:45:40.023045 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:40.022936    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-07292b-74459b4d58-g8xfm" event={"ID":"ea310863-7424-45c4-a916-8b824e270c40","Type":"ContainerStarted","Data":"8c1abbe9eb7f8e80afe30712ad2f1c9fd673f04914eb1b92b7bc41d2f7c637c5"}
Oct 30 19:45:40.023045 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:40.022977    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-07292b-74459b4d58-g8xfm"
Oct 30 19:45:40.043562 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:40.043510    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-07292b-6c54c6b8c5-m67zk" podStartSLOduration=2.043495363 podStartE2EDuration="2.043495363s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:45:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:45:40.038131629 +0000 UTC m=+815.366249782" watchObservedRunningTime="2026-10-30 19:45:40.043495363 +0000 UTC m=+815.371613513"
Oct 30 19:45:40.067991 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:40.067924    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-07292b-74459b4d58-g8xfm" podStartSLOduration=2.067904778 podStartE2EDuration="2.067904778s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:45:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:45:40.063277979 +0000 UTC m=+815.391396134" watchObservedRunningTime="2026-10-30 19:45:40.067904778 +0000 UTC m=+815.396022934"
Oct 30 19:45:44.481091 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:44.481060    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-07292b-6ddc569f9d-tjwhr"]
Oct 30 19:45:44.486066 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:44.486041    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-07292b-6ddc569f9d-tjwhr"
Oct 30 19:45:44.492994 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:44.492969    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-07292b-serving-cert\""
Oct 30 19:45:44.498587 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:44.498554    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-07292b-6ddc569f9d-tjwhr"]
Oct 30 19:45:44.605728 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:44.605691    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbjjs\" (UniqueName: \"kubernetes.io/projected/c0cdfad7-6226-43d7-9c51-5551852639ba-kube-api-access-qbjjs\") pod \"maas-api-e2e-api-a-07292b-6ddc569f9d-tjwhr\" (UID: \"c0cdfad7-6226-43d7-9c51-5551852639ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-07292b-6ddc569f9d-tjwhr"
Oct 30 19:45:44.605901 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:44.605760    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0cdfad7-6226-43d7-9c51-5551852639ba-metrics-certs\") pod \"maas-api-e2e-api-a-07292b-6ddc569f9d-tjwhr\" (UID: \"c0cdfad7-6226-43d7-9c51-5551852639ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-07292b-6ddc569f9d-tjwhr"
Oct 30 19:45:44.605901 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:44.605860    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0cdfad7-6226-43d7-9c51-5551852639ba-openshift-service-ca\") pod \"maas-api-e2e-api-a-07292b-6ddc569f9d-tjwhr\" (UID: \"c0cdfad7-6226-43d7-9c51-5551852639ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-07292b-6ddc569f9d-tjwhr"
Oct 30 19:45:44.606017 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:44.605996    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0cdfad7-6226-43d7-9c51-5551852639ba-maas-api-tls\") pod \"maas-api-e2e-api-a-07292b-6ddc569f9d-tjwhr\" (UID: \"c0cdfad7-6226-43d7-9c51-5551852639ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-07292b-6ddc569f9d-tjwhr"
Oct 30 19:45:44.708345 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:44.708301    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0cdfad7-6226-43d7-9c51-5551852639ba-metrics-certs\") pod \"maas-api-e2e-api-a-07292b-6ddc569f9d-tjwhr\" (UID: \"c0cdfad7-6226-43d7-9c51-5551852639ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-07292b-6ddc569f9d-tjwhr"
Oct 30 19:45:44.708685 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:44.708398    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0cdfad7-6226-43d7-9c51-5551852639ba-openshift-service-ca\") pod \"maas-api-e2e-api-a-07292b-6ddc569f9d-tjwhr\" (UID: \"c0cdfad7-6226-43d7-9c51-5551852639ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-07292b-6ddc569f9d-tjwhr"
Oct 30 19:45:44.708685 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:44.708593    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0cdfad7-6226-43d7-9c51-5551852639ba-maas-api-tls\") pod \"maas-api-e2e-api-a-07292b-6ddc569f9d-tjwhr\" (UID: \"c0cdfad7-6226-43d7-9c51-5551852639ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-07292b-6ddc569f9d-tjwhr"
Oct 30 19:45:44.708909 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:44.708768    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbjjs\" (UniqueName: \"kubernetes.io/projected/c0cdfad7-6226-43d7-9c51-5551852639ba-kube-api-access-qbjjs\") pod \"maas-api-e2e-api-a-07292b-6ddc569f9d-tjwhr\" (UID: \"c0cdfad7-6226-43d7-9c51-5551852639ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-07292b-6ddc569f9d-tjwhr"
Oct 30 19:45:44.710737 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:44.710416    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0cdfad7-6226-43d7-9c51-5551852639ba-openshift-service-ca\") pod \"maas-api-e2e-api-a-07292b-6ddc569f9d-tjwhr\" (UID: \"c0cdfad7-6226-43d7-9c51-5551852639ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-07292b-6ddc569f9d-tjwhr"
Oct 30 19:45:44.742301 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:44.740832    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0cdfad7-6226-43d7-9c51-5551852639ba-maas-api-tls\") pod \"maas-api-e2e-api-a-07292b-6ddc569f9d-tjwhr\" (UID: \"c0cdfad7-6226-43d7-9c51-5551852639ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-07292b-6ddc569f9d-tjwhr"
Oct 30 19:45:44.757822 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:44.757199    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbjjs\" (UniqueName: \"kubernetes.io/projected/c0cdfad7-6226-43d7-9c51-5551852639ba-kube-api-access-qbjjs\") pod \"maas-api-e2e-api-a-07292b-6ddc569f9d-tjwhr\" (UID: \"c0cdfad7-6226-43d7-9c51-5551852639ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-07292b-6ddc569f9d-tjwhr"
Oct 30 19:45:44.760940 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:44.760897    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0cdfad7-6226-43d7-9c51-5551852639ba-metrics-certs\") pod \"maas-api-e2e-api-a-07292b-6ddc569f9d-tjwhr\" (UID: \"c0cdfad7-6226-43d7-9c51-5551852639ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-07292b-6ddc569f9d-tjwhr"
Oct 30 19:45:44.812857 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:44.812823    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-07292b-6ddc569f9d-tjwhr"
Oct 30 19:45:45.110324 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:45.110290    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-07292b-6ddc569f9d-tjwhr"]
Oct 30 19:45:45.124127 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:45:45.124087    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0cdfad7_6226_43d7_9c51_5551852639ba.slice/crio-90f3f4d108fb24ab0cf19752d6e81dc20939d8e81dbb31de05e654a8f41aa8d5 WatchSource:0}: Error finding container 90f3f4d108fb24ab0cf19752d6e81dc20939d8e81dbb31de05e654a8f41aa8d5: Status 404 returned error can't find the container with id 90f3f4d108fb24ab0cf19752d6e81dc20939d8e81dbb31de05e654a8f41aa8d5
Oct 30 19:45:45.125284 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:45.125265    6996 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 19:45:46.066146 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:46.066117    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-07292b-6ddc569f9d-tjwhr" event={"ID":"c0cdfad7-6226-43d7-9c51-5551852639ba","Type":"ContainerStarted","Data":"00fbc687bfa38dc6e13eb6fcb6916967456bc1823275974a557aaa14dccc84f1"}
Oct 30 19:45:46.066532 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:46.066172    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-07292b-6ddc569f9d-tjwhr" event={"ID":"c0cdfad7-6226-43d7-9c51-5551852639ba","Type":"ContainerStarted","Data":"90f3f4d108fb24ab0cf19752d6e81dc20939d8e81dbb31de05e654a8f41aa8d5"}
Oct 30 19:45:46.066532 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:46.066325    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-07292b-6ddc569f9d-tjwhr"
Oct 30 19:45:46.107371 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:46.107321    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-07292b-6ddc569f9d-tjwhr" podStartSLOduration=1.5940186299999999 podStartE2EDuration="2.107306292s" totalImagesPullingTime="513.287662ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:45:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:45:46.105137853 +0000 UTC m=+821.433256007" watchObservedRunningTime="2026-10-30 19:45:46.107306292 +0000 UTC m=+821.435424445"
Oct 30 19:45:51.032839 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:51.032808    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-07292b-74459b4d58-g8xfm"
Oct 30 19:45:51.033248 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:51.032872    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-07292b-6c54c6b8c5-m67zk"
Oct 30 19:45:52.079524 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:52.079449    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-07292b-6ddc569f9d-tjwhr"
Oct 30 19:45:54.495510 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:54.494536    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8"]
Oct 30 19:45:54.580506 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:54.567508    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8"]
Oct 30 19:45:54.585610 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:54.567602    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8"
Oct 30 19:45:54.691849 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:54.691798    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/003d559b-ba77-46e7-b4bc-4b9dd37653c5-workload-socket\") pod \"e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8\" (UID: \"003d559b-ba77-46e7-b4bc-4b9dd37653c5\") " pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8"
Oct 30 19:45:54.692220 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:54.692185    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/003d559b-ba77-46e7-b4bc-4b9dd37653c5-istio-data\") pod \"e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8\" (UID: \"003d559b-ba77-46e7-b4bc-4b9dd37653c5\") " pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8"
Oct 30 19:45:54.692392 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:54.692373    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/003d559b-ba77-46e7-b4bc-4b9dd37653c5-istio-podinfo\") pod \"e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8\" (UID: \"003d559b-ba77-46e7-b4bc-4b9dd37653c5\") " pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8"
Oct 30 19:45:54.692552 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:54.692532    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p22fk\" (UniqueName: \"kubernetes.io/projected/003d559b-ba77-46e7-b4bc-4b9dd37653c5-kube-api-access-p22fk\") pod \"e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8\" (UID: \"003d559b-ba77-46e7-b4bc-4b9dd37653c5\") " pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8"
Oct 30 19:45:54.692742 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:54.692713    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/003d559b-ba77-46e7-b4bc-4b9dd37653c5-credential-socket\") pod \"e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8\" (UID: \"003d559b-ba77-46e7-b4bc-4b9dd37653c5\") " pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8"
Oct 30 19:45:54.692891 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:54.692876    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/003d559b-ba77-46e7-b4bc-4b9dd37653c5-istio-token\") pod \"e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8\" (UID: \"003d559b-ba77-46e7-b4bc-4b9dd37653c5\") " pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8"
Oct 30 19:45:54.693025 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:54.693010    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/003d559b-ba77-46e7-b4bc-4b9dd37653c5-istio-envoy\") pod \"e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8\" (UID: \"003d559b-ba77-46e7-b4bc-4b9dd37653c5\") " pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8"
Oct 30 19:45:54.693358 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:54.693324    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/003d559b-ba77-46e7-b4bc-4b9dd37653c5-workload-certs\") pod \"e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8\" (UID: \"003d559b-ba77-46e7-b4bc-4b9dd37653c5\") " pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8"
Oct 30 19:45:54.693650 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:54.693630    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/003d559b-ba77-46e7-b4bc-4b9dd37653c5-istiod-ca-cert\") pod \"e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8\" (UID: \"003d559b-ba77-46e7-b4bc-4b9dd37653c5\") " pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8"
Oct 30 19:45:54.752569 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:54.744947    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s" podUID="ae905dfb-f30d-480a-899b-ea676abc1d8d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:45:54.753617 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:54.753571    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk" podUID="a1857d2d-3b40-4e63-be33-be4ba476f926" containerName="main" probeResult="failure" output="Get \"https://10.134.0.90:8000/ready\": context deadline exceeded"
Oct 30 19:45:54.807493 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:54.807434    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/003d559b-ba77-46e7-b4bc-4b9dd37653c5-workload-certs\") pod \"e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8\" (UID: \"003d559b-ba77-46e7-b4bc-4b9dd37653c5\") " pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8"
Oct 30 19:45:54.807775 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:54.807749    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/003d559b-ba77-46e7-b4bc-4b9dd37653c5-istiod-ca-cert\") pod \"e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8\" (UID: \"003d559b-ba77-46e7-b4bc-4b9dd37653c5\") " pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8"
Oct 30 19:45:54.808022 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:54.808002    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/003d559b-ba77-46e7-b4bc-4b9dd37653c5-workload-socket\") pod \"e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8\" (UID: \"003d559b-ba77-46e7-b4bc-4b9dd37653c5\") " pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8"
Oct 30 19:45:54.808159 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:54.808141    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/003d559b-ba77-46e7-b4bc-4b9dd37653c5-istio-data\") pod \"e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8\" (UID: \"003d559b-ba77-46e7-b4bc-4b9dd37653c5\") " pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8"
Oct 30 19:45:54.808287 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:54.808272    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/003d559b-ba77-46e7-b4bc-4b9dd37653c5-istio-podinfo\") pod \"e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8\" (UID: \"003d559b-ba77-46e7-b4bc-4b9dd37653c5\") " pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8"
Oct 30 19:45:54.808397 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:54.808380    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p22fk\" (UniqueName: \"kubernetes.io/projected/003d559b-ba77-46e7-b4bc-4b9dd37653c5-kube-api-access-p22fk\") pod \"e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8\" (UID: \"003d559b-ba77-46e7-b4bc-4b9dd37653c5\") " pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8"
Oct 30 19:45:54.813625 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:54.813591    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/003d559b-ba77-46e7-b4bc-4b9dd37653c5-credential-socket\") pod \"e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8\" (UID: \"003d559b-ba77-46e7-b4bc-4b9dd37653c5\") " pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8"
Oct 30 19:45:54.813848 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:54.813827    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/003d559b-ba77-46e7-b4bc-4b9dd37653c5-istio-token\") pod \"e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8\" (UID: \"003d559b-ba77-46e7-b4bc-4b9dd37653c5\") " pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8"
Oct 30 19:45:54.813988 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:54.813969    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/003d559b-ba77-46e7-b4bc-4b9dd37653c5-istio-envoy\") pod \"e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8\" (UID: \"003d559b-ba77-46e7-b4bc-4b9dd37653c5\") " pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8"
Oct 30 19:45:54.815847 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:54.815819    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/003d559b-ba77-46e7-b4bc-4b9dd37653c5-workload-certs\") pod \"e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8\" (UID: \"003d559b-ba77-46e7-b4bc-4b9dd37653c5\") " pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8"
Oct 30 19:45:54.816637 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:54.816617    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/003d559b-ba77-46e7-b4bc-4b9dd37653c5-credential-socket\") pod \"e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8\" (UID: \"003d559b-ba77-46e7-b4bc-4b9dd37653c5\") " pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8"
Oct 30 19:45:54.817352 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:54.817329    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/003d559b-ba77-46e7-b4bc-4b9dd37653c5-workload-socket\") pod \"e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8\" (UID: \"003d559b-ba77-46e7-b4bc-4b9dd37653c5\") " pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8"
Oct 30 19:45:54.818136 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:54.818115    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/003d559b-ba77-46e7-b4bc-4b9dd37653c5-istiod-ca-cert\") pod \"e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8\" (UID: \"003d559b-ba77-46e7-b4bc-4b9dd37653c5\") " pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8"
Oct 30 19:45:54.818514 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:54.818494    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/003d559b-ba77-46e7-b4bc-4b9dd37653c5-istio-data\") pod \"e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8\" (UID: \"003d559b-ba77-46e7-b4bc-4b9dd37653c5\") " pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8"
Oct 30 19:45:54.826978 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:54.825563    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b" podUID="19fe4fba-09ae-4620-b4d2-0ef8c2463d71" containerName="main" probeResult="failure" output="Get \"https://10.134.0.81:8000/ready\": context deadline exceeded"
Oct 30 19:45:54.852745 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:54.849924    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/003d559b-ba77-46e7-b4bc-4b9dd37653c5-istio-envoy\") pod \"e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8\" (UID: \"003d559b-ba77-46e7-b4bc-4b9dd37653c5\") " pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8"
Oct 30 19:45:54.924503 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:54.914268    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/003d559b-ba77-46e7-b4bc-4b9dd37653c5-istio-podinfo\") pod \"e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8\" (UID: \"003d559b-ba77-46e7-b4bc-4b9dd37653c5\") " pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8"
Oct 30 19:45:54.979507 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:54.975891    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p22fk\" (UniqueName: \"kubernetes.io/projected/003d559b-ba77-46e7-b4bc-4b9dd37653c5-kube-api-access-p22fk\") pod \"e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8\" (UID: \"003d559b-ba77-46e7-b4bc-4b9dd37653c5\") " pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8"
Oct 30 19:45:55.016591 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:54.983002    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/003d559b-ba77-46e7-b4bc-4b9dd37653c5-istio-token\") pod \"e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8\" (UID: \"003d559b-ba77-46e7-b4bc-4b9dd37653c5\") " pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8"
Oct 30 19:45:55.080795 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:55.080758    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8"
Oct 30 19:45:55.197957 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:55.197580    6996 patch_prober.go:29] interesting pod/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:45:55.197957 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:55.197643    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v" podUID="b99ab818-3be9-406e-be18-c0ed26c43bea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:45:55.301113 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:55.300595    6996 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s" podUID="ae905dfb-f30d-480a-899b-ea676abc1d8d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.73:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:45:55.379757 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:55.379068    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr" podUID="0e470010-625d-443f-81ec-9bf965988212" containerName="main" probeResult="failure" output="Get \"https://10.134.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:45:55.415960 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:55.415601    6996 patch_prober.go:29] interesting pod/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:45:55.415960 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:55.415668    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw" podUID="e41df835-fccd-45bb-bfca-ca1f81f6261a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:45:55.520503 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:55.517171    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq" podUID="89009638-003d-4a5d-ab13-1f8c8f64d3b7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.88:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:45:55.750561 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:55.749876    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s" podUID="ae905dfb-f30d-480a-899b-ea676abc1d8d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:45:55.751705 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:55.750756    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s"
Oct 30 19:45:55.755594 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:55.755557    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk" podUID="a1857d2d-3b40-4e63-be33-be4ba476f926" containerName="main" probeResult="failure" output="Get \"https://10.134.0.90:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:45:55.756093 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:55.756070    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk"
Oct 30 19:45:55.866651 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:55.862082    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b" podUID="19fe4fba-09ae-4620-b4d2-0ef8c2463d71" containerName="main" probeResult="failure" output="Get \"https://10.134.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:45:55.866651 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:55.862824    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b"
Oct 30 19:45:56.049979 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:56.049600    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-bw5vt" podUID="5e7e4cc1-bbcd-40f9-9daf-0f2051e23a04" containerName="cert-manager-webhook" probeResult="failure" output="Get \"http://10.134.0.29:6080/healthz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:45:56.095512 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:56.086283    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b"
Oct 30 19:45:56.467212 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:56.440972    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr" podUID="0e470010-625d-443f-81ec-9bf965988212" containerName="main" probeResult="failure" output="Get \"https://10.134.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:45:56.467212 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:56.462557    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr"
Oct 30 19:45:56.532335 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:56.515646    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:45:56.532335 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:56.515750    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:45:56.532335 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:56.515794    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:45:56.688891 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:56.688802    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr"
Oct 30 19:45:56.795524 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:56.778868    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk" podUID="a1857d2d-3b40-4e63-be33-be4ba476f926" containerName="main" probeResult="failure" output="Get \"https://10.134.0.90:8000/ready\": context deadline exceeded"
Oct 30 19:45:56.810507 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:56.809460    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s" podUID="ae905dfb-f30d-480a-899b-ea676abc1d8d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.73:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:45:56.891918 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:56.885742    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s"
Oct 30 19:45:56.897014 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:56.896179    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8" event={"ID":"003d559b-ba77-46e7-b4bc-4b9dd37653c5","Type":"ContainerStarted","Data":"73fb6e8d9f7b04860c85f886467004fcf470e6257128c9a1878e624af86d272d"}
Oct 30 19:45:56.897014 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:56.896240    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8"]
Oct 30 19:45:56.897677 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:56.897653    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk"
Oct 30 19:45:57.868544 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:57.865646    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8" event={"ID":"003d559b-ba77-46e7-b4bc-4b9dd37653c5","Type":"ContainerStarted","Data":"5ffa3bc2e7a928b969ae3ce2229f6388ac686dd1f31f98e37ec023c19fd927dd"}
Oct 30 19:45:57.931522 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:57.929852    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8" podStartSLOduration=4.929832558 podStartE2EDuration="4.929832558s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:45:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:45:57.917742257 +0000 UTC m=+833.245860412" watchObservedRunningTime="2026-10-30 19:45:57.929832558 +0000 UTC m=+833.257950713"
Oct 30 19:45:58.086502 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:58.085472    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8"
Oct 30 19:45:58.086502 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:58.085623    6996 patch_prober.go:29] interesting pod/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused" start-of-body=
Oct 30 19:45:58.086502 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:58.085669    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8" podUID="003d559b-ba77-46e7-b4bc-4b9dd37653c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused"
Oct 30 19:45:59.086506 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:59.085837    6996 patch_prober.go:29] interesting pod/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused" start-of-body=
Oct 30 19:45:59.086506 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:45:59.085885    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8" podUID="003d559b-ba77-46e7-b4bc-4b9dd37653c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused"
Oct 30 19:46:00.083295 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:00.082831    6996 patch_prober.go:29] interesting pod/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused" start-of-body=
Oct 30 19:46:00.083295 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:00.082890    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8" podUID="003d559b-ba77-46e7-b4bc-4b9dd37653c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused"
Oct 30 19:46:01.081824 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:01.081772    6996 patch_prober.go:29] interesting pod/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused" start-of-body=
Oct 30 19:46:01.082296 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:01.081848    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8" podUID="003d559b-ba77-46e7-b4bc-4b9dd37653c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused"
Oct 30 19:46:01.205381 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:46:01.205341    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 19:46:01.990982 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:01.990937    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-6ac47d-7544f97974-s4cp5"]
Oct 30 19:46:01.997812 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:01.997785    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-6ac47d-7544f97974-s4cp5"
Oct 30 19:46:02.004625 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:02.004593    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-6ac47d-7544f97974-s4cp5"]
Oct 30 19:46:02.033254 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:02.033211    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zd7fz\" (UniqueName: \"kubernetes.io/projected/5bf6d5d9-fb0a-48e6-bbb1-cf6a161e2c6c-kube-api-access-zd7fz\") pod \"payload-processing-e2e-api-b-6ac47d-7544f97974-s4cp5\" (UID: \"5bf6d5d9-fb0a-48e6-bbb1-cf6a161e2c6c\") " pod="openshift-ingress/payload-processing-e2e-api-b-6ac47d-7544f97974-s4cp5"
Oct 30 19:46:02.033772 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:02.033747    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5bf6d5d9-fb0a-48e6-bbb1-cf6a161e2c6c-plugins-config-volume\") pod \"payload-processing-e2e-api-b-6ac47d-7544f97974-s4cp5\" (UID: \"5bf6d5d9-fb0a-48e6-bbb1-cf6a161e2c6c\") " pod="openshift-ingress/payload-processing-e2e-api-b-6ac47d-7544f97974-s4cp5"
Oct 30 19:46:02.150504 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:02.071789    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-6ac47d-dockercfg-7g7ss\""
Oct 30 19:46:02.220544 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:02.098684    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-6ac47d\""
Oct 30 19:46:02.220544 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:02.099513    6996 patch_prober.go:29] interesting pod/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused" start-of-body=
Oct 30 19:46:02.220544 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:02.157551    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8" podUID="003d559b-ba77-46e7-b4bc-4b9dd37653c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused"
Oct 30 19:46:02.220544 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:02.136322    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zd7fz\" (UniqueName: \"kubernetes.io/projected/5bf6d5d9-fb0a-48e6-bbb1-cf6a161e2c6c-kube-api-access-zd7fz\") pod \"payload-processing-e2e-api-b-6ac47d-7544f97974-s4cp5\" (UID: \"5bf6d5d9-fb0a-48e6-bbb1-cf6a161e2c6c\") " pod="openshift-ingress/payload-processing-e2e-api-b-6ac47d-7544f97974-s4cp5"
Oct 30 19:46:02.220544 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:02.158271    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5bf6d5d9-fb0a-48e6-bbb1-cf6a161e2c6c-plugins-config-volume\") pod \"payload-processing-e2e-api-b-6ac47d-7544f97974-s4cp5\" (UID: \"5bf6d5d9-fb0a-48e6-bbb1-cf6a161e2c6c\") " pod="openshift-ingress/payload-processing-e2e-api-b-6ac47d-7544f97974-s4cp5"
Oct 30 19:46:02.220544 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:02.159354    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5bf6d5d9-fb0a-48e6-bbb1-cf6a161e2c6c-plugins-config-volume\") pod \"payload-processing-e2e-api-b-6ac47d-7544f97974-s4cp5\" (UID: \"5bf6d5d9-fb0a-48e6-bbb1-cf6a161e2c6c\") " pod="openshift-ingress/payload-processing-e2e-api-b-6ac47d-7544f97974-s4cp5"
Oct 30 19:46:02.220544 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:02.187339    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zd7fz\" (UniqueName: \"kubernetes.io/projected/5bf6d5d9-fb0a-48e6-bbb1-cf6a161e2c6c-kube-api-access-zd7fz\") pod \"payload-processing-e2e-api-b-6ac47d-7544f97974-s4cp5\" (UID: \"5bf6d5d9-fb0a-48e6-bbb1-cf6a161e2c6c\") " pod="openshift-ingress/payload-processing-e2e-api-b-6ac47d-7544f97974-s4cp5"
Oct 30 19:46:02.324308 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:02.324255    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-6ac47d-7544f97974-s4cp5"
Oct 30 19:46:02.732513 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:02.731949    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-6ac47d-7544f97974-s4cp5"]
Oct 30 19:46:02.975762 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:02.975672    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-6ac47d-7544f97974-s4cp5" event={"ID":"5bf6d5d9-fb0a-48e6-bbb1-cf6a161e2c6c","Type":"ContainerStarted","Data":"5e516b182f6f30de0aea26f39cd5dee5165eaa69d0491e4318f0d4ddb044f8e0"}
Oct 30 19:46:02.975762 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:02.975741    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-6ac47d-7544f97974-s4cp5" event={"ID":"5bf6d5d9-fb0a-48e6-bbb1-cf6a161e2c6c","Type":"ContainerStarted","Data":"4e4373024340a918aa824dae6abd3d2afa3c36ef2f3e61806055e50895d8c969"}
Oct 30 19:46:02.975934 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:02.975793    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-6ac47d-7544f97974-s4cp5"
Oct 30 19:46:03.081765 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:03.081728    6996 patch_prober.go:29] interesting pod/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused" start-of-body=
Oct 30 19:46:03.081937 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:03.081797    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8" podUID="003d559b-ba77-46e7-b4bc-4b9dd37653c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused"
Oct 30 19:46:03.766682 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:03.766637    6996 patch_prober.go:29] interesting pod/e2e-api-a-07292b-openshift-default-744696b8d9-pljg8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:46:03.767105 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:03.766703    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-07292b-openshift-default-744696b8d9-pljg8" podUID="a8797217-8ec7-46db-8f6a-4a11f77164f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:46:04.601474 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:04.601444    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8"
Oct 30 19:46:04.601917 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:04.601888    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8"
Oct 30 19:46:04.602827 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:04.602805    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8"
Oct 30 19:46:04.627749 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:04.627697    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-6ac47d-7544f97974-s4cp5" podStartSLOduration=3.627682849 podStartE2EDuration="3.627682849s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:46:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:46:02.998754245 +0000 UTC m=+838.326872400" watchObservedRunningTime="2026-10-30 19:46:04.627682849 +0000 UTC m=+839.955801005"
Oct 30 19:46:06.016943 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:06.016847    6996 generic.go:356] "Generic (PLEG): container finished" podID="a8797217-8ec7-46db-8f6a-4a11f77164f6" containerID="5b83384e092dbde5ebf88716e5b3c3c1297a9079b809893e40eb2874ceb3db5b" exitCode=0
Oct 30 19:46:06.017440 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:06.016928    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-07292b-openshift-default-744696b8d9-pljg8" event={"ID":"a8797217-8ec7-46db-8f6a-4a11f77164f6","Type":"ContainerDied","Data":"5b83384e092dbde5ebf88716e5b3c3c1297a9079b809893e40eb2874ceb3db5b"}
Oct 30 19:46:06.017920 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:06.017900    6996 scope.go:122] "RemoveContainer" containerID="5b83384e092dbde5ebf88716e5b3c3c1297a9079b809893e40eb2874ceb3db5b"
Oct 30 19:46:06.019781 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:06.019744    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:46:06.019943 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:06.019868    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:46:06.019943 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:06.019924    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:46:07.026895 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:07.026860    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-07292b-openshift-default-744696b8d9-pljg8" event={"ID":"a8797217-8ec7-46db-8f6a-4a11f77164f6","Type":"ContainerStarted","Data":"be439269355e42075078e4c05234e78ba2f4f1316052b50844bd1edb7db109e3"}
Oct 30 19:46:07.514392 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:07.514356    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-07292b-openshift-default-744696b8d9-pljg8"
Oct 30 19:46:07.514630 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:07.514601    6996 patch_prober.go:29] interesting pod/e2e-api-a-07292b-openshift-default-744696b8d9-pljg8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Oct 30 19:46:07.514723 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:07.514654    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-07292b-openshift-default-744696b8d9-pljg8" podUID="a8797217-8ec7-46db-8f6a-4a11f77164f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Oct 30 19:46:08.514204 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:08.514173    6996 patch_prober.go:29] interesting pod/e2e-api-a-07292b-openshift-default-744696b8d9-pljg8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Oct 30 19:46:08.514601 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:08.514223    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-07292b-openshift-default-744696b8d9-pljg8" podUID="a8797217-8ec7-46db-8f6a-4a11f77164f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Oct 30 19:46:09.064505 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:09.060903    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-6ac47d-67967b944d-p26th"]
Oct 30 19:46:09.093991 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:09.092777    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-6ac47d-67967b944d-p26th"]
Oct 30 19:46:09.093991 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:09.093254    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ac47d-67967b944d-p26th"
Oct 30 19:46:09.102500 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:09.099867    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-6ac47d-serving-cert\""
Oct 30 19:46:09.205501 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:09.205458    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d36e9a4f-49b3-4544-a98c-536aace46c1f-maas-api-tls\") pod \"maas-api-e2e-api-b-6ac47d-67967b944d-p26th\" (UID: \"d36e9a4f-49b3-4544-a98c-536aace46c1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ac47d-67967b944d-p26th"
Oct 30 19:46:09.205674 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:09.205539    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njvkz\" (UniqueName: \"kubernetes.io/projected/d36e9a4f-49b3-4544-a98c-536aace46c1f-kube-api-access-njvkz\") pod \"maas-api-e2e-api-b-6ac47d-67967b944d-p26th\" (UID: \"d36e9a4f-49b3-4544-a98c-536aace46c1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ac47d-67967b944d-p26th"
Oct 30 19:46:09.205674 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:09.205596    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d36e9a4f-49b3-4544-a98c-536aace46c1f-openshift-service-ca\") pod \"maas-api-e2e-api-b-6ac47d-67967b944d-p26th\" (UID: \"d36e9a4f-49b3-4544-a98c-536aace46c1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ac47d-67967b944d-p26th"
Oct 30 19:46:09.205795 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:09.205782    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d36e9a4f-49b3-4544-a98c-536aace46c1f-metrics-certs\") pod \"maas-api-e2e-api-b-6ac47d-67967b944d-p26th\" (UID: \"d36e9a4f-49b3-4544-a98c-536aace46c1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ac47d-67967b944d-p26th"
Oct 30 19:46:09.322501 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:09.319412    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d36e9a4f-49b3-4544-a98c-536aace46c1f-openshift-service-ca\") pod \"maas-api-e2e-api-b-6ac47d-67967b944d-p26th\" (UID: \"d36e9a4f-49b3-4544-a98c-536aace46c1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ac47d-67967b944d-p26th"
Oct 30 19:46:09.322501 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:09.319582    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d36e9a4f-49b3-4544-a98c-536aace46c1f-metrics-certs\") pod \"maas-api-e2e-api-b-6ac47d-67967b944d-p26th\" (UID: \"d36e9a4f-49b3-4544-a98c-536aace46c1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ac47d-67967b944d-p26th"
Oct 30 19:46:09.322501 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:09.319745    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d36e9a4f-49b3-4544-a98c-536aace46c1f-maas-api-tls\") pod \"maas-api-e2e-api-b-6ac47d-67967b944d-p26th\" (UID: \"d36e9a4f-49b3-4544-a98c-536aace46c1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ac47d-67967b944d-p26th"
Oct 30 19:46:09.322501 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:09.319817    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njvkz\" (UniqueName: \"kubernetes.io/projected/d36e9a4f-49b3-4544-a98c-536aace46c1f-kube-api-access-njvkz\") pod \"maas-api-e2e-api-b-6ac47d-67967b944d-p26th\" (UID: \"d36e9a4f-49b3-4544-a98c-536aace46c1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ac47d-67967b944d-p26th"
Oct 30 19:46:09.322501 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:09.320877    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d36e9a4f-49b3-4544-a98c-536aace46c1f-openshift-service-ca\") pod \"maas-api-e2e-api-b-6ac47d-67967b944d-p26th\" (UID: \"d36e9a4f-49b3-4544-a98c-536aace46c1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ac47d-67967b944d-p26th"
Oct 30 19:46:09.322501 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:46:09.321671    6996 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-6ac47d-serving-cert: secret "maas-api-e2e-api-b-6ac47d-serving-cert" not found
Oct 30 19:46:09.322501 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:46:09.321731    6996 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d36e9a4f-49b3-4544-a98c-536aace46c1f-maas-api-tls podName:d36e9a4f-49b3-4544-a98c-536aace46c1f nodeName:}" failed. No retries permitted until 2026-10-30 19:46:09.821714181 +0000 UTC m=+845.149832330 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d36e9a4f-49b3-4544-a98c-536aace46c1f-maas-api-tls") pod "maas-api-e2e-api-b-6ac47d-67967b944d-p26th" (UID: "d36e9a4f-49b3-4544-a98c-536aace46c1f") : secret "maas-api-e2e-api-b-6ac47d-serving-cert" not found
Oct 30 19:46:09.357506 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:09.355318    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-njvkz\" (UniqueName: \"kubernetes.io/projected/d36e9a4f-49b3-4544-a98c-536aace46c1f-kube-api-access-njvkz\") pod \"maas-api-e2e-api-b-6ac47d-67967b944d-p26th\" (UID: \"d36e9a4f-49b3-4544-a98c-536aace46c1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ac47d-67967b944d-p26th"
Oct 30 19:46:09.387504 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:09.377601    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d36e9a4f-49b3-4544-a98c-536aace46c1f-metrics-certs\") pod \"maas-api-e2e-api-b-6ac47d-67967b944d-p26th\" (UID: \"d36e9a4f-49b3-4544-a98c-536aace46c1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ac47d-67967b944d-p26th"
Oct 30 19:46:09.514007 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:09.513964    6996 patch_prober.go:29] interesting pod/e2e-api-a-07292b-openshift-default-744696b8d9-pljg8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Oct 30 19:46:09.514212 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:09.514028    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-07292b-openshift-default-744696b8d9-pljg8" podUID="a8797217-8ec7-46db-8f6a-4a11f77164f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Oct 30 19:46:09.825976 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:09.825944    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d36e9a4f-49b3-4544-a98c-536aace46c1f-maas-api-tls\") pod \"maas-api-e2e-api-b-6ac47d-67967b944d-p26th\" (UID: \"d36e9a4f-49b3-4544-a98c-536aace46c1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ac47d-67967b944d-p26th"
Oct 30 19:46:09.829036 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:09.829006    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d36e9a4f-49b3-4544-a98c-536aace46c1f-maas-api-tls\") pod \"maas-api-e2e-api-b-6ac47d-67967b944d-p26th\" (UID: \"d36e9a4f-49b3-4544-a98c-536aace46c1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ac47d-67967b944d-p26th"
Oct 30 19:46:10.026180 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:10.026145    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ac47d-67967b944d-p26th"
Oct 30 19:46:10.364177 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:10.364149    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-6ac47d-67967b944d-p26th"]
Oct 30 19:46:10.366412 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:46:10.366376    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd36e9a4f_49b3_4544_a98c_536aace46c1f.slice/crio-624e00ab51a7f5637abab001727570d319687483ed520ca3b46d3eec23c888cb WatchSource:0}: Error finding container 624e00ab51a7f5637abab001727570d319687483ed520ca3b46d3eec23c888cb: Status 404 returned error can't find the container with id 624e00ab51a7f5637abab001727570d319687483ed520ca3b46d3eec23c888cb
Oct 30 19:46:10.514626 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:10.514585    6996 patch_prober.go:29] interesting pod/e2e-api-a-07292b-openshift-default-744696b8d9-pljg8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Oct 30 19:46:10.514805 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:10.514648    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-07292b-openshift-default-744696b8d9-pljg8" podUID="a8797217-8ec7-46db-8f6a-4a11f77164f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Oct 30 19:46:11.102262 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:11.102176    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ac47d-67967b944d-p26th" event={"ID":"d36e9a4f-49b3-4544-a98c-536aace46c1f","Type":"ContainerStarted","Data":"452e2ad3f1a1435ef84be668200c5410f6c830cc544e33a81a467ca5ed30b7f6"}
Oct 30 19:46:11.102262 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:11.102242    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ac47d-67967b944d-p26th" event={"ID":"d36e9a4f-49b3-4544-a98c-536aace46c1f","Type":"ContainerStarted","Data":"624e00ab51a7f5637abab001727570d319687483ed520ca3b46d3eec23c888cb"}
Oct 30 19:46:11.102734 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:11.102297    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ac47d-67967b944d-p26th"
Oct 30 19:46:11.127044 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:11.126974    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ac47d-67967b944d-p26th" podStartSLOduration=1.677247494 podStartE2EDuration="2.126949067s" totalImagesPullingTime="449.701573ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:46:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:46:11.12684038 +0000 UTC m=+846.454958534" watchObservedRunningTime="2026-10-30 19:46:11.126949067 +0000 UTC m=+846.455067222"
Oct 30 19:46:11.514077 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:11.513973    6996 patch_prober.go:29] interesting pod/e2e-api-a-07292b-openshift-default-744696b8d9-pljg8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Oct 30 19:46:11.514077 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:11.514034    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-07292b-openshift-default-744696b8d9-pljg8" podUID="a8797217-8ec7-46db-8f6a-4a11f77164f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Oct 30 19:46:12.518331 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:12.518301    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-07292b-openshift-default-744696b8d9-pljg8"
Oct 30 19:46:12.518815 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:12.518658    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-07292b-openshift-default-744696b8d9-pljg8"
Oct 30 19:46:12.519384 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:12.519367    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-07292b-openshift-default-744696b8d9-pljg8"
Oct 30 19:46:13.989022 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:13.988994    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-6ac47d-7544f97974-s4cp5"
Oct 30 19:46:17.117171 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:17.117141    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ac47d-67967b944d-p26th"
Oct 30 19:46:20.188764 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:20.188729    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 19:46:31.190630 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:46:31.190592    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 19:46:32.127520 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:32.127473    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-07292b-74459b4d58-g8xfm"]
Oct 30 19:46:32.144540 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:32.144505    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-07292b-6c54c6b8c5-m67zk"]
Oct 30 19:46:37.128103 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:37.128064    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-07292b-74459b4d58-g8xfm" podUID="ea310863-7424-45c4-a916-8b824e270c40" containerName="payload-processing" containerID="cri-o://4abbcbe25c12aad6df334d0654dc60fa2111c41fc20b1868f79e3474685b051c" gracePeriod=25
Oct 30 19:46:37.144860 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:37.144810    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-07292b-6c54c6b8c5-m67zk" podUID="aa2c7806-fc8c-4c51-8108-c14d31c23be8" containerName="payload-pre-processing" containerID="cri-o://038ff78af4900cbe4f698e8eaeaf93c02cce384476db6db082634cf5f7eed454" gracePeriod=25
Oct 30 19:46:37.385998 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:37.385919    6996 generic.go:356] "Generic (PLEG): container finished" podID="aa2c7806-fc8c-4c51-8108-c14d31c23be8" containerID="038ff78af4900cbe4f698e8eaeaf93c02cce384476db6db082634cf5f7eed454" exitCode=0
Oct 30 19:46:37.386163 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:37.386045    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-07292b-6c54c6b8c5-m67zk" event={"ID":"aa2c7806-fc8c-4c51-8108-c14d31c23be8","Type":"ContainerDied","Data":"038ff78af4900cbe4f698e8eaeaf93c02cce384476db6db082634cf5f7eed454"}
Oct 30 19:46:37.387877 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:37.387849    6996 generic.go:356] "Generic (PLEG): container finished" podID="ea310863-7424-45c4-a916-8b824e270c40" containerID="4abbcbe25c12aad6df334d0654dc60fa2111c41fc20b1868f79e3474685b051c" exitCode=0
Oct 30 19:46:37.387999 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:37.387917    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-07292b-74459b4d58-g8xfm" event={"ID":"ea310863-7424-45c4-a916-8b824e270c40","Type":"ContainerDied","Data":"4abbcbe25c12aad6df334d0654dc60fa2111c41fc20b1868f79e3474685b051c"}
Oct 30 19:46:37.465834 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:37.465811    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-07292b-74459b4d58-g8xfm"
Oct 30 19:46:37.469027 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:37.469005    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-07292b-6c54c6b8c5-m67zk"
Oct 30 19:46:37.560353 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:37.560319    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/aa2c7806-fc8c-4c51-8108-c14d31c23be8-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aa2c7806-fc8c-4c51-8108-c14d31c23be8-plugins-config-volume\") pod \"aa2c7806-fc8c-4c51-8108-c14d31c23be8\" (UID: \"aa2c7806-fc8c-4c51-8108-c14d31c23be8\") "
Oct 30 19:46:37.560562 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:37.560420    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aa2c7806-fc8c-4c51-8108-c14d31c23be8-kube-api-access-ljrpf\" (UniqueName: \"kubernetes.io/projected/aa2c7806-fc8c-4c51-8108-c14d31c23be8-kube-api-access-ljrpf\") pod \"aa2c7806-fc8c-4c51-8108-c14d31c23be8\" (UID: \"aa2c7806-fc8c-4c51-8108-c14d31c23be8\") "
Oct 30 19:46:37.560562 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:37.560518    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ea310863-7424-45c4-a916-8b824e270c40-kube-api-access-fkdkp\" (UniqueName: \"kubernetes.io/projected/ea310863-7424-45c4-a916-8b824e270c40-kube-api-access-fkdkp\") pod \"ea310863-7424-45c4-a916-8b824e270c40\" (UID: \"ea310863-7424-45c4-a916-8b824e270c40\") "
Oct 30 19:46:37.560691 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:37.560599    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ea310863-7424-45c4-a916-8b824e270c40-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea310863-7424-45c4-a916-8b824e270c40-plugins-config-volume\") pod \"ea310863-7424-45c4-a916-8b824e270c40\" (UID: \"ea310863-7424-45c4-a916-8b824e270c40\") "
Oct 30 19:46:37.560795 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:37.560766    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa2c7806-fc8c-4c51-8108-c14d31c23be8-plugins-config-volume" pod "aa2c7806-fc8c-4c51-8108-c14d31c23be8" (UID: "aa2c7806-fc8c-4c51-8108-c14d31c23be8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:46:37.561015 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:37.560990    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea310863-7424-45c4-a916-8b824e270c40-plugins-config-volume" pod "ea310863-7424-45c4-a916-8b824e270c40" (UID: "ea310863-7424-45c4-a916-8b824e270c40"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:46:37.561348 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:37.561326    6996 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aa2c7806-fc8c-4c51-8108-c14d31c23be8-plugins-config-volume\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:37.561389 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:37.561357    6996 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea310863-7424-45c4-a916-8b824e270c40-plugins-config-volume\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:37.562574 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:37.562548    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa2c7806-fc8c-4c51-8108-c14d31c23be8-kube-api-access-ljrpf" pod "aa2c7806-fc8c-4c51-8108-c14d31c23be8" (UID: "aa2c7806-fc8c-4c51-8108-c14d31c23be8"). InnerVolumeSpecName "kube-api-access-ljrpf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:46:37.562687 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:37.562605    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea310863-7424-45c4-a916-8b824e270c40-kube-api-access-fkdkp" pod "ea310863-7424-45c4-a916-8b824e270c40" (UID: "ea310863-7424-45c4-a916-8b824e270c40"). InnerVolumeSpecName "kube-api-access-fkdkp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:46:37.663376 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:37.663308    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ljrpf\" (UniqueName: \"kubernetes.io/projected/aa2c7806-fc8c-4c51-8108-c14d31c23be8-kube-api-access-ljrpf\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:37.663376 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:37.663332    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fkdkp\" (UniqueName: \"kubernetes.io/projected/ea310863-7424-45c4-a916-8b824e270c40-kube-api-access-fkdkp\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:38.396532 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:38.396434    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-07292b-6c54c6b8c5-m67zk"
Oct 30 19:46:38.396532 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:38.396471    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-07292b-6c54c6b8c5-m67zk" event={"ID":"aa2c7806-fc8c-4c51-8108-c14d31c23be8","Type":"ContainerDied","Data":"94ab9bef0173608cb2126f002f8f49247721d49b68ef451bcaf15fe40b8fb036"}
Oct 30 19:46:38.396987 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:38.396566    6996 scope.go:122] "RemoveContainer" containerID="038ff78af4900cbe4f698e8eaeaf93c02cce384476db6db082634cf5f7eed454"
Oct 30 19:46:38.398500 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:38.398241    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-07292b-74459b4d58-g8xfm"
Oct 30 19:46:38.398500 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:38.398285    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-07292b-74459b4d58-g8xfm" event={"ID":"ea310863-7424-45c4-a916-8b824e270c40","Type":"ContainerDied","Data":"8c1abbe9eb7f8e80afe30712ad2f1c9fd673f04914eb1b92b7bc41d2f7c637c5"}
Oct 30 19:46:38.411522 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:38.411502    6996 scope.go:122] "RemoveContainer" containerID="4abbcbe25c12aad6df334d0654dc60fa2111c41fc20b1868f79e3474685b051c"
Oct 30 19:46:38.428192 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:38.428168    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-07292b-74459b4d58-g8xfm"]
Oct 30 19:46:38.435129 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:38.435101    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-07292b-74459b4d58-g8xfm"]
Oct 30 19:46:38.446627 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:38.446600    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-07292b-6c54c6b8c5-m67zk"]
Oct 30 19:46:38.458176 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:38.458154    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-07292b-6c54c6b8c5-m67zk"]
Oct 30 19:46:38.792439 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:38.792362    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="aa2c7806-fc8c-4c51-8108-c14d31c23be8" path="/var/lib/kubelet/pods/aa2c7806-fc8c-4c51-8108-c14d31c23be8/volumes"
Oct 30 19:46:38.792859 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:38.792843    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ea310863-7424-45c4-a916-8b824e270c40" path="/var/lib/kubelet/pods/ea310863-7424-45c4-a916-8b824e270c40/volumes"
Oct 30 19:46:44.377805 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:44.377770    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 19:46:44.454300 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:44.448792    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-07292b-6ddc569f9d-tjwhr"]
Oct 30 19:46:44.454300 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:44.449035    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-07292b-6ddc569f9d-tjwhr" podUID="c0cdfad7-6226-43d7-9c51-5551852639ba" containerName="maas-api" containerID="cri-o://00fbc687bfa38dc6e13eb6fcb6916967456bc1823275974a557aaa14dccc84f1" gracePeriod=30
Oct 30 19:46:44.914185 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:44.914109    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-07292b-6ddc569f9d-tjwhr"
Oct 30 19:46:45.005499 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:45.004964    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c0cdfad7-6226-43d7-9c51-5551852639ba-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0cdfad7-6226-43d7-9c51-5551852639ba-openshift-service-ca\") pod \"c0cdfad7-6226-43d7-9c51-5551852639ba\" (UID: \"c0cdfad7-6226-43d7-9c51-5551852639ba\") "
Oct 30 19:46:45.005499 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:45.005041    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c0cdfad7-6226-43d7-9c51-5551852639ba-metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0cdfad7-6226-43d7-9c51-5551852639ba-metrics-certs\") pod \"c0cdfad7-6226-43d7-9c51-5551852639ba\" (UID: \"c0cdfad7-6226-43d7-9c51-5551852639ba\") "
Oct 30 19:46:45.005499 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:45.005240    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c0cdfad7-6226-43d7-9c51-5551852639ba-kube-api-access-qbjjs\" (UniqueName: \"kubernetes.io/projected/c0cdfad7-6226-43d7-9c51-5551852639ba-kube-api-access-qbjjs\") pod \"c0cdfad7-6226-43d7-9c51-5551852639ba\" (UID: \"c0cdfad7-6226-43d7-9c51-5551852639ba\") "
Oct 30 19:46:45.005499 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:45.005303    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c0cdfad7-6226-43d7-9c51-5551852639ba-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0cdfad7-6226-43d7-9c51-5551852639ba-maas-api-tls\") pod \"c0cdfad7-6226-43d7-9c51-5551852639ba\" (UID: \"c0cdfad7-6226-43d7-9c51-5551852639ba\") "
Oct 30 19:46:45.019705 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:45.019661    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0cdfad7-6226-43d7-9c51-5551852639ba-openshift-service-ca" pod "c0cdfad7-6226-43d7-9c51-5551852639ba" (UID: "c0cdfad7-6226-43d7-9c51-5551852639ba"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:46:45.028437 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:45.028390    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0cdfad7-6226-43d7-9c51-5551852639ba-maas-api-tls" pod "c0cdfad7-6226-43d7-9c51-5551852639ba" (UID: "c0cdfad7-6226-43d7-9c51-5551852639ba"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:46:45.054673 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:45.054638    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0cdfad7-6226-43d7-9c51-5551852639ba-kube-api-access-qbjjs" pod "c0cdfad7-6226-43d7-9c51-5551852639ba" (UID: "c0cdfad7-6226-43d7-9c51-5551852639ba"). InnerVolumeSpecName "kube-api-access-qbjjs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:46:45.085603 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:45.054898    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0cdfad7-6226-43d7-9c51-5551852639ba-metrics-certs" pod "c0cdfad7-6226-43d7-9c51-5551852639ba" (UID: "c0cdfad7-6226-43d7-9c51-5551852639ba"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:46:45.111193 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:45.111023    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qbjjs\" (UniqueName: \"kubernetes.io/projected/c0cdfad7-6226-43d7-9c51-5551852639ba-kube-api-access-qbjjs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:45.112313 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:45.112283    6996 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0cdfad7-6226-43d7-9c51-5551852639ba-maas-api-tls\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:45.112504 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:45.112488    6996 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0cdfad7-6226-43d7-9c51-5551852639ba-openshift-service-ca\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:45.112603 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:45.112592    6996 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0cdfad7-6226-43d7-9c51-5551852639ba-metrics-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:45.615686 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:45.615650    6996 generic.go:356] "Generic (PLEG): container finished" podID="c0cdfad7-6226-43d7-9c51-5551852639ba" containerID="00fbc687bfa38dc6e13eb6fcb6916967456bc1823275974a557aaa14dccc84f1" exitCode=0
Oct 30 19:46:45.616326 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:45.616300    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-07292b-6ddc569f9d-tjwhr" event={"ID":"c0cdfad7-6226-43d7-9c51-5551852639ba","Type":"ContainerDied","Data":"00fbc687bfa38dc6e13eb6fcb6916967456bc1823275974a557aaa14dccc84f1"}
Oct 30 19:46:45.616494 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:45.616462    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-07292b-6ddc569f9d-tjwhr" event={"ID":"c0cdfad7-6226-43d7-9c51-5551852639ba","Type":"ContainerDied","Data":"90f3f4d108fb24ab0cf19752d6e81dc20939d8e81dbb31de05e654a8f41aa8d5"}
Oct 30 19:46:45.616610 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:45.616598    6996 scope.go:122] "RemoveContainer" containerID="00fbc687bfa38dc6e13eb6fcb6916967456bc1823275974a557aaa14dccc84f1"
Oct 30 19:46:45.616917 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:45.616900    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-07292b-6ddc569f9d-tjwhr"
Oct 30 19:46:45.676435 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:45.676410    6996 scope.go:122] "RemoveContainer" containerID="00fbc687bfa38dc6e13eb6fcb6916967456bc1823275974a557aaa14dccc84f1"
Oct 30 19:46:45.676757 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:46:45.676717    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00fbc687bfa38dc6e13eb6fcb6916967456bc1823275974a557aaa14dccc84f1\": container with ID starting with 00fbc687bfa38dc6e13eb6fcb6916967456bc1823275974a557aaa14dccc84f1 not found: ID does not exist" containerID="00fbc687bfa38dc6e13eb6fcb6916967456bc1823275974a557aaa14dccc84f1"
Oct 30 19:46:45.676833 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:45.676769    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00fbc687bfa38dc6e13eb6fcb6916967456bc1823275974a557aaa14dccc84f1"} err="failed to get container status \"00fbc687bfa38dc6e13eb6fcb6916967456bc1823275974a557aaa14dccc84f1\": rpc error: code = NotFound desc = could not find container \"00fbc687bfa38dc6e13eb6fcb6916967456bc1823275974a557aaa14dccc84f1\": container with ID starting with 00fbc687bfa38dc6e13eb6fcb6916967456bc1823275974a557aaa14dccc84f1 not found: ID does not exist"
Oct 30 19:46:45.682294 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:45.682239    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-07292b-6ddc569f9d-tjwhr"]
Oct 30 19:46:45.685346 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:45.685323    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-07292b-6ddc569f9d-tjwhr"]
Oct 30 19:46:46.792061 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:46.792029    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c0cdfad7-6226-43d7-9c51-5551852639ba" path="/var/lib/kubelet/pods/c0cdfad7-6226-43d7-9c51-5551852639ba/volumes"
Oct 30 19:46:52.370571 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:52.370538    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-07292b-openshift-default-744696b8d9-pljg8"]
Oct 30 19:46:52.371001 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:52.370804    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-07292b-openshift-default-744696b8d9-pljg8" podUID="a8797217-8ec7-46db-8f6a-4a11f77164f6" containerName="istio-proxy" containerID="cri-o://be439269355e42075078e4c05234e78ba2f4f1316052b50844bd1edb7db109e3" gracePeriod=30
Oct 30 19:46:57.519941 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:57.519903    6996 patch_prober.go:29] interesting pod/e2e-api-a-07292b-openshift-default-744696b8d9-pljg8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Oct 30 19:46:57.520389 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:57.519973    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-07292b-openshift-default-744696b8d9-pljg8" podUID="a8797217-8ec7-46db-8f6a-4a11f77164f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Oct 30 19:46:57.651270 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:57.651200    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-07292b-openshift-default-744696b8d9-pljg8"
Oct 30 19:46:57.700583 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:57.700552    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a8797217-8ec7-46db-8f6a-4a11f77164f6-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8797217-8ec7-46db-8f6a-4a11f77164f6-istio-envoy\") pod \"a8797217-8ec7-46db-8f6a-4a11f77164f6\" (UID: \"a8797217-8ec7-46db-8f6a-4a11f77164f6\") "
Oct 30 19:46:57.700770 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:57.700609    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a8797217-8ec7-46db-8f6a-4a11f77164f6-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8797217-8ec7-46db-8f6a-4a11f77164f6-workload-socket\") pod \"a8797217-8ec7-46db-8f6a-4a11f77164f6\" (UID: \"a8797217-8ec7-46db-8f6a-4a11f77164f6\") "
Oct 30 19:46:57.700770 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:57.700645    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a8797217-8ec7-46db-8f6a-4a11f77164f6-istio-token\" (UniqueName: \"kubernetes.io/projected/a8797217-8ec7-46db-8f6a-4a11f77164f6-istio-token\") pod \"a8797217-8ec7-46db-8f6a-4a11f77164f6\" (UID: \"a8797217-8ec7-46db-8f6a-4a11f77164f6\") "
Oct 30 19:46:57.700886 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:57.700826    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a8797217-8ec7-46db-8f6a-4a11f77164f6-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8797217-8ec7-46db-8f6a-4a11f77164f6-istio-podinfo\") pod \"a8797217-8ec7-46db-8f6a-4a11f77164f6\" (UID: \"a8797217-8ec7-46db-8f6a-4a11f77164f6\") "
Oct 30 19:46:57.700944 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:57.700883    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a8797217-8ec7-46db-8f6a-4a11f77164f6-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8797217-8ec7-46db-8f6a-4a11f77164f6-istiod-ca-cert\") pod \"a8797217-8ec7-46db-8f6a-4a11f77164f6\" (UID: \"a8797217-8ec7-46db-8f6a-4a11f77164f6\") "
Oct 30 19:46:57.700998 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:57.700957    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a8797217-8ec7-46db-8f6a-4a11f77164f6-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8797217-8ec7-46db-8f6a-4a11f77164f6-istio-data\") pod \"a8797217-8ec7-46db-8f6a-4a11f77164f6\" (UID: \"a8797217-8ec7-46db-8f6a-4a11f77164f6\") "
Oct 30 19:46:57.702590 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:57.701207    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a8797217-8ec7-46db-8f6a-4a11f77164f6-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8797217-8ec7-46db-8f6a-4a11f77164f6-credential-socket\") pod \"a8797217-8ec7-46db-8f6a-4a11f77164f6\" (UID: \"a8797217-8ec7-46db-8f6a-4a11f77164f6\") "
Oct 30 19:46:57.702590 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:57.701366    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a8797217-8ec7-46db-8f6a-4a11f77164f6-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8797217-8ec7-46db-8f6a-4a11f77164f6-workload-certs\") pod \"a8797217-8ec7-46db-8f6a-4a11f77164f6\" (UID: \"a8797217-8ec7-46db-8f6a-4a11f77164f6\") "
Oct 30 19:46:57.702590 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:57.701516    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8797217-8ec7-46db-8f6a-4a11f77164f6-workload-socket" pod "a8797217-8ec7-46db-8f6a-4a11f77164f6" (UID: "a8797217-8ec7-46db-8f6a-4a11f77164f6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:46:57.702590 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:57.701646    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8797217-8ec7-46db-8f6a-4a11f77164f6-istio-data" pod "a8797217-8ec7-46db-8f6a-4a11f77164f6" (UID: "a8797217-8ec7-46db-8f6a-4a11f77164f6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:46:57.702590 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:57.701694    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8797217-8ec7-46db-8f6a-4a11f77164f6-istiod-ca-cert" pod "a8797217-8ec7-46db-8f6a-4a11f77164f6" (UID: "a8797217-8ec7-46db-8f6a-4a11f77164f6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:46:57.702590 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:57.701779    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a8797217-8ec7-46db-8f6a-4a11f77164f6-kube-api-access-9j9qc\" (UniqueName: \"kubernetes.io/projected/a8797217-8ec7-46db-8f6a-4a11f77164f6-kube-api-access-9j9qc\") pod \"a8797217-8ec7-46db-8f6a-4a11f77164f6\" (UID: \"a8797217-8ec7-46db-8f6a-4a11f77164f6\") "
Oct 30 19:46:57.702590 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:57.701856    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8797217-8ec7-46db-8f6a-4a11f77164f6-credential-socket" pod "a8797217-8ec7-46db-8f6a-4a11f77164f6" (UID: "a8797217-8ec7-46db-8f6a-4a11f77164f6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:46:57.702590 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:57.701939    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8797217-8ec7-46db-8f6a-4a11f77164f6-workload-certs" pod "a8797217-8ec7-46db-8f6a-4a11f77164f6" (UID: "a8797217-8ec7-46db-8f6a-4a11f77164f6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:46:57.702590 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:57.702472    6996 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8797217-8ec7-46db-8f6a-4a11f77164f6-workload-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:57.702590 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:57.702512    6996 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8797217-8ec7-46db-8f6a-4a11f77164f6-workload-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:57.702590 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:57.702527    6996 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8797217-8ec7-46db-8f6a-4a11f77164f6-istiod-ca-cert\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:57.702590 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:57.702541    6996 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8797217-8ec7-46db-8f6a-4a11f77164f6-istio-data\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:57.702590 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:57.702557    6996 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8797217-8ec7-46db-8f6a-4a11f77164f6-credential-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:57.704364 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:57.704336    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8797217-8ec7-46db-8f6a-4a11f77164f6-kube-api-access-9j9qc" pod "a8797217-8ec7-46db-8f6a-4a11f77164f6" (UID: "a8797217-8ec7-46db-8f6a-4a11f77164f6"). InnerVolumeSpecName "kube-api-access-9j9qc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:46:57.705941 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:57.705914    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8797217-8ec7-46db-8f6a-4a11f77164f6-istio-envoy" pod "a8797217-8ec7-46db-8f6a-4a11f77164f6" (UID: "a8797217-8ec7-46db-8f6a-4a11f77164f6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:46:57.706056 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:57.705957    6996 generic.go:356] "Generic (PLEG): container finished" podID="a8797217-8ec7-46db-8f6a-4a11f77164f6" containerID="be439269355e42075078e4c05234e78ba2f4f1316052b50844bd1edb7db109e3" exitCode=0
Oct 30 19:46:57.706056 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:57.705984    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a8797217-8ec7-46db-8f6a-4a11f77164f6-istio-podinfo" pod "a8797217-8ec7-46db-8f6a-4a11f77164f6" (UID: "a8797217-8ec7-46db-8f6a-4a11f77164f6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:46:57.706056 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:57.706020    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-07292b-openshift-default-744696b8d9-pljg8"
Oct 30 19:46:57.706304 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:57.706020    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-07292b-openshift-default-744696b8d9-pljg8" event={"ID":"a8797217-8ec7-46db-8f6a-4a11f77164f6","Type":"ContainerDied","Data":"be439269355e42075078e4c05234e78ba2f4f1316052b50844bd1edb7db109e3"}
Oct 30 19:46:57.706304 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:57.706139    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-07292b-openshift-default-744696b8d9-pljg8" event={"ID":"a8797217-8ec7-46db-8f6a-4a11f77164f6","Type":"ContainerDied","Data":"8c11250a6c07ec43a705c9bb032cb32c9ea249fd7099b2df563d55fc44594c71"}
Oct 30 19:46:57.706304 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:57.706163    6996 scope.go:122] "RemoveContainer" containerID="be439269355e42075078e4c05234e78ba2f4f1316052b50844bd1edb7db109e3"
Oct 30 19:46:57.707375 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:57.707357    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8797217-8ec7-46db-8f6a-4a11f77164f6-istio-token" pod "a8797217-8ec7-46db-8f6a-4a11f77164f6" (UID: "a8797217-8ec7-46db-8f6a-4a11f77164f6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:46:57.742033 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:57.742001    6996 scope.go:122] "RemoveContainer" containerID="5b83384e092dbde5ebf88716e5b3c3c1297a9079b809893e40eb2874ceb3db5b"
Oct 30 19:46:57.751198 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:57.751182    6996 scope.go:122] "RemoveContainer" containerID="be439269355e42075078e4c05234e78ba2f4f1316052b50844bd1edb7db109e3"
Oct 30 19:46:57.751579 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:46:57.751470    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be439269355e42075078e4c05234e78ba2f4f1316052b50844bd1edb7db109e3\": container with ID starting with be439269355e42075078e4c05234e78ba2f4f1316052b50844bd1edb7db109e3 not found: ID does not exist" containerID="be439269355e42075078e4c05234e78ba2f4f1316052b50844bd1edb7db109e3"
Oct 30 19:46:57.751658 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:57.751590    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be439269355e42075078e4c05234e78ba2f4f1316052b50844bd1edb7db109e3"} err="failed to get container status \"be439269355e42075078e4c05234e78ba2f4f1316052b50844bd1edb7db109e3\": rpc error: code = NotFound desc = could not find container \"be439269355e42075078e4c05234e78ba2f4f1316052b50844bd1edb7db109e3\": container with ID starting with be439269355e42075078e4c05234e78ba2f4f1316052b50844bd1edb7db109e3 not found: ID does not exist"
Oct 30 19:46:57.751658 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:57.751609    6996 scope.go:122] "RemoveContainer" containerID="5b83384e092dbde5ebf88716e5b3c3c1297a9079b809893e40eb2874ceb3db5b"
Oct 30 19:46:57.751867 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:46:57.751846    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b83384e092dbde5ebf88716e5b3c3c1297a9079b809893e40eb2874ceb3db5b\": container with ID starting with 5b83384e092dbde5ebf88716e5b3c3c1297a9079b809893e40eb2874ceb3db5b not found: ID does not exist" containerID="5b83384e092dbde5ebf88716e5b3c3c1297a9079b809893e40eb2874ceb3db5b"
Oct 30 19:46:57.751908 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:57.751875    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b83384e092dbde5ebf88716e5b3c3c1297a9079b809893e40eb2874ceb3db5b"} err="failed to get container status \"5b83384e092dbde5ebf88716e5b3c3c1297a9079b809893e40eb2874ceb3db5b\": rpc error: code = NotFound desc = could not find container \"5b83384e092dbde5ebf88716e5b3c3c1297a9079b809893e40eb2874ceb3db5b\": container with ID starting with 5b83384e092dbde5ebf88716e5b3c3c1297a9079b809893e40eb2874ceb3db5b not found: ID does not exist"
Oct 30 19:46:57.804342 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:57.804318    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9j9qc\" (UniqueName: \"kubernetes.io/projected/a8797217-8ec7-46db-8f6a-4a11f77164f6-kube-api-access-9j9qc\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:57.804342 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:57.804343    6996 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8797217-8ec7-46db-8f6a-4a11f77164f6-istio-envoy\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:57.804474 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:57.804353    6996 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8797217-8ec7-46db-8f6a-4a11f77164f6-istio-token\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:57.804474 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:57.804362    6996 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8797217-8ec7-46db-8f6a-4a11f77164f6-istio-podinfo\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:58.030508 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:58.030467    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-07292b-openshift-default-744696b8d9-pljg8"]
Oct 30 19:46:58.037039 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:58.037011    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-07292b-openshift-default-744696b8d9-pljg8"]
Oct 30 19:46:58.800602 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:46:58.800569    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a8797217-8ec7-46db-8f6a-4a11f77164f6" path="/var/lib/kubelet/pods/a8797217-8ec7-46db-8f6a-4a11f77164f6/volumes"
Oct 30 19:47:01.190259 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:47:01.190222    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 19:47:03.027651 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:03.027619    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-6ac47d-7544f97974-s4cp5"]
Oct 30 19:47:08.029471 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:08.028787    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-6ac47d-7544f97974-s4cp5" podUID="5bf6d5d9-fb0a-48e6-bbb1-cf6a161e2c6c" containerName="payload-processing" containerID="cri-o://5e516b182f6f30de0aea26f39cd5dee5165eaa69d0491e4318f0d4ddb044f8e0" gracePeriod=25
Oct 30 19:47:08.312456 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:08.312428    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-6ac47d-7544f97974-s4cp5"
Oct 30 19:47:08.365964 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:08.365934    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5bf6d5d9-fb0a-48e6-bbb1-cf6a161e2c6c-kube-api-access-zd7fz\" (UniqueName: \"kubernetes.io/projected/5bf6d5d9-fb0a-48e6-bbb1-cf6a161e2c6c-kube-api-access-zd7fz\") pod \"5bf6d5d9-fb0a-48e6-bbb1-cf6a161e2c6c\" (UID: \"5bf6d5d9-fb0a-48e6-bbb1-cf6a161e2c6c\") "
Oct 30 19:47:08.366196 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:08.366177    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5bf6d5d9-fb0a-48e6-bbb1-cf6a161e2c6c-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5bf6d5d9-fb0a-48e6-bbb1-cf6a161e2c6c-plugins-config-volume\") pod \"5bf6d5d9-fb0a-48e6-bbb1-cf6a161e2c6c\" (UID: \"5bf6d5d9-fb0a-48e6-bbb1-cf6a161e2c6c\") "
Oct 30 19:47:08.367684 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:08.367602    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5bf6d5d9-fb0a-48e6-bbb1-cf6a161e2c6c-plugins-config-volume" pod "5bf6d5d9-fb0a-48e6-bbb1-cf6a161e2c6c" (UID: "5bf6d5d9-fb0a-48e6-bbb1-cf6a161e2c6c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:47:08.369040 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:08.369011    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5bf6d5d9-fb0a-48e6-bbb1-cf6a161e2c6c-kube-api-access-zd7fz" pod "5bf6d5d9-fb0a-48e6-bbb1-cf6a161e2c6c" (UID: "5bf6d5d9-fb0a-48e6-bbb1-cf6a161e2c6c"). InnerVolumeSpecName "kube-api-access-zd7fz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:47:08.372415 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:08.372386    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 19:47:08.469093 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:08.469055    6996 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5bf6d5d9-fb0a-48e6-bbb1-cf6a161e2c6c-plugins-config-volume\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:08.469093 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:08.469092    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zd7fz\" (UniqueName: \"kubernetes.io/projected/5bf6d5d9-fb0a-48e6-bbb1-cf6a161e2c6c-kube-api-access-zd7fz\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:08.783392 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:08.783362    6996 generic.go:356] "Generic (PLEG): container finished" podID="5bf6d5d9-fb0a-48e6-bbb1-cf6a161e2c6c" containerID="5e516b182f6f30de0aea26f39cd5dee5165eaa69d0491e4318f0d4ddb044f8e0" exitCode=0
Oct 30 19:47:08.783603 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:08.783406    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-6ac47d-7544f97974-s4cp5"
Oct 30 19:47:08.783603 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:08.783447    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-6ac47d-7544f97974-s4cp5" event={"ID":"5bf6d5d9-fb0a-48e6-bbb1-cf6a161e2c6c","Type":"ContainerDied","Data":"5e516b182f6f30de0aea26f39cd5dee5165eaa69d0491e4318f0d4ddb044f8e0"}
Oct 30 19:47:08.783603 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:08.783510    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-6ac47d-7544f97974-s4cp5" event={"ID":"5bf6d5d9-fb0a-48e6-bbb1-cf6a161e2c6c","Type":"ContainerDied","Data":"4e4373024340a918aa824dae6abd3d2afa3c36ef2f3e61806055e50895d8c969"}
Oct 30 19:47:08.783603 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:08.783536    6996 scope.go:122] "RemoveContainer" containerID="5e516b182f6f30de0aea26f39cd5dee5165eaa69d0491e4318f0d4ddb044f8e0"
Oct 30 19:47:08.796094 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:08.796082    6996 scope.go:122] "RemoveContainer" containerID="5e516b182f6f30de0aea26f39cd5dee5165eaa69d0491e4318f0d4ddb044f8e0"
Oct 30 19:47:08.796343 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:47:08.796322    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e516b182f6f30de0aea26f39cd5dee5165eaa69d0491e4318f0d4ddb044f8e0\": container with ID starting with 5e516b182f6f30de0aea26f39cd5dee5165eaa69d0491e4318f0d4ddb044f8e0 not found: ID does not exist" containerID="5e516b182f6f30de0aea26f39cd5dee5165eaa69d0491e4318f0d4ddb044f8e0"
Oct 30 19:47:08.796405 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:08.796350    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e516b182f6f30de0aea26f39cd5dee5165eaa69d0491e4318f0d4ddb044f8e0"} err="failed to get container status \"5e516b182f6f30de0aea26f39cd5dee5165eaa69d0491e4318f0d4ddb044f8e0\": rpc error: code = NotFound desc = could not find container \"5e516b182f6f30de0aea26f39cd5dee5165eaa69d0491e4318f0d4ddb044f8e0\": container with ID starting with 5e516b182f6f30de0aea26f39cd5dee5165eaa69d0491e4318f0d4ddb044f8e0 not found: ID does not exist"
Oct 30 19:47:08.813905 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:08.813879    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-6ac47d-7544f97974-s4cp5"]
Oct 30 19:47:08.818728 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:08.818706    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-6ac47d-7544f97974-s4cp5"]
Oct 30 19:47:10.791907 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:10.791872    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5bf6d5d9-fb0a-48e6-bbb1-cf6a161e2c6c" path="/var/lib/kubelet/pods/5bf6d5d9-fb0a-48e6-bbb1-cf6a161e2c6c/volumes"
Oct 30 19:47:11.250551 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:11.250516    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7325-40b08f"]
Oct 30 19:47:11.251203 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:11.251182    6996 cpu_manager.go:414] "containerMap: removing container" podUID="c0cdfad7-6226-43d7-9c51-5551852639ba" containerName="maas-api"
Oct 30 19:47:11.251322 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:11.251205    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c0cdfad7-6226-43d7-9c51-5551852639ba" containerName="maas-api"
Oct 30 19:47:11.251322 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:11.251228    6996 cpu_manager.go:414] "containerMap: removing container" podUID="aa2c7806-fc8c-4c51-8108-c14d31c23be8" containerName="payload-pre-processing"
Oct 30 19:47:11.251322 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:11.251236    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aa2c7806-fc8c-4c51-8108-c14d31c23be8" containerName="payload-pre-processing"
Oct 30 19:47:11.251322 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:11.251259    6996 cpu_manager.go:414] "containerMap: removing container" podUID="a8797217-8ec7-46db-8f6a-4a11f77164f6" containerName="istio-proxy"
Oct 30 19:47:11.251322 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:11.251267    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a8797217-8ec7-46db-8f6a-4a11f77164f6" containerName="istio-proxy"
Oct 30 19:47:11.251322 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:11.251285    6996 cpu_manager.go:414] "containerMap: removing container" podUID="5bf6d5d9-fb0a-48e6-bbb1-cf6a161e2c6c" containerName="payload-processing"
Oct 30 19:47:11.251322 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:11.251291    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5bf6d5d9-fb0a-48e6-bbb1-cf6a161e2c6c" containerName="payload-processing"
Oct 30 19:47:11.251322 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:11.251315    6996 cpu_manager.go:414] "containerMap: removing container" podUID="a8797217-8ec7-46db-8f6a-4a11f77164f6" containerName="istio-proxy"
Oct 30 19:47:11.251322 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:11.251322    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a8797217-8ec7-46db-8f6a-4a11f77164f6" containerName="istio-proxy"
Oct 30 19:47:11.251797 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:11.251340    6996 cpu_manager.go:414] "containerMap: removing container" podUID="ea310863-7424-45c4-a916-8b824e270c40" containerName="payload-processing"
Oct 30 19:47:11.251797 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:11.251349    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ea310863-7424-45c4-a916-8b824e270c40" containerName="payload-processing"
Oct 30 19:47:11.251797 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:11.251516    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="a8797217-8ec7-46db-8f6a-4a11f77164f6" containerName="istio-proxy"
Oct 30 19:47:11.251797 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:11.251533    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="ea310863-7424-45c4-a916-8b824e270c40" containerName="payload-processing"
Oct 30 19:47:11.251797 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:11.251553    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="c0cdfad7-6226-43d7-9c51-5551852639ba" containerName="maas-api"
Oct 30 19:47:11.251797 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:11.251564    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="aa2c7806-fc8c-4c51-8108-c14d31c23be8" containerName="payload-pre-processing"
Oct 30 19:47:11.251797 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:11.251576    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="5bf6d5d9-fb0a-48e6-bbb1-cf6a161e2c6c" containerName="payload-processing"
Oct 30 19:47:11.256638 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:11.256618    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7325-40b08f"
Oct 30 19:47:11.262286 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:11.262262    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7325-40b08f"]
Oct 30 19:47:11.262765 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:11.262724    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-dqx9r\""
Oct 30 19:47:11.310248 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:11.310200    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkh6l\" (UniqueName: \"kubernetes.io/projected/6929f200-d5d1-4616-a9b4-48b7f02045d8-kube-api-access-dkh6l\") pod \"test-curl-7325-40b08f\" (UID: \"6929f200-d5d1-4616-a9b4-48b7f02045d8\") " pod="opendatahub/test-curl-7325-40b08f"
Oct 30 19:47:11.412089 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:11.412058    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkh6l\" (UniqueName: \"kubernetes.io/projected/6929f200-d5d1-4616-a9b4-48b7f02045d8-kube-api-access-dkh6l\") pod \"test-curl-7325-40b08f\" (UID: \"6929f200-d5d1-4616-a9b4-48b7f02045d8\") " pod="opendatahub/test-curl-7325-40b08f"
Oct 30 19:47:11.423808 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:11.423767    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkh6l\" (UniqueName: \"kubernetes.io/projected/6929f200-d5d1-4616-a9b4-48b7f02045d8-kube-api-access-dkh6l\") pod \"test-curl-7325-40b08f\" (UID: \"6929f200-d5d1-4616-a9b4-48b7f02045d8\") " pod="opendatahub/test-curl-7325-40b08f"
Oct 30 19:47:11.572708 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:11.572674    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7325-40b08f"
Oct 30 19:47:11.866720 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:11.866693    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7325-40b08f"]
Oct 30 19:47:11.872611 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:47:11.868545    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6929f200_d5d1_4616_a9b4_48b7f02045d8.slice/crio-de69178b71fd75db6c36e19de9fcf0496dec8d9e622d4e7f2abb590a89b13051 WatchSource:0}: Error finding container de69178b71fd75db6c36e19de9fcf0496dec8d9e622d4e7f2abb590a89b13051: Status 404 returned error can't find the container with id de69178b71fd75db6c36e19de9fcf0496dec8d9e622d4e7f2abb590a89b13051
Oct 30 19:47:12.812031 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:12.812004    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7325-40b08f" event={"ID":"6929f200-d5d1-4616-a9b4-48b7f02045d8","Type":"ContainerStarted","Data":"de69178b71fd75db6c36e19de9fcf0496dec8d9e622d4e7f2abb590a89b13051"}
Oct 30 19:47:13.820557 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:13.820520    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7325-40b08f" event={"ID":"6929f200-d5d1-4616-a9b4-48b7f02045d8","Type":"ContainerStarted","Data":"f5aaa5a26b6d1f6d49e01c9d1ec9fb23f7660e95da39d5c50939ee1bd6dc4b03"}
Oct 30 19:47:13.839456 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:13.839405    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7325-40b08f" podStartSLOduration=1.969706296 podStartE2EDuration="2.839392984s" totalImagesPullingTime="869.686688ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:47:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:47:13.836524535 +0000 UTC m=+909.164642686" watchObservedRunningTime="2026-10-30 19:47:13.839392984 +0000 UTC m=+909.167511139"
Oct 30 19:47:14.742262 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:14.742229    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7325-40b08f"]
Oct 30 19:47:14.747761 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:14.747734    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7325-40b08f"]
Oct 30 19:47:15.834202 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:15.834120    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7325-40b08f" podUID="6929f200-d5d1-4616-a9b4-48b7f02045d8" containerName="test-curl-7325-40b08f" containerID="cri-o://f5aaa5a26b6d1f6d49e01c9d1ec9fb23f7660e95da39d5c50939ee1bd6dc4b03" gracePeriod=2
Oct 30 19:47:17.053596 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:17.053564    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-6ac47d-67967b944d-p26th"]
Oct 30 19:47:17.054054 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:17.053761    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ac47d-67967b944d-p26th" podUID="d36e9a4f-49b3-4544-a98c-536aace46c1f" containerName="maas-api" containerID="cri-o://452e2ad3f1a1435ef84be668200c5410f6c830cc544e33a81a467ca5ed30b7f6" gracePeriod=30
Oct 30 19:47:17.112647 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:17.112613    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ac47d-67967b944d-p26th" podUID="d36e9a4f-49b3-4544-a98c-536aace46c1f" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.97:8443/health\": dial tcp 10.134.0.97:8443: connect: connection refused"
Oct 30 19:47:17.417458 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:17.417426    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ac47d-67967b944d-p26th"
Oct 30 19:47:17.499283 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:17.499254    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d36e9a4f-49b3-4544-a98c-536aace46c1f-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d36e9a4f-49b3-4544-a98c-536aace46c1f-maas-api-tls\") pod \"d36e9a4f-49b3-4544-a98c-536aace46c1f\" (UID: \"d36e9a4f-49b3-4544-a98c-536aace46c1f\") "
Oct 30 19:47:17.499450 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:17.499309    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d36e9a4f-49b3-4544-a98c-536aace46c1f-kube-api-access-njvkz\" (UniqueName: \"kubernetes.io/projected/d36e9a4f-49b3-4544-a98c-536aace46c1f-kube-api-access-njvkz\") pod \"d36e9a4f-49b3-4544-a98c-536aace46c1f\" (UID: \"d36e9a4f-49b3-4544-a98c-536aace46c1f\") "
Oct 30 19:47:17.499555 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:17.499539    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d36e9a4f-49b3-4544-a98c-536aace46c1f-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d36e9a4f-49b3-4544-a98c-536aace46c1f-openshift-service-ca\") pod \"d36e9a4f-49b3-4544-a98c-536aace46c1f\" (UID: \"d36e9a4f-49b3-4544-a98c-536aace46c1f\") "
Oct 30 19:47:17.499612 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:17.499573    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d36e9a4f-49b3-4544-a98c-536aace46c1f-metrics-certs\" (UniqueName: \"kubernetes.io/secret/d36e9a4f-49b3-4544-a98c-536aace46c1f-metrics-certs\") pod \"d36e9a4f-49b3-4544-a98c-536aace46c1f\" (UID: \"d36e9a4f-49b3-4544-a98c-536aace46c1f\") "
Oct 30 19:47:17.499885 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:17.499855    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d36e9a4f-49b3-4544-a98c-536aace46c1f-openshift-service-ca" pod "d36e9a4f-49b3-4544-a98c-536aace46c1f" (UID: "d36e9a4f-49b3-4544-a98c-536aace46c1f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:47:17.500129 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:17.500111    6996 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d36e9a4f-49b3-4544-a98c-536aace46c1f-openshift-service-ca\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:17.501558 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:17.501529    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d36e9a4f-49b3-4544-a98c-536aace46c1f-kube-api-access-njvkz" pod "d36e9a4f-49b3-4544-a98c-536aace46c1f" (UID: "d36e9a4f-49b3-4544-a98c-536aace46c1f"). InnerVolumeSpecName "kube-api-access-njvkz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:47:17.501684 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:17.501571    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d36e9a4f-49b3-4544-a98c-536aace46c1f-maas-api-tls" pod "d36e9a4f-49b3-4544-a98c-536aace46c1f" (UID: "d36e9a4f-49b3-4544-a98c-536aace46c1f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:47:17.501752 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:17.501676    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d36e9a4f-49b3-4544-a98c-536aace46c1f-metrics-certs" pod "d36e9a4f-49b3-4544-a98c-536aace46c1f" (UID: "d36e9a4f-49b3-4544-a98c-536aace46c1f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:47:17.601657 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:17.601628    6996 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d36e9a4f-49b3-4544-a98c-536aace46c1f-metrics-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:17.601657 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:17.601655    6996 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d36e9a4f-49b3-4544-a98c-536aace46c1f-maas-api-tls\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:17.601831 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:17.601665    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-njvkz\" (UniqueName: \"kubernetes.io/projected/d36e9a4f-49b3-4544-a98c-536aace46c1f-kube-api-access-njvkz\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:17.852074 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:17.852048    6996 generic.go:356] "Generic (PLEG): container finished" podID="d36e9a4f-49b3-4544-a98c-536aace46c1f" containerID="452e2ad3f1a1435ef84be668200c5410f6c830cc544e33a81a467ca5ed30b7f6" exitCode=0
Oct 30 19:47:17.852187 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:17.852092    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ac47d-67967b944d-p26th"
Oct 30 19:47:17.852187 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:17.852124    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ac47d-67967b944d-p26th" event={"ID":"d36e9a4f-49b3-4544-a98c-536aace46c1f","Type":"ContainerDied","Data":"452e2ad3f1a1435ef84be668200c5410f6c830cc544e33a81a467ca5ed30b7f6"}
Oct 30 19:47:17.852187 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:17.852177    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6ac47d-67967b944d-p26th" event={"ID":"d36e9a4f-49b3-4544-a98c-536aace46c1f","Type":"ContainerDied","Data":"624e00ab51a7f5637abab001727570d319687483ed520ca3b46d3eec23c888cb"}
Oct 30 19:47:17.852288 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:17.852201    6996 scope.go:122] "RemoveContainer" containerID="452e2ad3f1a1435ef84be668200c5410f6c830cc544e33a81a467ca5ed30b7f6"
Oct 30 19:47:17.885967 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:17.885945    6996 scope.go:122] "RemoveContainer" containerID="452e2ad3f1a1435ef84be668200c5410f6c830cc544e33a81a467ca5ed30b7f6"
Oct 30 19:47:17.886291 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:47:17.886267    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"452e2ad3f1a1435ef84be668200c5410f6c830cc544e33a81a467ca5ed30b7f6\": container with ID starting with 452e2ad3f1a1435ef84be668200c5410f6c830cc544e33a81a467ca5ed30b7f6 not found: ID does not exist" containerID="452e2ad3f1a1435ef84be668200c5410f6c830cc544e33a81a467ca5ed30b7f6"
Oct 30 19:47:17.886365 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:17.886305    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"452e2ad3f1a1435ef84be668200c5410f6c830cc544e33a81a467ca5ed30b7f6"} err="failed to get container status \"452e2ad3f1a1435ef84be668200c5410f6c830cc544e33a81a467ca5ed30b7f6\": rpc error: code = NotFound desc = could not find container \"452e2ad3f1a1435ef84be668200c5410f6c830cc544e33a81a467ca5ed30b7f6\": container with ID starting with 452e2ad3f1a1435ef84be668200c5410f6c830cc544e33a81a467ca5ed30b7f6 not found: ID does not exist"
Oct 30 19:47:17.922696 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:17.922663    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-6ac47d-67967b944d-p26th"]
Oct 30 19:47:17.928255 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:17.928228    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-6ac47d-67967b944d-p26th"]
Oct 30 19:47:17.979871 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:17.979844    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7325-40b08f"
Oct 30 19:47:18.113233 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:18.113159    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6929f200-d5d1-4616-a9b4-48b7f02045d8-kube-api-access-dkh6l\" (UniqueName: \"kubernetes.io/projected/6929f200-d5d1-4616-a9b4-48b7f02045d8-kube-api-access-dkh6l\") pod \"6929f200-d5d1-4616-a9b4-48b7f02045d8\" (UID: \"6929f200-d5d1-4616-a9b4-48b7f02045d8\") "
Oct 30 19:47:18.115170 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:18.115141    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6929f200-d5d1-4616-a9b4-48b7f02045d8-kube-api-access-dkh6l" pod "6929f200-d5d1-4616-a9b4-48b7f02045d8" (UID: "6929f200-d5d1-4616-a9b4-48b7f02045d8"). InnerVolumeSpecName "kube-api-access-dkh6l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:47:18.215120 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:18.215092    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dkh6l\" (UniqueName: \"kubernetes.io/projected/6929f200-d5d1-4616-a9b4-48b7f02045d8-kube-api-access-dkh6l\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:18.791713 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:18.791680    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6929f200-d5d1-4616-a9b4-48b7f02045d8" path="/var/lib/kubelet/pods/6929f200-d5d1-4616-a9b4-48b7f02045d8/volumes"
Oct 30 19:47:18.792115 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:18.792100    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d36e9a4f-49b3-4544-a98c-536aace46c1f" path="/var/lib/kubelet/pods/d36e9a4f-49b3-4544-a98c-536aace46c1f/volumes"
Oct 30 19:47:18.870104 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:18.870061    6996 generic.go:356] "Generic (PLEG): container finished" podID="6929f200-d5d1-4616-a9b4-48b7f02045d8" containerID="f5aaa5a26b6d1f6d49e01c9d1ec9fb23f7660e95da39d5c50939ee1bd6dc4b03" exitCode=137
Oct 30 19:47:18.870280 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:18.870118    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7325-40b08f"
Oct 30 19:47:18.870280 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:18.870158    6996 scope.go:122] "RemoveContainer" containerID="f5aaa5a26b6d1f6d49e01c9d1ec9fb23f7660e95da39d5c50939ee1bd6dc4b03"
Oct 30 19:47:18.882086 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:18.882065    6996 scope.go:122] "RemoveContainer" containerID="f5aaa5a26b6d1f6d49e01c9d1ec9fb23f7660e95da39d5c50939ee1bd6dc4b03"
Oct 30 19:47:18.882382 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:47:18.882361    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5aaa5a26b6d1f6d49e01c9d1ec9fb23f7660e95da39d5c50939ee1bd6dc4b03\": container with ID starting with f5aaa5a26b6d1f6d49e01c9d1ec9fb23f7660e95da39d5c50939ee1bd6dc4b03 not found: ID does not exist" containerID="f5aaa5a26b6d1f6d49e01c9d1ec9fb23f7660e95da39d5c50939ee1bd6dc4b03"
Oct 30 19:47:18.882463 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:18.882389    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5aaa5a26b6d1f6d49e01c9d1ec9fb23f7660e95da39d5c50939ee1bd6dc4b03"} err="failed to get container status \"f5aaa5a26b6d1f6d49e01c9d1ec9fb23f7660e95da39d5c50939ee1bd6dc4b03\": rpc error: code = NotFound desc = could not find container \"f5aaa5a26b6d1f6d49e01c9d1ec9fb23f7660e95da39d5c50939ee1bd6dc4b03\": container with ID starting with f5aaa5a26b6d1f6d49e01c9d1ec9fb23f7660e95da39d5c50939ee1bd6dc4b03 not found: ID does not exist"
Oct 30 19:47:31.152139 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:31.152100    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7"]
Oct 30 19:47:31.152932 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:31.152910    6996 cpu_manager.go:414] "containerMap: removing container" podUID="6929f200-d5d1-4616-a9b4-48b7f02045d8" containerName="test-curl-7325-40b08f"
Oct 30 19:47:31.152932 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:31.152933    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6929f200-d5d1-4616-a9b4-48b7f02045d8" containerName="test-curl-7325-40b08f"
Oct 30 19:47:31.153050 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:31.152949    6996 cpu_manager.go:414] "containerMap: removing container" podUID="d36e9a4f-49b3-4544-a98c-536aace46c1f" containerName="maas-api"
Oct 30 19:47:31.153050 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:31.152954    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d36e9a4f-49b3-4544-a98c-536aace46c1f" containerName="maas-api"
Oct 30 19:47:31.153128 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:31.153099    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="6929f200-d5d1-4616-a9b4-48b7f02045d8" containerName="test-curl-7325-40b08f"
Oct 30 19:47:31.153128 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:31.153110    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="d36e9a4f-49b3-4544-a98c-536aace46c1f" containerName="maas-api"
Oct 30 19:47:31.153128 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:31.153126    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="a8797217-8ec7-46db-8f6a-4a11f77164f6" containerName="istio-proxy"
Oct 30 19:47:31.156961 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:31.156942    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7"
Oct 30 19:47:31.169218 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:31.169194    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-501ac8-openshift-default-dockercfg-x8d6l\""
Oct 30 19:47:31.191973 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:31.191937    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7"]
Oct 30 19:47:31.211219 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:31.211185    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a61de104-8144-4107-a532-61cba1cf53e1-workload-certs\") pod \"e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7\" (UID: \"a61de104-8144-4107-a532-61cba1cf53e1\") " pod="openshift-ingress/e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7"
Oct 30 19:47:31.211368 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:31.211318    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a61de104-8144-4107-a532-61cba1cf53e1-istio-data\") pod \"e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7\" (UID: \"a61de104-8144-4107-a532-61cba1cf53e1\") " pod="openshift-ingress/e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7"
Oct 30 19:47:31.211416 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:31.211388    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a61de104-8144-4107-a532-61cba1cf53e1-credential-socket\") pod \"e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7\" (UID: \"a61de104-8144-4107-a532-61cba1cf53e1\") " pod="openshift-ingress/e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7"
Oct 30 19:47:31.211474 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:31.211442    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a61de104-8144-4107-a532-61cba1cf53e1-workload-socket\") pod \"e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7\" (UID: \"a61de104-8144-4107-a532-61cba1cf53e1\") " pod="openshift-ingress/e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7"
Oct 30 19:47:31.211565 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:31.211541    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a61de104-8144-4107-a532-61cba1cf53e1-istio-envoy\") pod \"e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7\" (UID: \"a61de104-8144-4107-a532-61cba1cf53e1\") " pod="openshift-ingress/e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7"
Oct 30 19:47:31.211766 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:31.211746    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a61de104-8144-4107-a532-61cba1cf53e1-istio-podinfo\") pod \"e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7\" (UID: \"a61de104-8144-4107-a532-61cba1cf53e1\") " pod="openshift-ingress/e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7"
Oct 30 19:47:31.211846 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:31.211804    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wglbw\" (UniqueName: \"kubernetes.io/projected/a61de104-8144-4107-a532-61cba1cf53e1-kube-api-access-wglbw\") pod \"e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7\" (UID: \"a61de104-8144-4107-a532-61cba1cf53e1\") " pod="openshift-ingress/e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7"
Oct 30 19:47:31.211909 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:31.211871    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a61de104-8144-4107-a532-61cba1cf53e1-istiod-ca-cert\") pod \"e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7\" (UID: \"a61de104-8144-4107-a532-61cba1cf53e1\") " pod="openshift-ingress/e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7"
Oct 30 19:47:31.211963 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:31.211931    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a61de104-8144-4107-a532-61cba1cf53e1-istio-token\") pod \"e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7\" (UID: \"a61de104-8144-4107-a532-61cba1cf53e1\") " pod="openshift-ingress/e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7"
Oct 30 19:47:31.217748 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:47:31.217719    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 19:47:31.365503 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:31.355811    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a61de104-8144-4107-a532-61cba1cf53e1-istiod-ca-cert\") pod \"e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7\" (UID: \"a61de104-8144-4107-a532-61cba1cf53e1\") " pod="openshift-ingress/e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7"
Oct 30 19:47:31.365503 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:31.355876    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a61de104-8144-4107-a532-61cba1cf53e1-istio-token\") pod \"e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7\" (UID: \"a61de104-8144-4107-a532-61cba1cf53e1\") " pod="openshift-ingress/e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7"
Oct 30 19:47:31.365503 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:31.355952    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a61de104-8144-4107-a532-61cba1cf53e1-workload-certs\") pod \"e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7\" (UID: \"a61de104-8144-4107-a532-61cba1cf53e1\") " pod="openshift-ingress/e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7"
Oct 30 19:47:31.365503 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:31.356007    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a61de104-8144-4107-a532-61cba1cf53e1-istio-data\") pod \"e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7\" (UID: \"a61de104-8144-4107-a532-61cba1cf53e1\") " pod="openshift-ingress/e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7"
Oct 30 19:47:31.365503 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:31.357130    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a61de104-8144-4107-a532-61cba1cf53e1-istiod-ca-cert\") pod \"e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7\" (UID: \"a61de104-8144-4107-a532-61cba1cf53e1\") " pod="openshift-ingress/e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7"
Oct 30 19:47:31.365503 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:31.357420    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a61de104-8144-4107-a532-61cba1cf53e1-workload-certs\") pod \"e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7\" (UID: \"a61de104-8144-4107-a532-61cba1cf53e1\") " pod="openshift-ingress/e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7"
Oct 30 19:47:31.365503 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:31.358786    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a61de104-8144-4107-a532-61cba1cf53e1-istio-data\") pod \"e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7\" (UID: \"a61de104-8144-4107-a532-61cba1cf53e1\") " pod="openshift-ingress/e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7"
Oct 30 19:47:31.365503 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:31.362177    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a61de104-8144-4107-a532-61cba1cf53e1-credential-socket\") pod \"e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7\" (UID: \"a61de104-8144-4107-a532-61cba1cf53e1\") " pod="openshift-ingress/e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7"
Oct 30 19:47:31.365503 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:31.362242    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a61de104-8144-4107-a532-61cba1cf53e1-workload-socket\") pod \"e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7\" (UID: \"a61de104-8144-4107-a532-61cba1cf53e1\") " pod="openshift-ingress/e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7"
Oct 30 19:47:31.365503 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:31.362275    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a61de104-8144-4107-a532-61cba1cf53e1-istio-envoy\") pod \"e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7\" (UID: \"a61de104-8144-4107-a532-61cba1cf53e1\") " pod="openshift-ingress/e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7"
Oct 30 19:47:31.365503 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:31.362496    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a61de104-8144-4107-a532-61cba1cf53e1-istio-podinfo\") pod \"e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7\" (UID: \"a61de104-8144-4107-a532-61cba1cf53e1\") " pod="openshift-ingress/e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7"
Oct 30 19:47:31.365503 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:31.362558    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wglbw\" (UniqueName: \"kubernetes.io/projected/a61de104-8144-4107-a532-61cba1cf53e1-kube-api-access-wglbw\") pod \"e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7\" (UID: \"a61de104-8144-4107-a532-61cba1cf53e1\") " pod="openshift-ingress/e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7"
Oct 30 19:47:31.382503 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:31.372116    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a61de104-8144-4107-a532-61cba1cf53e1-credential-socket\") pod \"e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7\" (UID: \"a61de104-8144-4107-a532-61cba1cf53e1\") " pod="openshift-ingress/e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7"
Oct 30 19:47:31.382503 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:31.372840    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a61de104-8144-4107-a532-61cba1cf53e1-workload-socket\") pod \"e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7\" (UID: \"a61de104-8144-4107-a532-61cba1cf53e1\") " pod="openshift-ingress/e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7"
Oct 30 19:47:31.386315 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:31.386277    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a61de104-8144-4107-a532-61cba1cf53e1-istio-envoy\") pod \"e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7\" (UID: \"a61de104-8144-4107-a532-61cba1cf53e1\") " pod="openshift-ingress/e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7"
Oct 30 19:47:31.388178 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:31.388124    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a61de104-8144-4107-a532-61cba1cf53e1-istio-token\") pod \"e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7\" (UID: \"a61de104-8144-4107-a532-61cba1cf53e1\") " pod="openshift-ingress/e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7"
Oct 30 19:47:31.388710 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:31.388676    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wglbw\" (UniqueName: \"kubernetes.io/projected/a61de104-8144-4107-a532-61cba1cf53e1-kube-api-access-wglbw\") pod \"e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7\" (UID: \"a61de104-8144-4107-a532-61cba1cf53e1\") " pod="openshift-ingress/e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7"
Oct 30 19:47:31.390190 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:31.390167    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a61de104-8144-4107-a532-61cba1cf53e1-istio-podinfo\") pod \"e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7\" (UID: \"a61de104-8144-4107-a532-61cba1cf53e1\") " pod="openshift-ingress/e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7"
Oct 30 19:47:31.473172 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:31.473086    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7"
Oct 30 19:47:31.764877 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:31.764843    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7"]
Oct 30 19:47:31.766226 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:47:31.766202    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda61de104_8144_4107_a532_61cba1cf53e1.slice/crio-ff4895635fe9cc4306dff8af408ef9713af6630385f698d461239d44f7b2485d WatchSource:0}: Error finding container ff4895635fe9cc4306dff8af408ef9713af6630385f698d461239d44f7b2485d: Status 404 returned error can't find the container with id ff4895635fe9cc4306dff8af408ef9713af6630385f698d461239d44f7b2485d
Oct 30 19:47:31.768878 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:31.768839    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:47:31.768996 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:31.768911    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:47:31.768996 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:31.768940    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:47:32.012584 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:32.012546    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7" event={"ID":"a61de104-8144-4107-a532-61cba1cf53e1","Type":"ContainerStarted","Data":"b1a3f3d7b4883cad7a223d26a2d85141b389c8b54535927589932316607b26ec"}
Oct 30 19:47:32.012759 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:32.012616    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7" event={"ID":"a61de104-8144-4107-a532-61cba1cf53e1","Type":"ContainerStarted","Data":"ff4895635fe9cc4306dff8af408ef9713af6630385f698d461239d44f7b2485d"}
Oct 30 19:47:32.034932 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:32.034821    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7" podStartSLOduration=1.03480275 podStartE2EDuration="1.03480275s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:47:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:47:32.034368507 +0000 UTC m=+927.362486662" watchObservedRunningTime="2026-10-30 19:47:32.03480275 +0000 UTC m=+927.362920905"
Oct 30 19:47:32.473937 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:32.473902    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7"
Oct 30 19:47:32.479415 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:32.479383    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7"
Oct 30 19:47:33.019845 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:33.019814    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7"
Oct 30 19:47:33.020799 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:33.020776    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7"
Oct 30 19:47:34.805159 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:34.805126    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 19:47:35.691311 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:35.691259    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-501ac8-768b76884b-8wrfj"]
Oct 30 19:47:35.696779 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:35.696753    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-501ac8-dd4dc7b8c-bnggs"]
Oct 30 19:47:35.696925 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:35.696864    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-501ac8-768b76884b-8wrfj"
Oct 30 19:47:35.700873 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:35.700848    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-501ac8-dd4dc7b8c-bnggs"
Oct 30 19:47:35.704311 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:35.704288    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-501ac8\""
Oct 30 19:47:35.704440 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:35.704341    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-501ac8-dockercfg-tcmm5\""
Oct 30 19:47:35.713694 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:35.712636    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-501ac8-768b76884b-8wrfj"]
Oct 30 19:47:35.714414 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:35.714387    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-501ac8-dd4dc7b8c-bnggs"]
Oct 30 19:47:35.836504 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:35.835509    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af9d2807-c5e6-44a5-85d3-3ec0a5577602-plugins-config-volume\") pod \"payload-processing-e2e-api-a-501ac8-768b76884b-8wrfj\" (UID: \"af9d2807-c5e6-44a5-85d3-3ec0a5577602\") " pod="openshift-ingress/payload-processing-e2e-api-a-501ac8-768b76884b-8wrfj"
Oct 30 19:47:35.836504 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:35.835609    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gw2s2\" (UniqueName: \"kubernetes.io/projected/af9d2807-c5e6-44a5-85d3-3ec0a5577602-kube-api-access-gw2s2\") pod \"payload-processing-e2e-api-a-501ac8-768b76884b-8wrfj\" (UID: \"af9d2807-c5e6-44a5-85d3-3ec0a5577602\") " pod="openshift-ingress/payload-processing-e2e-api-a-501ac8-768b76884b-8wrfj"
Oct 30 19:47:35.836504 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:35.835732    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5w542\" (UniqueName: \"kubernetes.io/projected/26ef3405-8610-49bd-a79b-52279ca775b1-kube-api-access-5w542\") pod \"payload-pre-processing-e2e-api-a-501ac8-dd4dc7b8c-bnggs\" (UID: \"26ef3405-8610-49bd-a79b-52279ca775b1\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-501ac8-dd4dc7b8c-bnggs"
Oct 30 19:47:35.836504 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:35.835809    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26ef3405-8610-49bd-a79b-52279ca775b1-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-501ac8-dd4dc7b8c-bnggs\" (UID: \"26ef3405-8610-49bd-a79b-52279ca775b1\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-501ac8-dd4dc7b8c-bnggs"
Oct 30 19:47:35.937526 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:35.937474    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26ef3405-8610-49bd-a79b-52279ca775b1-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-501ac8-dd4dc7b8c-bnggs\" (UID: \"26ef3405-8610-49bd-a79b-52279ca775b1\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-501ac8-dd4dc7b8c-bnggs"
Oct 30 19:47:35.937741 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:35.937724    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af9d2807-c5e6-44a5-85d3-3ec0a5577602-plugins-config-volume\") pod \"payload-processing-e2e-api-a-501ac8-768b76884b-8wrfj\" (UID: \"af9d2807-c5e6-44a5-85d3-3ec0a5577602\") " pod="openshift-ingress/payload-processing-e2e-api-a-501ac8-768b76884b-8wrfj"
Oct 30 19:47:35.937859 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:35.937838    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gw2s2\" (UniqueName: \"kubernetes.io/projected/af9d2807-c5e6-44a5-85d3-3ec0a5577602-kube-api-access-gw2s2\") pod \"payload-processing-e2e-api-a-501ac8-768b76884b-8wrfj\" (UID: \"af9d2807-c5e6-44a5-85d3-3ec0a5577602\") " pod="openshift-ingress/payload-processing-e2e-api-a-501ac8-768b76884b-8wrfj"
Oct 30 19:47:35.938101 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:35.938071    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5w542\" (UniqueName: \"kubernetes.io/projected/26ef3405-8610-49bd-a79b-52279ca775b1-kube-api-access-5w542\") pod \"payload-pre-processing-e2e-api-a-501ac8-dd4dc7b8c-bnggs\" (UID: \"26ef3405-8610-49bd-a79b-52279ca775b1\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-501ac8-dd4dc7b8c-bnggs"
Oct 30 19:47:35.938387 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:35.938366    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26ef3405-8610-49bd-a79b-52279ca775b1-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-501ac8-dd4dc7b8c-bnggs\" (UID: \"26ef3405-8610-49bd-a79b-52279ca775b1\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-501ac8-dd4dc7b8c-bnggs"
Oct 30 19:47:35.939323 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:35.939293    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af9d2807-c5e6-44a5-85d3-3ec0a5577602-plugins-config-volume\") pod \"payload-processing-e2e-api-a-501ac8-768b76884b-8wrfj\" (UID: \"af9d2807-c5e6-44a5-85d3-3ec0a5577602\") " pod="openshift-ingress/payload-processing-e2e-api-a-501ac8-768b76884b-8wrfj"
Oct 30 19:47:35.950548 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:35.950430    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gw2s2\" (UniqueName: \"kubernetes.io/projected/af9d2807-c5e6-44a5-85d3-3ec0a5577602-kube-api-access-gw2s2\") pod \"payload-processing-e2e-api-a-501ac8-768b76884b-8wrfj\" (UID: \"af9d2807-c5e6-44a5-85d3-3ec0a5577602\") " pod="openshift-ingress/payload-processing-e2e-api-a-501ac8-768b76884b-8wrfj"
Oct 30 19:47:35.950873 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:35.950840    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5w542\" (UniqueName: \"kubernetes.io/projected/26ef3405-8610-49bd-a79b-52279ca775b1-kube-api-access-5w542\") pod \"payload-pre-processing-e2e-api-a-501ac8-dd4dc7b8c-bnggs\" (UID: \"26ef3405-8610-49bd-a79b-52279ca775b1\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-501ac8-dd4dc7b8c-bnggs"
Oct 30 19:47:36.019063 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:36.018238    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-501ac8-768b76884b-8wrfj"
Oct 30 19:47:36.026534 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:36.026002    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-501ac8-dd4dc7b8c-bnggs"
Oct 30 19:47:36.329806 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:36.329774    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-501ac8-768b76884b-8wrfj"]
Oct 30 19:47:36.335003 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:47:36.334967    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf9d2807_c5e6_44a5_85d3_3ec0a5577602.slice/crio-30c5c134e4f863f6c4ecb41ad67f3683f27dcc6f8ec878a5622f61bb17f54ec5 WatchSource:0}: Error finding container 30c5c134e4f863f6c4ecb41ad67f3683f27dcc6f8ec878a5622f61bb17f54ec5: Status 404 returned error can't find the container with id 30c5c134e4f863f6c4ecb41ad67f3683f27dcc6f8ec878a5622f61bb17f54ec5
Oct 30 19:47:36.349185 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:36.349161    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-501ac8-dd4dc7b8c-bnggs"]
Oct 30 19:47:36.350883 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:47:36.350830    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26ef3405_8610_49bd_a79b_52279ca775b1.slice/crio-4c66df7b7d563a68ac03ca4729084d466e27a63c48a75f56253eeaa6afbb793b WatchSource:0}: Error finding container 4c66df7b7d563a68ac03ca4729084d466e27a63c48a75f56253eeaa6afbb793b: Status 404 returned error can't find the container with id 4c66df7b7d563a68ac03ca4729084d466e27a63c48a75f56253eeaa6afbb793b
Oct 30 19:47:37.060887 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:37.060849    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-501ac8-768b76884b-8wrfj" event={"ID":"af9d2807-c5e6-44a5-85d3-3ec0a5577602","Type":"ContainerStarted","Data":"7d29cc1d8c6cbbd43b437fa4551dfbdebdff35be33492781cbe590925e5d6979"}
Oct 30 19:47:37.061379 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:37.060909    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-501ac8-768b76884b-8wrfj" event={"ID":"af9d2807-c5e6-44a5-85d3-3ec0a5577602","Type":"ContainerStarted","Data":"30c5c134e4f863f6c4ecb41ad67f3683f27dcc6f8ec878a5622f61bb17f54ec5"}
Oct 30 19:47:37.061379 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:37.061036    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-501ac8-768b76884b-8wrfj"
Oct 30 19:47:37.062669 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:37.062644    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-501ac8-dd4dc7b8c-bnggs" event={"ID":"26ef3405-8610-49bd-a79b-52279ca775b1","Type":"ContainerStarted","Data":"5d85770eb2346e590a05d3e0a7946e7a596e8c7ee944ed795e98f01f9a219d8f"}
Oct 30 19:47:37.062813 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:37.062690    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-501ac8-dd4dc7b8c-bnggs" event={"ID":"26ef3405-8610-49bd-a79b-52279ca775b1","Type":"ContainerStarted","Data":"4c66df7b7d563a68ac03ca4729084d466e27a63c48a75f56253eeaa6afbb793b"}
Oct 30 19:47:37.062813 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:37.062716    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-501ac8-dd4dc7b8c-bnggs"
Oct 30 19:47:37.089967 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:37.089923    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-501ac8-768b76884b-8wrfj" podStartSLOduration=2.089910376 podStartE2EDuration="2.089910376s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:47:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:47:37.086026717 +0000 UTC m=+932.414144870" watchObservedRunningTime="2026-10-30 19:47:37.089910376 +0000 UTC m=+932.418028527"
Oct 30 19:47:37.107491 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:37.107441    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-501ac8-dd4dc7b8c-bnggs" podStartSLOduration=2.107426999 podStartE2EDuration="2.107426999s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:47:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:47:37.103946082 +0000 UTC m=+932.432064237" watchObservedRunningTime="2026-10-30 19:47:37.107426999 +0000 UTC m=+932.435545207"
Oct 30 19:47:40.133136 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:40.131077    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-501ac8-5c4f9d85bb-pvmxr"]
Oct 30 19:47:40.136739 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:40.136711    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-501ac8-5c4f9d85bb-pvmxr"
Oct 30 19:47:40.140266 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:40.140238    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-501ac8-serving-cert\""
Oct 30 19:47:40.150963 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:40.150827    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-501ac8-5c4f9d85bb-pvmxr"]
Oct 30 19:47:40.161223 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:40.161190    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce28e93b-5db7-448f-8c7b-445428b81a7d-maas-api-tls\") pod \"maas-api-e2e-api-a-501ac8-5c4f9d85bb-pvmxr\" (UID: \"ce28e93b-5db7-448f-8c7b-445428b81a7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-501ac8-5c4f9d85bb-pvmxr"
Oct 30 19:47:40.161223 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:40.161223    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce28e93b-5db7-448f-8c7b-445428b81a7d-openshift-service-ca\") pod \"maas-api-e2e-api-a-501ac8-5c4f9d85bb-pvmxr\" (UID: \"ce28e93b-5db7-448f-8c7b-445428b81a7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-501ac8-5c4f9d85bb-pvmxr"
Oct 30 19:47:40.161455 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:40.161282    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9z989\" (UniqueName: \"kubernetes.io/projected/ce28e93b-5db7-448f-8c7b-445428b81a7d-kube-api-access-9z989\") pod \"maas-api-e2e-api-a-501ac8-5c4f9d85bb-pvmxr\" (UID: \"ce28e93b-5db7-448f-8c7b-445428b81a7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-501ac8-5c4f9d85bb-pvmxr"
Oct 30 19:47:40.161455 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:40.161401    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce28e93b-5db7-448f-8c7b-445428b81a7d-metrics-certs\") pod \"maas-api-e2e-api-a-501ac8-5c4f9d85bb-pvmxr\" (UID: \"ce28e93b-5db7-448f-8c7b-445428b81a7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-501ac8-5c4f9d85bb-pvmxr"
Oct 30 19:47:40.278518 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:40.278100    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce28e93b-5db7-448f-8c7b-445428b81a7d-metrics-certs\") pod \"maas-api-e2e-api-a-501ac8-5c4f9d85bb-pvmxr\" (UID: \"ce28e93b-5db7-448f-8c7b-445428b81a7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-501ac8-5c4f9d85bb-pvmxr"
Oct 30 19:47:40.278518 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:40.278218    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce28e93b-5db7-448f-8c7b-445428b81a7d-maas-api-tls\") pod \"maas-api-e2e-api-a-501ac8-5c4f9d85bb-pvmxr\" (UID: \"ce28e93b-5db7-448f-8c7b-445428b81a7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-501ac8-5c4f9d85bb-pvmxr"
Oct 30 19:47:40.278518 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:40.278243    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce28e93b-5db7-448f-8c7b-445428b81a7d-openshift-service-ca\") pod \"maas-api-e2e-api-a-501ac8-5c4f9d85bb-pvmxr\" (UID: \"ce28e93b-5db7-448f-8c7b-445428b81a7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-501ac8-5c4f9d85bb-pvmxr"
Oct 30 19:47:40.278518 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:40.278356    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9z989\" (UniqueName: \"kubernetes.io/projected/ce28e93b-5db7-448f-8c7b-445428b81a7d-kube-api-access-9z989\") pod \"maas-api-e2e-api-a-501ac8-5c4f9d85bb-pvmxr\" (UID: \"ce28e93b-5db7-448f-8c7b-445428b81a7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-501ac8-5c4f9d85bb-pvmxr"
Oct 30 19:47:40.284833 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:40.284804    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce28e93b-5db7-448f-8c7b-445428b81a7d-metrics-certs\") pod \"maas-api-e2e-api-a-501ac8-5c4f9d85bb-pvmxr\" (UID: \"ce28e93b-5db7-448f-8c7b-445428b81a7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-501ac8-5c4f9d85bb-pvmxr"
Oct 30 19:47:40.285020 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:47:40.284926    6996 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-501ac8-serving-cert: secret "maas-api-e2e-api-a-501ac8-serving-cert" not found
Oct 30 19:47:40.285020 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:47:40.284986    6996 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce28e93b-5db7-448f-8c7b-445428b81a7d-maas-api-tls podName:ce28e93b-5db7-448f-8c7b-445428b81a7d nodeName:}" failed. No retries permitted until 2026-10-30 19:47:40.784968338 +0000 UTC m=+936.113086469 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ce28e93b-5db7-448f-8c7b-445428b81a7d-maas-api-tls") pod "maas-api-e2e-api-a-501ac8-5c4f9d85bb-pvmxr" (UID: "ce28e93b-5db7-448f-8c7b-445428b81a7d") : secret "maas-api-e2e-api-a-501ac8-serving-cert" not found
Oct 30 19:47:40.285759 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:40.285732    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce28e93b-5db7-448f-8c7b-445428b81a7d-openshift-service-ca\") pod \"maas-api-e2e-api-a-501ac8-5c4f9d85bb-pvmxr\" (UID: \"ce28e93b-5db7-448f-8c7b-445428b81a7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-501ac8-5c4f9d85bb-pvmxr"
Oct 30 19:47:40.296254 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:40.296221    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9z989\" (UniqueName: \"kubernetes.io/projected/ce28e93b-5db7-448f-8c7b-445428b81a7d-kube-api-access-9z989\") pod \"maas-api-e2e-api-a-501ac8-5c4f9d85bb-pvmxr\" (UID: \"ce28e93b-5db7-448f-8c7b-445428b81a7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-501ac8-5c4f9d85bb-pvmxr"
Oct 30 19:47:40.785228 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:40.785187    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce28e93b-5db7-448f-8c7b-445428b81a7d-maas-api-tls\") pod \"maas-api-e2e-api-a-501ac8-5c4f9d85bb-pvmxr\" (UID: \"ce28e93b-5db7-448f-8c7b-445428b81a7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-501ac8-5c4f9d85bb-pvmxr"
Oct 30 19:47:40.789210 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:40.789175    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce28e93b-5db7-448f-8c7b-445428b81a7d-maas-api-tls\") pod \"maas-api-e2e-api-a-501ac8-5c4f9d85bb-pvmxr\" (UID: \"ce28e93b-5db7-448f-8c7b-445428b81a7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-501ac8-5c4f9d85bb-pvmxr"
Oct 30 19:47:41.063675 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:41.063601    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-501ac8-5c4f9d85bb-pvmxr"
Oct 30 19:47:41.386950 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:41.386920    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-501ac8-5c4f9d85bb-pvmxr"]
Oct 30 19:47:41.387435 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:47:41.387139    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce28e93b_5db7_448f_8c7b_445428b81a7d.slice/crio-72ca4c96f693c72d33ba85d6d6ce208be2fa1af4612b950ec96205a658154ae6 WatchSource:0}: Error finding container 72ca4c96f693c72d33ba85d6d6ce208be2fa1af4612b950ec96205a658154ae6: Status 404 returned error can't find the container with id 72ca4c96f693c72d33ba85d6d6ce208be2fa1af4612b950ec96205a658154ae6
Oct 30 19:47:42.117235 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:42.117086    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-501ac8-5c4f9d85bb-pvmxr" event={"ID":"ce28e93b-5db7-448f-8c7b-445428b81a7d","Type":"ContainerStarted","Data":"825ed48e3209dd1468a29559b4ad593ffc42f3a76f3ad96053d4773915e2a786"}
Oct 30 19:47:42.117235 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:42.117154    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-501ac8-5c4f9d85bb-pvmxr" event={"ID":"ce28e93b-5db7-448f-8c7b-445428b81a7d","Type":"ContainerStarted","Data":"72ca4c96f693c72d33ba85d6d6ce208be2fa1af4612b950ec96205a658154ae6"}
Oct 30 19:47:42.117235 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:42.117181    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-501ac8-5c4f9d85bb-pvmxr"
Oct 30 19:47:42.143809 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:42.143756    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-501ac8-5c4f9d85bb-pvmxr" podStartSLOduration=1.599261973 podStartE2EDuration="2.143741847s" totalImagesPullingTime="544.479874ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:47:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:47:42.140768222 +0000 UTC m=+937.468886377" watchObservedRunningTime="2026-10-30 19:47:42.143741847 +0000 UTC m=+937.471860019"
Oct 30 19:47:45.370501 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:45.366876    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4"]
Oct 30 19:47:45.381552 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:45.374316    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4"
Oct 30 19:47:45.410794 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:45.410763    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4"]
Oct 30 19:47:45.415096 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:45.415071    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l"]
Oct 30 19:47:45.429225 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:45.429103    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l"]
Oct 30 19:47:45.429733 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:45.429395    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l"
Oct 30 19:47:45.530047 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:45.530010    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e7778a88-c2f0-40f6-987a-8749b0cf3378-istiod-ca-cert\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4\" (UID: \"e7778a88-c2f0-40f6-987a-8749b0cf3378\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4"
Oct 30 19:47:45.530239 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:45.530083    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grz7n\" (UniqueName: \"kubernetes.io/projected/e7778a88-c2f0-40f6-987a-8749b0cf3378-kube-api-access-grz7n\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4\" (UID: \"e7778a88-c2f0-40f6-987a-8749b0cf3378\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4"
Oct 30 19:47:45.530239 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:45.530152    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e7778a88-c2f0-40f6-987a-8749b0cf3378-istio-envoy\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4\" (UID: \"e7778a88-c2f0-40f6-987a-8749b0cf3378\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4"
Oct 30 19:47:45.530239 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:45.530228    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9584be0d-abe4-4783-8849-f4b79f3094db-workload-socket\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l\" (UID: \"9584be0d-abe4-4783-8849-f4b79f3094db\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l"
Oct 30 19:47:45.530384 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:45.530254    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9584be0d-abe4-4783-8849-f4b79f3094db-istio-token\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l\" (UID: \"9584be0d-abe4-4783-8849-f4b79f3094db\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l"
Oct 30 19:47:45.530384 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:45.530278    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9584be0d-abe4-4783-8849-f4b79f3094db-istio-envoy\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l\" (UID: \"9584be0d-abe4-4783-8849-f4b79f3094db\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l"
Oct 30 19:47:45.530384 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:45.530347    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9584be0d-abe4-4783-8849-f4b79f3094db-istio-podinfo\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l\" (UID: \"9584be0d-abe4-4783-8849-f4b79f3094db\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l"
Oct 30 19:47:45.530500 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:45.530393    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9584be0d-abe4-4783-8849-f4b79f3094db-istio-data\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l\" (UID: \"9584be0d-abe4-4783-8849-f4b79f3094db\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l"
Oct 30 19:47:45.530500 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:45.530430    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e7778a88-c2f0-40f6-987a-8749b0cf3378-istio-data\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4\" (UID: \"e7778a88-c2f0-40f6-987a-8749b0cf3378\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4"
Oct 30 19:47:45.530500 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:45.530465    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e7778a88-c2f0-40f6-987a-8749b0cf3378-istio-podinfo\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4\" (UID: \"e7778a88-c2f0-40f6-987a-8749b0cf3378\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4"
Oct 30 19:47:45.530601 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:45.530523    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9584be0d-abe4-4783-8849-f4b79f3094db-workload-certs\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l\" (UID: \"9584be0d-abe4-4783-8849-f4b79f3094db\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l"
Oct 30 19:47:45.530601 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:45.530551    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9584be0d-abe4-4783-8849-f4b79f3094db-istiod-ca-cert\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l\" (UID: \"9584be0d-abe4-4783-8849-f4b79f3094db\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l"
Oct 30 19:47:45.530601 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:45.530570    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7778a88-c2f0-40f6-987a-8749b0cf3378-istio-token\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4\" (UID: \"e7778a88-c2f0-40f6-987a-8749b0cf3378\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4"
Oct 30 19:47:45.530693 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:45.530600    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7778a88-c2f0-40f6-987a-8749b0cf3378-credential-socket\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4\" (UID: \"e7778a88-c2f0-40f6-987a-8749b0cf3378\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4"
Oct 30 19:47:45.530693 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:45.530653    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9584be0d-abe4-4783-8849-f4b79f3094db-credential-socket\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l\" (UID: \"9584be0d-abe4-4783-8849-f4b79f3094db\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l"
Oct 30 19:47:45.530780 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:45.530753    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7778a88-c2f0-40f6-987a-8749b0cf3378-workload-socket\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4\" (UID: \"e7778a88-c2f0-40f6-987a-8749b0cf3378\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4"
Oct 30 19:47:45.530871 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:45.530852    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8jsh\" (UniqueName: \"kubernetes.io/projected/9584be0d-abe4-4783-8849-f4b79f3094db-kube-api-access-k8jsh\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l\" (UID: \"9584be0d-abe4-4783-8849-f4b79f3094db\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l"
Oct 30 19:47:45.530956 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:45.530916    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7778a88-c2f0-40f6-987a-8749b0cf3378-workload-certs\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4\" (UID: \"e7778a88-c2f0-40f6-987a-8749b0cf3378\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4"
Oct 30 19:47:45.632315 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:45.632227    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e7778a88-c2f0-40f6-987a-8749b0cf3378-istio-envoy\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4\" (UID: \"e7778a88-c2f0-40f6-987a-8749b0cf3378\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4"
Oct 30 19:47:45.632474 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:45.632343    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9584be0d-abe4-4783-8849-f4b79f3094db-workload-socket\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l\" (UID: \"9584be0d-abe4-4783-8849-f4b79f3094db\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l"
Oct 30 19:47:45.632474 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:45.632398    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9584be0d-abe4-4783-8849-f4b79f3094db-istio-token\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l\" (UID: \"9584be0d-abe4-4783-8849-f4b79f3094db\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l"
Oct 30 19:47:45.632474 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:45.632436    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9584be0d-abe4-4783-8849-f4b79f3094db-istio-envoy\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l\" (UID: \"9584be0d-abe4-4783-8849-f4b79f3094db\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l"
Oct 30 19:47:45.632625 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:45.632520    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9584be0d-abe4-4783-8849-f4b79f3094db-istio-podinfo\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l\" (UID: \"9584be0d-abe4-4783-8849-f4b79f3094db\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l"
Oct 30 19:47:45.632849 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:45.632819    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9584be0d-abe4-4783-8849-f4b79f3094db-workload-socket\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l\" (UID: \"9584be0d-abe4-4783-8849-f4b79f3094db\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l"
Oct 30 19:47:45.633157 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:45.632548    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9584be0d-abe4-4783-8849-f4b79f3094db-istio-data\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l\" (UID: \"9584be0d-abe4-4783-8849-f4b79f3094db\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l"
Oct 30 19:47:45.633266 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:45.633157    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9584be0d-abe4-4783-8849-f4b79f3094db-istio-data\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l\" (UID: \"9584be0d-abe4-4783-8849-f4b79f3094db\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l"
Oct 30 19:47:45.633266 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:45.633197    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e7778a88-c2f0-40f6-987a-8749b0cf3378-istio-data\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4\" (UID: \"e7778a88-c2f0-40f6-987a-8749b0cf3378\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4"
Oct 30 19:47:45.633266 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:45.633249    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e7778a88-c2f0-40f6-987a-8749b0cf3378-istio-podinfo\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4\" (UID: \"e7778a88-c2f0-40f6-987a-8749b0cf3378\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4"
Oct 30 19:47:45.633443 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:45.633281    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9584be0d-abe4-4783-8849-f4b79f3094db-workload-certs\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l\" (UID: \"9584be0d-abe4-4783-8849-f4b79f3094db\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l"
Oct 30 19:47:45.633443 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:45.633326    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9584be0d-abe4-4783-8849-f4b79f3094db-istiod-ca-cert\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l\" (UID: \"9584be0d-abe4-4783-8849-f4b79f3094db\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l"
Oct 30 19:47:45.633443 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:45.633417    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e7778a88-c2f0-40f6-987a-8749b0cf3378-istio-data\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4\" (UID: \"e7778a88-c2f0-40f6-987a-8749b0cf3378\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4"
Oct 30 19:47:45.633628 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:45.633456    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7778a88-c2f0-40f6-987a-8749b0cf3378-istio-token\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4\" (UID: \"e7778a88-c2f0-40f6-987a-8749b0cf3378\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4"
Oct 30 19:47:45.633628 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:45.633515    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7778a88-c2f0-40f6-987a-8749b0cf3378-credential-socket\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4\" (UID: \"e7778a88-c2f0-40f6-987a-8749b0cf3378\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4"
Oct 30 19:47:45.633628 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:45.633601    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9584be0d-abe4-4783-8849-f4b79f3094db-credential-socket\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l\" (UID: \"9584be0d-abe4-4783-8849-f4b79f3094db\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l"
Oct 30 19:47:45.633628 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:45.633607    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9584be0d-abe4-4783-8849-f4b79f3094db-workload-certs\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l\" (UID: \"9584be0d-abe4-4783-8849-f4b79f3094db\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l"
Oct 30 19:47:45.633843 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:45.633708    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7778a88-c2f0-40f6-987a-8749b0cf3378-workload-socket\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4\" (UID: \"e7778a88-c2f0-40f6-987a-8749b0cf3378\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4"
Oct 30 19:47:45.633843 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:45.633753    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8jsh\" (UniqueName: \"kubernetes.io/projected/9584be0d-abe4-4783-8849-f4b79f3094db-kube-api-access-k8jsh\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l\" (UID: \"9584be0d-abe4-4783-8849-f4b79f3094db\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l"
Oct 30 19:47:45.633843 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:45.633798    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7778a88-c2f0-40f6-987a-8749b0cf3378-workload-certs\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4\" (UID: \"e7778a88-c2f0-40f6-987a-8749b0cf3378\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4"
Oct 30 19:47:45.633843 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:45.633833    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e7778a88-c2f0-40f6-987a-8749b0cf3378-istiod-ca-cert\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4\" (UID: \"e7778a88-c2f0-40f6-987a-8749b0cf3378\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4"
Oct 30 19:47:45.634043 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:45.633862    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9584be0d-abe4-4783-8849-f4b79f3094db-credential-socket\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l\" (UID: \"9584be0d-abe4-4783-8849-f4b79f3094db\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l"
Oct 30 19:47:45.634043 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:45.633894    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grz7n\" (UniqueName: \"kubernetes.io/projected/e7778a88-c2f0-40f6-987a-8749b0cf3378-kube-api-access-grz7n\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4\" (UID: \"e7778a88-c2f0-40f6-987a-8749b0cf3378\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4"
Oct 30 19:47:45.634043 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:45.633948    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9584be0d-abe4-4783-8849-f4b79f3094db-istiod-ca-cert\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l\" (UID: \"9584be0d-abe4-4783-8849-f4b79f3094db\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l"
Oct 30 19:47:45.634371 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:45.634338    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7778a88-c2f0-40f6-987a-8749b0cf3378-credential-socket\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4\" (UID: \"e7778a88-c2f0-40f6-987a-8749b0cf3378\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4"
Oct 30 19:47:45.634561 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:45.634462    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7778a88-c2f0-40f6-987a-8749b0cf3378-workload-certs\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4\" (UID: \"e7778a88-c2f0-40f6-987a-8749b0cf3378\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4"
Oct 30 19:47:45.634760 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:45.634743    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7778a88-c2f0-40f6-987a-8749b0cf3378-workload-socket\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4\" (UID: \"e7778a88-c2f0-40f6-987a-8749b0cf3378\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4"
Oct 30 19:47:45.636236 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:45.635308    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e7778a88-c2f0-40f6-987a-8749b0cf3378-istio-envoy\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4\" (UID: \"e7778a88-c2f0-40f6-987a-8749b0cf3378\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4"
Oct 30 19:47:45.636236 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:45.635326    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e7778a88-c2f0-40f6-987a-8749b0cf3378-istiod-ca-cert\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4\" (UID: \"e7778a88-c2f0-40f6-987a-8749b0cf3378\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4"
Oct 30 19:47:45.636236 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:45.635569    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9584be0d-abe4-4783-8849-f4b79f3094db-istio-envoy\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l\" (UID: \"9584be0d-abe4-4783-8849-f4b79f3094db\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l"
Oct 30 19:47:45.636236 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:45.635616    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9584be0d-abe4-4783-8849-f4b79f3094db-istio-podinfo\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l\" (UID: \"9584be0d-abe4-4783-8849-f4b79f3094db\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l"
Oct 30 19:47:45.636776 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:45.636758    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e7778a88-c2f0-40f6-987a-8749b0cf3378-istio-podinfo\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4\" (UID: \"e7778a88-c2f0-40f6-987a-8749b0cf3378\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4"
Oct 30 19:47:45.645548 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:45.645525    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9584be0d-abe4-4783-8849-f4b79f3094db-istio-token\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l\" (UID: \"9584be0d-abe4-4783-8849-f4b79f3094db\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l"
Oct 30 19:47:45.650890 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:45.650845    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-grz7n\" (UniqueName: \"kubernetes.io/projected/e7778a88-c2f0-40f6-987a-8749b0cf3378-kube-api-access-grz7n\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4\" (UID: \"e7778a88-c2f0-40f6-987a-8749b0cf3378\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4"
Oct 30 19:47:45.651327 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:45.651303    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7778a88-c2f0-40f6-987a-8749b0cf3378-istio-token\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4\" (UID: \"e7778a88-c2f0-40f6-987a-8749b0cf3378\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4"
Oct 30 19:47:45.651753 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:45.651703    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8jsh\" (UniqueName: \"kubernetes.io/projected/9584be0d-abe4-4783-8849-f4b79f3094db-kube-api-access-k8jsh\") pod \"e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l\" (UID: \"9584be0d-abe4-4783-8849-f4b79f3094db\") " pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l"
Oct 30 19:47:45.730176 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:45.729772    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4"
Oct 30 19:47:45.752522 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:45.752491    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l"
Oct 30 19:47:46.111653 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:46.111627    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4"]
Oct 30 19:47:46.114529 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:47:46.114474    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7778a88_c2f0_40f6_987a_8749b0cf3378.slice/crio-6dd3264cb7d446d6997a126c731af3d73877a1d43a76053790ed78f9d9b1b8ff WatchSource:0}: Error finding container 6dd3264cb7d446d6997a126c731af3d73877a1d43a76053790ed78f9d9b1b8ff: Status 404 returned error can't find the container with id 6dd3264cb7d446d6997a126c731af3d73877a1d43a76053790ed78f9d9b1b8ff
Oct 30 19:47:46.115407 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:46.115382    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l"]
Oct 30 19:47:46.115814 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:47:46.115792    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9584be0d_abe4_4783_8849_f4b79f3094db.slice/crio-2aaca1e06eee25cceb1c97204eb643c3e9b8b22cc17d94e2fd5242aa84c24054 WatchSource:0}: Error finding container 2aaca1e06eee25cceb1c97204eb643c3e9b8b22cc17d94e2fd5242aa84c24054: Status 404 returned error can't find the container with id 2aaca1e06eee25cceb1c97204eb643c3e9b8b22cc17d94e2fd5242aa84c24054
Oct 30 19:47:46.117260 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:46.117223    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:47:46.117363 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:46.117291    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:47:46.117363 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:46.117320    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:47:46.119012 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:46.118976    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:47:46.119075 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:46.119059    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:47:46.119125 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:46.119110    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:47:46.151593 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:46.151447    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4" event={"ID":"e7778a88-c2f0-40f6-987a-8749b0cf3378","Type":"ContainerStarted","Data":"6dd3264cb7d446d6997a126c731af3d73877a1d43a76053790ed78f9d9b1b8ff"}
Oct 30 19:47:46.153269 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:46.153188    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l" event={"ID":"9584be0d-abe4-4783-8849-f4b79f3094db","Type":"ContainerStarted","Data":"2aaca1e06eee25cceb1c97204eb643c3e9b8b22cc17d94e2fd5242aa84c24054"}
Oct 30 19:47:47.203307 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:47.203263    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4" event={"ID":"e7778a88-c2f0-40f6-987a-8749b0cf3378","Type":"ContainerStarted","Data":"acf5ec0f48e7c4607b5377428016569da612b3cc40648339cf303c66a0e57fb2"}
Oct 30 19:47:47.205189 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:47.205159    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l" event={"ID":"9584be0d-abe4-4783-8849-f4b79f3094db","Type":"ContainerStarted","Data":"391c0de62fd01a10bb0e6642a23af18309b2acae2d4ebe39f9c032df75b9cca5"}
Oct 30 19:47:47.235666 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:47.235620    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4" podStartSLOduration=2.235606077 podStartE2EDuration="2.235606077s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:47:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:47:47.232732827 +0000 UTC m=+942.560850980" watchObservedRunningTime="2026-10-30 19:47:47.235606077 +0000 UTC m=+942.563724223"
Oct 30 19:47:47.731348 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:47.731316    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4"
Oct 30 19:47:47.732851 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:47.732824    6996 patch_prober.go:29] interesting pod/e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.103:15021/healthz/ready\": dial tcp 10.134.0.103:15021: connect: connection refused" start-of-body=
Oct 30 19:47:47.732966 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:47.732871    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4" podUID="e7778a88-c2f0-40f6-987a-8749b0cf3378" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.103:15021/healthz/ready\": dial tcp 10.134.0.103:15021: connect: connection refused"
Oct 30 19:47:47.755838 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:47.755810    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l"
Oct 30 19:47:47.761638 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:47.761609    6996 patch_prober.go:29] interesting pod/e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused" start-of-body=
Oct 30 19:47:47.762068 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:47.762039    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l" podUID="9584be0d-abe4-4783-8849-f4b79f3094db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused"
Oct 30 19:47:48.089524 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:48.089181    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-501ac8-dd4dc7b8c-bnggs"
Oct 30 19:47:48.090701 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:48.090677    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-501ac8-768b76884b-8wrfj"
Oct 30 19:47:48.114351 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:48.114296    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l" podStartSLOduration=3.114282152 podStartE2EDuration="3.114282152s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:47:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:47:47.281022356 +0000 UTC m=+942.609140511" watchObservedRunningTime="2026-10-30 19:47:48.114282152 +0000 UTC m=+943.442400308"
Oct 30 19:47:48.132206 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:48.132185    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-501ac8-5c4f9d85bb-pvmxr"
Oct 30 19:47:48.742503 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:48.742239    6996 patch_prober.go:29] interesting pod/e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.103:15021/healthz/ready\": dial tcp 10.134.0.103:15021: connect: connection refused" start-of-body=
Oct 30 19:47:48.742503 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:48.742301    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4" podUID="e7778a88-c2f0-40f6-987a-8749b0cf3378" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.103:15021/healthz/ready\": dial tcp 10.134.0.103:15021: connect: connection refused"
Oct 30 19:47:48.754435 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:48.754402    6996 patch_prober.go:29] interesting pod/e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused" start-of-body=
Oct 30 19:47:48.754757 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:48.754723    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l" podUID="9584be0d-abe4-4783-8849-f4b79f3094db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused"
Oct 30 19:47:49.730490 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:49.730441    6996 patch_prober.go:29] interesting pod/e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.103:15021/healthz/ready\": dial tcp 10.134.0.103:15021: connect: connection refused" start-of-body=
Oct 30 19:47:49.730643 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:49.730515    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4" podUID="e7778a88-c2f0-40f6-987a-8749b0cf3378" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.103:15021/healthz/ready\": dial tcp 10.134.0.103:15021: connect: connection refused"
Oct 30 19:47:49.758345 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:49.757889    6996 patch_prober.go:29] interesting pod/e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused" start-of-body=
Oct 30 19:47:49.758345 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:49.757949    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l" podUID="9584be0d-abe4-4783-8849-f4b79f3094db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused"
Oct 30 19:47:50.731570 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:50.731532    6996 patch_prober.go:29] interesting pod/e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.103:15021/healthz/ready\": dial tcp 10.134.0.103:15021: connect: connection refused" start-of-body=
Oct 30 19:47:50.731755 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:50.731589    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4" podUID="e7778a88-c2f0-40f6-987a-8749b0cf3378" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.103:15021/healthz/ready\": dial tcp 10.134.0.103:15021: connect: connection refused"
Oct 30 19:47:50.753936 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:50.753888    6996 patch_prober.go:29] interesting pod/e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused" start-of-body=
Oct 30 19:47:50.754140 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:50.753964    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l" podUID="9584be0d-abe4-4783-8849-f4b79f3094db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused"
Oct 30 19:47:51.730754 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:51.730707    6996 patch_prober.go:29] interesting pod/e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.103:15021/healthz/ready\": dial tcp 10.134.0.103:15021: connect: connection refused" start-of-body=
Oct 30 19:47:51.731182 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:51.730772    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4" podUID="e7778a88-c2f0-40f6-987a-8749b0cf3378" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.103:15021/healthz/ready\": dial tcp 10.134.0.103:15021: connect: connection refused"
Oct 30 19:47:51.763505 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:51.760015    6996 patch_prober.go:29] interesting pod/e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused" start-of-body=
Oct 30 19:47:51.763505 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:51.760093    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l" podUID="9584be0d-abe4-4783-8849-f4b79f3094db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused"
Oct 30 19:47:51.999456 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:51.999372    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-477fd9c0-84cd565b7f-sggp9"]
Oct 30 19:47:52.004335 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:52.004315    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-477fd9c0-84cd565b7f-sggp9"
Oct 30 19:47:52.025516 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:52.025457    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7xwf\" (UniqueName: \"kubernetes.io/projected/6e6e973c-daab-4ccc-b552-ce2e718708a8-kube-api-access-n7xwf\") pod \"payload-pre-processing-e2e-mt-477fd9c0-84cd565b7f-sggp9\" (UID: \"6e6e973c-daab-4ccc-b552-ce2e718708a8\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-477fd9c0-84cd565b7f-sggp9"
Oct 30 19:47:52.026755 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:52.025577    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e6e973c-daab-4ccc-b552-ce2e718708a8-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-477fd9c0-84cd565b7f-sggp9\" (UID: \"6e6e973c-daab-4ccc-b552-ce2e718708a8\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-477fd9c0-84cd565b7f-sggp9"
Oct 30 19:47:52.030784 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:52.030749    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-477fd9c0-84cd565b7f-sggp9"]
Oct 30 19:47:52.106502 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:52.100832    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-477fd9c0-dockercfg-fmnbv\""
Oct 30 19:47:52.106502 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:52.100934    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-477fd9c0\""
Oct 30 19:47:52.182522 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:52.181558    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7xwf\" (UniqueName: \"kubernetes.io/projected/6e6e973c-daab-4ccc-b552-ce2e718708a8-kube-api-access-n7xwf\") pod \"payload-pre-processing-e2e-mt-477fd9c0-84cd565b7f-sggp9\" (UID: \"6e6e973c-daab-4ccc-b552-ce2e718708a8\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-477fd9c0-84cd565b7f-sggp9"
Oct 30 19:47:52.182522 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:52.181700    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e6e973c-daab-4ccc-b552-ce2e718708a8-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-477fd9c0-84cd565b7f-sggp9\" (UID: \"6e6e973c-daab-4ccc-b552-ce2e718708a8\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-477fd9c0-84cd565b7f-sggp9"
Oct 30 19:47:52.182859 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:52.182766    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e6e973c-daab-4ccc-b552-ce2e718708a8-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-477fd9c0-84cd565b7f-sggp9\" (UID: \"6e6e973c-daab-4ccc-b552-ce2e718708a8\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-477fd9c0-84cd565b7f-sggp9"
Oct 30 19:47:52.306853 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:52.305392    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7xwf\" (UniqueName: \"kubernetes.io/projected/6e6e973c-daab-4ccc-b552-ce2e718708a8-kube-api-access-n7xwf\") pod \"payload-pre-processing-e2e-mt-477fd9c0-84cd565b7f-sggp9\" (UID: \"6e6e973c-daab-4ccc-b552-ce2e718708a8\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-477fd9c0-84cd565b7f-sggp9"
Oct 30 19:47:52.319895 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:52.319860    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-477fd9c0-84cd565b7f-sggp9"
Oct 30 19:47:52.621459 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:47:52.621328    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e6e973c_daab_4ccc_b552_ce2e718708a8.slice/crio-1bdd57500f71fe991c3e049e7d769dc205b31243604457f2fb55f5aad60184ab WatchSource:0}: Error finding container 1bdd57500f71fe991c3e049e7d769dc205b31243604457f2fb55f5aad60184ab: Status 404 returned error can't find the container with id 1bdd57500f71fe991c3e049e7d769dc205b31243604457f2fb55f5aad60184ab
Oct 30 19:47:52.622687 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:52.622662    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-477fd9c0-84cd565b7f-sggp9"]
Oct 30 19:47:52.730765 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:52.730737    6996 patch_prober.go:29] interesting pod/e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.103:15021/healthz/ready\": dial tcp 10.134.0.103:15021: connect: connection refused" start-of-body=
Oct 30 19:47:52.731228 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:52.730782    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4" podUID="e7778a88-c2f0-40f6-987a-8749b0cf3378" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.103:15021/healthz/ready\": dial tcp 10.134.0.103:15021: connect: connection refused"
Oct 30 19:47:52.753308 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:52.753269    6996 patch_prober.go:29] interesting pod/e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused" start-of-body=
Oct 30 19:47:52.753459 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:52.753322    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l" podUID="9584be0d-abe4-4783-8849-f4b79f3094db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused"
Oct 30 19:47:53.343262 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:53.343227    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-477fd9c0-84cd565b7f-sggp9" event={"ID":"6e6e973c-daab-4ccc-b552-ce2e718708a8","Type":"ContainerStarted","Data":"68814908d4a8411ecd5af3f354a6e65f86c9418d143e7ce045ac96fec337ae3d"}
Oct 30 19:47:53.343500 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:53.343300    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-477fd9c0-84cd565b7f-sggp9"
Oct 30 19:47:53.343500 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:53.343322    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-477fd9c0-84cd565b7f-sggp9" event={"ID":"6e6e973c-daab-4ccc-b552-ce2e718708a8","Type":"ContainerStarted","Data":"1bdd57500f71fe991c3e049e7d769dc205b31243604457f2fb55f5aad60184ab"}
Oct 30 19:47:53.372967 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:53.372904    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-477fd9c0-84cd565b7f-sggp9" podStartSLOduration=2.372885999 podStartE2EDuration="2.372885999s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:47:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:47:53.371747904 +0000 UTC m=+948.699866058" watchObservedRunningTime="2026-10-30 19:47:53.372885999 +0000 UTC m=+948.701004168"
Oct 30 19:47:53.804163 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:53.803983    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4"
Oct 30 19:47:53.804662 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:53.804349    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4"
Oct 30 19:47:53.805148 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:53.805125    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4"
Oct 30 19:47:53.839823 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:53.839472    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l"
Oct 30 19:47:53.844517 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:53.841800    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l"
Oct 30 19:47:53.844517 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:53.843314    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l"
Oct 30 19:47:55.628502 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:55.626169    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r"]
Oct 30 19:47:55.676113 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:55.675261    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r"
Oct 30 19:47:55.681644 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:55.681615    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r"]
Oct 30 19:47:55.689502 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:55.689164    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-2761c7-openshift-default-dockercfg-qrq4t\""
Oct 30 19:47:55.774417 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:55.774378    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5aaa21a-1a12-4b2b-b05e-f08504685cd8-credential-socket\") pod \"e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r\" (UID: \"e5aaa21a-1a12-4b2b-b05e-f08504685cd8\") " pod="openshift-ingress/e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r"
Oct 30 19:47:55.774655 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:55.774449    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5aaa21a-1a12-4b2b-b05e-f08504685cd8-istio-podinfo\") pod \"e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r\" (UID: \"e5aaa21a-1a12-4b2b-b05e-f08504685cd8\") " pod="openshift-ingress/e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r"
Oct 30 19:47:55.774655 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:55.774599    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5aaa21a-1a12-4b2b-b05e-f08504685cd8-workload-certs\") pod \"e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r\" (UID: \"e5aaa21a-1a12-4b2b-b05e-f08504685cd8\") " pod="openshift-ingress/e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r"
Oct 30 19:47:55.774655 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:55.774631    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5aaa21a-1a12-4b2b-b05e-f08504685cd8-istio-token\") pod \"e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r\" (UID: \"e5aaa21a-1a12-4b2b-b05e-f08504685cd8\") " pod="openshift-ingress/e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r"
Oct 30 19:47:55.774783 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:55.774724    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5aaa21a-1a12-4b2b-b05e-f08504685cd8-istio-envoy\") pod \"e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r\" (UID: \"e5aaa21a-1a12-4b2b-b05e-f08504685cd8\") " pod="openshift-ingress/e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r"
Oct 30 19:47:55.774853 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:55.774832    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5aaa21a-1a12-4b2b-b05e-f08504685cd8-istio-data\") pod \"e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r\" (UID: \"e5aaa21a-1a12-4b2b-b05e-f08504685cd8\") " pod="openshift-ingress/e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r"
Oct 30 19:47:55.774913 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:55.774866    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5aaa21a-1a12-4b2b-b05e-f08504685cd8-istiod-ca-cert\") pod \"e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r\" (UID: \"e5aaa21a-1a12-4b2b-b05e-f08504685cd8\") " pod="openshift-ingress/e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r"
Oct 30 19:47:55.775002 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:55.774976    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5aaa21a-1a12-4b2b-b05e-f08504685cd8-workload-socket\") pod \"e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r\" (UID: \"e5aaa21a-1a12-4b2b-b05e-f08504685cd8\") " pod="openshift-ingress/e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r"
Oct 30 19:47:55.775145 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:55.775038    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xddnz\" (UniqueName: \"kubernetes.io/projected/e5aaa21a-1a12-4b2b-b05e-f08504685cd8-kube-api-access-xddnz\") pod \"e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r\" (UID: \"e5aaa21a-1a12-4b2b-b05e-f08504685cd8\") " pod="openshift-ingress/e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r"
Oct 30 19:47:55.876070 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:55.876033    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5aaa21a-1a12-4b2b-b05e-f08504685cd8-credential-socket\") pod \"e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r\" (UID: \"e5aaa21a-1a12-4b2b-b05e-f08504685cd8\") " pod="openshift-ingress/e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r"
Oct 30 19:47:55.876269 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:55.876217    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5aaa21a-1a12-4b2b-b05e-f08504685cd8-istio-podinfo\") pod \"e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r\" (UID: \"e5aaa21a-1a12-4b2b-b05e-f08504685cd8\") " pod="openshift-ingress/e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r"
Oct 30 19:47:55.876411 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:55.876373    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5aaa21a-1a12-4b2b-b05e-f08504685cd8-workload-certs\") pod \"e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r\" (UID: \"e5aaa21a-1a12-4b2b-b05e-f08504685cd8\") " pod="openshift-ingress/e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r"
Oct 30 19:47:55.876574 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:55.876424    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5aaa21a-1a12-4b2b-b05e-f08504685cd8-istio-token\") pod \"e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r\" (UID: \"e5aaa21a-1a12-4b2b-b05e-f08504685cd8\") " pod="openshift-ingress/e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r"
Oct 30 19:47:55.876574 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:55.876521    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5aaa21a-1a12-4b2b-b05e-f08504685cd8-istio-envoy\") pod \"e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r\" (UID: \"e5aaa21a-1a12-4b2b-b05e-f08504685cd8\") " pod="openshift-ingress/e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r"
Oct 30 19:47:55.876574 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:55.876539    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5aaa21a-1a12-4b2b-b05e-f08504685cd8-credential-socket\") pod \"e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r\" (UID: \"e5aaa21a-1a12-4b2b-b05e-f08504685cd8\") " pod="openshift-ingress/e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r"
Oct 30 19:47:55.876784 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:55.876761    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5aaa21a-1a12-4b2b-b05e-f08504685cd8-workload-certs\") pod \"e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r\" (UID: \"e5aaa21a-1a12-4b2b-b05e-f08504685cd8\") " pod="openshift-ingress/e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r"
Oct 30 19:47:55.876877 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:55.876858    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5aaa21a-1a12-4b2b-b05e-f08504685cd8-istio-data\") pod \"e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r\" (UID: \"e5aaa21a-1a12-4b2b-b05e-f08504685cd8\") " pod="openshift-ingress/e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r"
Oct 30 19:47:55.876955 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:55.876905    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5aaa21a-1a12-4b2b-b05e-f08504685cd8-istiod-ca-cert\") pod \"e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r\" (UID: \"e5aaa21a-1a12-4b2b-b05e-f08504685cd8\") " pod="openshift-ingress/e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r"
Oct 30 19:47:55.877008 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:55.876992    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5aaa21a-1a12-4b2b-b05e-f08504685cd8-workload-socket\") pod \"e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r\" (UID: \"e5aaa21a-1a12-4b2b-b05e-f08504685cd8\") " pod="openshift-ingress/e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r"
Oct 30 19:47:55.877133 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:55.877101    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xddnz\" (UniqueName: \"kubernetes.io/projected/e5aaa21a-1a12-4b2b-b05e-f08504685cd8-kube-api-access-xddnz\") pod \"e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r\" (UID: \"e5aaa21a-1a12-4b2b-b05e-f08504685cd8\") " pod="openshift-ingress/e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r"
Oct 30 19:47:55.877225 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:55.877140    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5aaa21a-1a12-4b2b-b05e-f08504685cd8-istio-data\") pod \"e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r\" (UID: \"e5aaa21a-1a12-4b2b-b05e-f08504685cd8\") " pod="openshift-ingress/e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r"
Oct 30 19:47:55.878771 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:55.877720    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5aaa21a-1a12-4b2b-b05e-f08504685cd8-workload-socket\") pod \"e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r\" (UID: \"e5aaa21a-1a12-4b2b-b05e-f08504685cd8\") " pod="openshift-ingress/e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r"
Oct 30 19:47:55.879281 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:55.879253    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5aaa21a-1a12-4b2b-b05e-f08504685cd8-istio-envoy\") pod \"e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r\" (UID: \"e5aaa21a-1a12-4b2b-b05e-f08504685cd8\") " pod="openshift-ingress/e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r"
Oct 30 19:47:55.879367 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:55.879289    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5aaa21a-1a12-4b2b-b05e-f08504685cd8-istiod-ca-cert\") pod \"e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r\" (UID: \"e5aaa21a-1a12-4b2b-b05e-f08504685cd8\") " pod="openshift-ingress/e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r"
Oct 30 19:47:55.879772 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:55.879748    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5aaa21a-1a12-4b2b-b05e-f08504685cd8-istio-podinfo\") pod \"e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r\" (UID: \"e5aaa21a-1a12-4b2b-b05e-f08504685cd8\") " pod="openshift-ingress/e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r"
Oct 30 19:47:55.906348 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:55.906313    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xddnz\" (UniqueName: \"kubernetes.io/projected/e5aaa21a-1a12-4b2b-b05e-f08504685cd8-kube-api-access-xddnz\") pod \"e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r\" (UID: \"e5aaa21a-1a12-4b2b-b05e-f08504685cd8\") " pod="openshift-ingress/e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r"
Oct 30 19:47:55.907117 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:55.907090    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5aaa21a-1a12-4b2b-b05e-f08504685cd8-istio-token\") pod \"e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r\" (UID: \"e5aaa21a-1a12-4b2b-b05e-f08504685cd8\") " pod="openshift-ingress/e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r"
Oct 30 19:47:56.009155 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:56.009118    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r"
Oct 30 19:47:56.317579 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:47:56.317547    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5aaa21a_1a12_4b2b_b05e_f08504685cd8.slice/crio-98e380ddc9a7677fc6bf5cdebfc44ed6107506f63ad0bed3268bd8df9c37b103 WatchSource:0}: Error finding container 98e380ddc9a7677fc6bf5cdebfc44ed6107506f63ad0bed3268bd8df9c37b103: Status 404 returned error can't find the container with id 98e380ddc9a7677fc6bf5cdebfc44ed6107506f63ad0bed3268bd8df9c37b103
Oct 30 19:47:56.318577 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:56.318550    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r"]
Oct 30 19:47:56.320741 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:56.320710    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:47:56.320824 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:56.320771    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:47:56.320824 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:56.320803    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:47:56.373411 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:56.373367    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r" event={"ID":"e5aaa21a-1a12-4b2b-b05e-f08504685cd8","Type":"ContainerStarted","Data":"98e380ddc9a7677fc6bf5cdebfc44ed6107506f63ad0bed3268bd8df9c37b103"}
Oct 30 19:47:57.385895 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:57.385852    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r" event={"ID":"e5aaa21a-1a12-4b2b-b05e-f08504685cd8","Type":"ContainerStarted","Data":"2607a1ffd03e61d227ab1169ca7c998fceec6b47285852a37afcafe1e231c02c"}
Oct 30 19:47:57.418731 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:57.418682    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r" podStartSLOduration=2.418667633 podStartE2EDuration="2.418667633s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:47:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:47:57.414435429 +0000 UTC m=+952.742553586" watchObservedRunningTime="2026-10-30 19:47:57.418667633 +0000 UTC m=+952.746785790"
Oct 30 19:47:58.012694 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:58.012638    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r"
Oct 30 19:47:58.021797 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:58.021762    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r"
Oct 30 19:47:58.417143 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:58.416972    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r"
Oct 30 19:47:58.433233 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:58.432849    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r"
Oct 30 19:47:59.524505 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:59.523588    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq" podUID="89009638-003d-4a5d-ab13-1f8c8f64d3b7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.88:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:47:59.775998 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:59.764650    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr" podUID="0e470010-625d-443f-81ec-9bf965988212" containerName="main" probeResult="failure" output="Get \"https://10.134.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:47:59.863053 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:59.862561    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s" podUID="ae905dfb-f30d-480a-899b-ea676abc1d8d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:47:59.913613 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:59.909574    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk" podUID="a1857d2d-3b40-4e63-be33-be4ba476f926" containerName="main" probeResult="failure" output="Get \"https://10.134.0.90:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:47:59.976327 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:47:59.972598    6996 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk" podUID="a1857d2d-3b40-4e63-be33-be4ba476f926" containerName="main" probeResult="failure" output="Get \"https://10.134.0.90:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:48:00.094948 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:00.094788    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b" podUID="19fe4fba-09ae-4620-b4d2-0ef8c2463d71" containerName="main" probeResult="failure" output="Get \"https://10.134.0.81:8000/ready\": context deadline exceeded"
Oct 30 19:48:00.103600 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:00.103559    6996 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr" podUID="0e470010-625d-443f-81ec-9bf965988212" containerName="main" probeResult="failure" output="Get \"https://10.134.0.72:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:48:00.512553 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:00.498587    6996 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5db77fbd9f-sjbzh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:48:00.512553 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:00.498658    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sjbzh" podUID="b79c69dc-9d5b-4b3b-802e-ac414149df33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:48:00.627504 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:00.577686    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq" podUID="89009638-003d-4a5d-ab13-1f8c8f64d3b7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.88:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:48:00.627504 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:00.578107    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq"
Oct 30 19:48:00.778830 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:00.777763    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr" podUID="0e470010-625d-443f-81ec-9bf965988212" containerName="main" probeResult="failure" output="Get \"https://10.134.0.72:8000/ready\": context deadline exceeded"
Oct 30 19:48:00.780557 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:00.779454    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr"
Oct 30 19:48:00.943151 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:00.943092    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s" podUID="ae905dfb-f30d-480a-899b-ea676abc1d8d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.73:8000/ready\": context deadline exceeded"
Oct 30 19:48:01.119825 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:01.117682    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s"
Oct 30 19:48:01.121082 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:01.121047    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b" podUID="19fe4fba-09ae-4620-b4d2-0ef8c2463d71" containerName="main" probeResult="failure" output="Get \"https://10.134.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:48:01.123853 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:01.123832    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b"
Oct 30 19:48:01.224502 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:01.205174    6996 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b" podUID="19fe4fba-09ae-4620-b4d2-0ef8c2463d71" containerName="main" probeResult="failure" output="Get \"https://10.134.0.81:8000/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:48:01.227533 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:01.227506    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s"
Oct 30 19:48:01.227741 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:01.227728    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq"
Oct 30 19:48:01.228766 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:01.228750    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr"
Oct 30 19:48:01.251555 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:01.248947    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b"
Oct 30 19:48:01.310903 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:48:01.308977    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 19:48:01.689249 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:01.688535    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-2761c7-776cb78fff-k5tqc"]
Oct 30 19:48:01.694289 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:01.694251    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-2761c7-776cb78fff-k5tqc"
Oct 30 19:48:01.705877 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:01.705848    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-2761c7-776cb78fff-k5tqc"]
Oct 30 19:48:01.790504 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:01.783333    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-2761c7\""
Oct 30 19:48:01.790504 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:01.788467    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7eefebe9-9a98-49fc-a091-151f8719e3f2-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-2761c7-776cb78fff-k5tqc\" (UID: \"7eefebe9-9a98-49fc-a091-151f8719e3f2\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-2761c7-776cb78fff-k5tqc"
Oct 30 19:48:01.790504 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:01.788669    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5t2s8\" (UniqueName: \"kubernetes.io/projected/7eefebe9-9a98-49fc-a091-151f8719e3f2-kube-api-access-5t2s8\") pod \"payload-pre-processing-e2e-api-b-2761c7-776cb78fff-k5tqc\" (UID: \"7eefebe9-9a98-49fc-a091-151f8719e3f2\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-2761c7-776cb78fff-k5tqc"
Oct 30 19:48:01.799510 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:01.797727    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-2761c7-dockercfg-gghph\""
Oct 30 19:48:01.964743 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:01.912812    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7eefebe9-9a98-49fc-a091-151f8719e3f2-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-2761c7-776cb78fff-k5tqc\" (UID: \"7eefebe9-9a98-49fc-a091-151f8719e3f2\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-2761c7-776cb78fff-k5tqc"
Oct 30 19:48:01.964743 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:01.913240    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5t2s8\" (UniqueName: \"kubernetes.io/projected/7eefebe9-9a98-49fc-a091-151f8719e3f2-kube-api-access-5t2s8\") pod \"payload-pre-processing-e2e-api-b-2761c7-776cb78fff-k5tqc\" (UID: \"7eefebe9-9a98-49fc-a091-151f8719e3f2\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-2761c7-776cb78fff-k5tqc"
Oct 30 19:48:01.993508 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:01.974058    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7eefebe9-9a98-49fc-a091-151f8719e3f2-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-2761c7-776cb78fff-k5tqc\" (UID: \"7eefebe9-9a98-49fc-a091-151f8719e3f2\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-2761c7-776cb78fff-k5tqc"
Oct 30 19:48:02.001632 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:02.000247    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5t2s8\" (UniqueName: \"kubernetes.io/projected/7eefebe9-9a98-49fc-a091-151f8719e3f2-kube-api-access-5t2s8\") pod \"payload-pre-processing-e2e-api-b-2761c7-776cb78fff-k5tqc\" (UID: \"7eefebe9-9a98-49fc-a091-151f8719e3f2\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-2761c7-776cb78fff-k5tqc"
Oct 30 19:48:02.019007 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:02.018973    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-2761c7-776cb78fff-k5tqc"
Oct 30 19:48:02.429182 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:02.429139    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-2761c7-776cb78fff-k5tqc"]
Oct 30 19:48:02.476697 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:48:02.476669    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7eefebe9_9a98_49fc_a091_151f8719e3f2.slice/crio-216b921ae9c9406d6f54b316466a20806355bb7a174843c78d5d3bbea2768cbd WatchSource:0}: Error finding container 216b921ae9c9406d6f54b316466a20806355bb7a174843c78d5d3bbea2768cbd: Status 404 returned error can't find the container with id 216b921ae9c9406d6f54b316466a20806355bb7a174843c78d5d3bbea2768cbd
Oct 30 19:48:02.909213 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:02.909170    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-2761c7-776cb78fff-k5tqc" event={"ID":"7eefebe9-9a98-49fc-a091-151f8719e3f2","Type":"ContainerStarted","Data":"2b598e119697484f88330b4a49baa3f9e430a7c1ab9b0a87ee8c0a2f9c388599"}
Oct 30 19:48:02.909875 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:02.909421    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-2761c7-776cb78fff-k5tqc"
Oct 30 19:48:02.909875 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:02.909461    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-2761c7-776cb78fff-k5tqc" event={"ID":"7eefebe9-9a98-49fc-a091-151f8719e3f2","Type":"ContainerStarted","Data":"216b921ae9c9406d6f54b316466a20806355bb7a174843c78d5d3bbea2768cbd"}
Oct 30 19:48:02.965567 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:02.963576    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-2761c7-776cb78fff-k5tqc" podStartSLOduration=1.963560091 podStartE2EDuration="1.963560091s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:48:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:48:02.962708713 +0000 UTC m=+958.290826869" watchObservedRunningTime="2026-10-30 19:48:02.963560091 +0000 UTC m=+958.291678244"
Oct 30 19:48:04.359255 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:04.359227    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-477fd9c0-84cd565b7f-sggp9"
Oct 30 19:48:08.968871 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:08.968830    6996 generic.go:356] "Generic (PLEG): container finished" podID="a61de104-8144-4107-a532-61cba1cf53e1" containerID="b1a3f3d7b4883cad7a223d26a2d85141b389c8b54535927589932316607b26ec" exitCode=0
Oct 30 19:48:08.969416 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:08.968896    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7" event={"ID":"a61de104-8144-4107-a532-61cba1cf53e1","Type":"ContainerDied","Data":"b1a3f3d7b4883cad7a223d26a2d85141b389c8b54535927589932316607b26ec"}
Oct 30 19:48:08.969522 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:08.969415    6996 scope.go:122] "RemoveContainer" containerID="b1a3f3d7b4883cad7a223d26a2d85141b389c8b54535927589932316607b26ec"
Oct 30 19:48:08.971111 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:08.971067    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:48:08.971348 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:08.971323    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:48:08.971429 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:08.971381    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:48:09.978502 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:09.978454    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7" event={"ID":"a61de104-8144-4107-a532-61cba1cf53e1","Type":"ContainerStarted","Data":"f5a0a42afdb4b9a595e1d4f596fd0061d8c85f4a9d6186e9108bc96b5e05f9c6"}
Oct 30 19:48:10.478505 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:10.477860    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7"
Oct 30 19:48:10.488571 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:10.479592    6996 patch_prober.go:29] interesting pod/e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused" start-of-body=
Oct 30 19:48:10.488571 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:10.479654    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7" podUID="a61de104-8144-4107-a532-61cba1cf53e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused"
Oct 30 19:48:11.473824 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:11.473787    6996 patch_prober.go:29] interesting pod/e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused" start-of-body=
Oct 30 19:48:11.474317 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:11.473856    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7" podUID="a61de104-8144-4107-a532-61cba1cf53e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused"
Oct 30 19:48:12.473944 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:12.473911    6996 patch_prober.go:29] interesting pod/e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused" start-of-body=
Oct 30 19:48:12.474338 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:12.473976    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7" podUID="a61de104-8144-4107-a532-61cba1cf53e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused"
Oct 30 19:48:13.364143 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:13.362748    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 19:48:13.485655 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:13.485616    6996 patch_prober.go:29] interesting pod/e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused" start-of-body=
Oct 30 19:48:13.486265 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:13.486229    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7" podUID="a61de104-8144-4107-a532-61cba1cf53e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused"
Oct 30 19:48:13.934960 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:13.934931    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-2761c7-776cb78fff-k5tqc"
Oct 30 19:48:14.481504 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:14.477025    6996 patch_prober.go:29] interesting pod/e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused" start-of-body=
Oct 30 19:48:14.481504 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:14.477090    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7" podUID="a61de104-8144-4107-a532-61cba1cf53e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused"
Oct 30 19:48:15.473710 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:15.473668    6996 patch_prober.go:29] interesting pod/e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused" start-of-body=
Oct 30 19:48:15.474204 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:15.473735    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7" podUID="a61de104-8144-4107-a532-61cba1cf53e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused"
Oct 30 19:48:16.478420 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:16.478389    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7"
Oct 30 19:48:16.478928 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:16.478909    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7"
Oct 30 19:48:16.479608 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:16.479589    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7"
Oct 30 19:48:19.382041 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:19.382005    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7325-0f2cc4"]
Oct 30 19:48:19.387864 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:19.387842    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7325-0f2cc4"
Oct 30 19:48:19.391727 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:19.391702    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-dqx9r\""
Oct 30 19:48:19.394757 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:19.394733    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7325-0f2cc4"]
Oct 30 19:48:19.477959 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:19.477910    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5j8rh\" (UniqueName: \"kubernetes.io/projected/2777eb12-505d-4f1b-becd-06e6e9104462-kube-api-access-5j8rh\") pod \"test-curl-7325-0f2cc4\" (UID: \"2777eb12-505d-4f1b-becd-06e6e9104462\") " pod="opendatahub/test-curl-7325-0f2cc4"
Oct 30 19:48:19.579380 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:19.579343    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5j8rh\" (UniqueName: \"kubernetes.io/projected/2777eb12-505d-4f1b-becd-06e6e9104462-kube-api-access-5j8rh\") pod \"test-curl-7325-0f2cc4\" (UID: \"2777eb12-505d-4f1b-becd-06e6e9104462\") " pod="opendatahub/test-curl-7325-0f2cc4"
Oct 30 19:48:19.592592 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:19.592559    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5j8rh\" (UniqueName: \"kubernetes.io/projected/2777eb12-505d-4f1b-becd-06e6e9104462-kube-api-access-5j8rh\") pod \"test-curl-7325-0f2cc4\" (UID: \"2777eb12-505d-4f1b-becd-06e6e9104462\") " pod="opendatahub/test-curl-7325-0f2cc4"
Oct 30 19:48:19.703401 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:19.703325    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7325-0f2cc4"
Oct 30 19:48:19.987234 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:19.987209    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7325-0f2cc4"]
Oct 30 19:48:19.989548 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:48:19.989516    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2777eb12_505d_4f1b_becd_06e6e9104462.slice/crio-92e36ba2f0ef3c7a18ad3dc972609924d28628df9b60666b1ccdf37940c4ae6e WatchSource:0}: Error finding container 92e36ba2f0ef3c7a18ad3dc972609924d28628df9b60666b1ccdf37940c4ae6e: Status 404 returned error can't find the container with id 92e36ba2f0ef3c7a18ad3dc972609924d28628df9b60666b1ccdf37940c4ae6e
Oct 30 19:48:20.096606 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:20.096570    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7325-0f2cc4" event={"ID":"2777eb12-505d-4f1b-becd-06e6e9104462","Type":"ContainerStarted","Data":"9dc26be775b2145ad39d4bc6060e7f88b7087a156f88c989fe073707c4e3b9fc"}
Oct 30 19:48:20.096797 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:20.096615    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7325-0f2cc4" event={"ID":"2777eb12-505d-4f1b-becd-06e6e9104462","Type":"ContainerStarted","Data":"92e36ba2f0ef3c7a18ad3dc972609924d28628df9b60666b1ccdf37940c4ae6e"}
Oct 30 19:48:20.121196 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:20.121144    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7325-0f2cc4" podStartSLOduration=1.121128776 podStartE2EDuration="1.121128776s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:48:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:48:20.116987956 +0000 UTC m=+975.445106109" watchObservedRunningTime="2026-10-30 19:48:20.121128776 +0000 UTC m=+975.449246929"
Oct 30 19:48:20.894179 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:20.894145    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7325-0f2cc4"]
Oct 30 19:48:20.901143 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:20.901114    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7325-0f2cc4"]
Oct 30 19:48:22.121347 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:22.121305    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7325-0f2cc4" podUID="2777eb12-505d-4f1b-becd-06e6e9104462" containerName="test-curl-7325-0f2cc4" containerID="cri-o://9dc26be775b2145ad39d4bc6060e7f88b7087a156f88c989fe073707c4e3b9fc" gracePeriod=2
Oct 30 19:48:22.123629 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:48:22.123584    6996 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7325-0f2cc4\" is forbidden: User \"system:node:ip-10-0-141-42.us-east-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-141-42.us-east-2.compute.internal' and this object" podUID="2777eb12-505d-4f1b-becd-06e6e9104462" pod="opendatahub/test-curl-7325-0f2cc4"
Oct 30 19:48:24.345558 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:24.345534    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7325-0f2cc4"
Oct 30 19:48:24.456052 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:24.455968    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2777eb12-505d-4f1b-becd-06e6e9104462-kube-api-access-5j8rh\" (UniqueName: \"kubernetes.io/projected/2777eb12-505d-4f1b-becd-06e6e9104462-kube-api-access-5j8rh\") pod \"2777eb12-505d-4f1b-becd-06e6e9104462\" (UID: \"2777eb12-505d-4f1b-becd-06e6e9104462\") "
Oct 30 19:48:24.459033 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:24.459005    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2777eb12-505d-4f1b-becd-06e6e9104462-kube-api-access-5j8rh" pod "2777eb12-505d-4f1b-becd-06e6e9104462" (UID: "2777eb12-505d-4f1b-becd-06e6e9104462"). InnerVolumeSpecName "kube-api-access-5j8rh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:48:24.558468 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:24.558429    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5j8rh\" (UniqueName: \"kubernetes.io/projected/2777eb12-505d-4f1b-becd-06e6e9104462-kube-api-access-5j8rh\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:24.794154 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:24.794077    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2777eb12-505d-4f1b-becd-06e6e9104462" path="/var/lib/kubelet/pods/2777eb12-505d-4f1b-becd-06e6e9104462/volumes"
Oct 30 19:48:25.143968 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:25.143931    6996 generic.go:356] "Generic (PLEG): container finished" podID="2777eb12-505d-4f1b-becd-06e6e9104462" containerID="9dc26be775b2145ad39d4bc6060e7f88b7087a156f88c989fe073707c4e3b9fc" exitCode=137
Oct 30 19:48:25.144137 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:25.143975    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7325-0f2cc4"
Oct 30 19:48:25.144137 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:25.144011    6996 scope.go:122] "RemoveContainer" containerID="9dc26be775b2145ad39d4bc6060e7f88b7087a156f88c989fe073707c4e3b9fc"
Oct 30 19:48:25.156028 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:25.156006    6996 scope.go:122] "RemoveContainer" containerID="9dc26be775b2145ad39d4bc6060e7f88b7087a156f88c989fe073707c4e3b9fc"
Oct 30 19:48:25.156355 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:48:25.156329    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9dc26be775b2145ad39d4bc6060e7f88b7087a156f88c989fe073707c4e3b9fc\": container with ID starting with 9dc26be775b2145ad39d4bc6060e7f88b7087a156f88c989fe073707c4e3b9fc not found: ID does not exist" containerID="9dc26be775b2145ad39d4bc6060e7f88b7087a156f88c989fe073707c4e3b9fc"
Oct 30 19:48:25.156409 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:25.156366    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9dc26be775b2145ad39d4bc6060e7f88b7087a156f88c989fe073707c4e3b9fc"} err="failed to get container status \"9dc26be775b2145ad39d4bc6060e7f88b7087a156f88c989fe073707c4e3b9fc\": rpc error: code = NotFound desc = could not find container \"9dc26be775b2145ad39d4bc6060e7f88b7087a156f88c989fe073707c4e3b9fc\": container with ID starting with 9dc26be775b2145ad39d4bc6060e7f88b7087a156f88c989fe073707c4e3b9fc not found: ID does not exist"
Oct 30 19:48:31.199103 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:48:31.198641    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 19:48:31.578867 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:31.578822    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-913351-5587d6644c-25q64"]
Oct 30 19:48:31.579656 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:31.579633    6996 cpu_manager.go:414] "containerMap: removing container" podUID="2777eb12-505d-4f1b-becd-06e6e9104462" containerName="test-curl-7325-0f2cc4"
Oct 30 19:48:31.579656 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:31.579656    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2777eb12-505d-4f1b-becd-06e6e9104462" containerName="test-curl-7325-0f2cc4"
Oct 30 19:48:31.579860 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:31.579846    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="2777eb12-505d-4f1b-becd-06e6e9104462" containerName="test-curl-7325-0f2cc4"
Oct 30 19:48:31.585062 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:31.585042    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-913351-5587d6644c-25q64"
Oct 30 19:48:31.590673 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:31.590649    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-913351\""
Oct 30 19:48:31.591864 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:31.591679    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-913351-dockercfg-nz4pl\""
Oct 30 19:48:31.602699 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:31.602660    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-913351-5587d6644c-25q64"]
Oct 30 19:48:31.626702 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:31.626663    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-913351-64c79d9748-wzvj5"]
Oct 30 19:48:31.632484 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:31.632449    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-913351-64c79d9748-wzvj5"
Oct 30 19:48:31.642710 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:31.642679    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-913351-64c79d9748-wzvj5"]
Oct 30 19:48:31.685511 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:31.685162    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f86eae85-6b3a-4c69-a001-b186bed6dd84-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-913351-5587d6644c-25q64\" (UID: \"f86eae85-6b3a-4c69-a001-b186bed6dd84\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-913351-5587d6644c-25q64"
Oct 30 19:48:31.685511 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:31.685331    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kr7xl\" (UniqueName: \"kubernetes.io/projected/f86eae85-6b3a-4c69-a001-b186bed6dd84-kube-api-access-kr7xl\") pod \"payload-pre-processing-e2e-ipp-a-913351-5587d6644c-25q64\" (UID: \"f86eae85-6b3a-4c69-a001-b186bed6dd84\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-913351-5587d6644c-25q64"
Oct 30 19:48:31.804503 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:31.789545    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1956ab1-08ac-4c8c-ab41-91993c5ccd75-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-913351-64c79d9748-wzvj5\" (UID: \"a1956ab1-08ac-4c8c-ab41-91993c5ccd75\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-913351-64c79d9748-wzvj5"
Oct 30 19:48:31.804503 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:31.789635    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f86eae85-6b3a-4c69-a001-b186bed6dd84-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-913351-5587d6644c-25q64\" (UID: \"f86eae85-6b3a-4c69-a001-b186bed6dd84\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-913351-5587d6644c-25q64"
Oct 30 19:48:31.804503 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:31.789770    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kr7xl\" (UniqueName: \"kubernetes.io/projected/f86eae85-6b3a-4c69-a001-b186bed6dd84-kube-api-access-kr7xl\") pod \"payload-pre-processing-e2e-ipp-a-913351-5587d6644c-25q64\" (UID: \"f86eae85-6b3a-4c69-a001-b186bed6dd84\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-913351-5587d6644c-25q64"
Oct 30 19:48:31.804503 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:31.789825    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ld2x\" (UniqueName: \"kubernetes.io/projected/a1956ab1-08ac-4c8c-ab41-91993c5ccd75-kube-api-access-9ld2x\") pod \"payload-processing-e2e-ipp-a-913351-64c79d9748-wzvj5\" (UID: \"a1956ab1-08ac-4c8c-ab41-91993c5ccd75\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-913351-64c79d9748-wzvj5"
Oct 30 19:48:31.804503 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:31.790903    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f86eae85-6b3a-4c69-a001-b186bed6dd84-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-913351-5587d6644c-25q64\" (UID: \"f86eae85-6b3a-4c69-a001-b186bed6dd84\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-913351-5587d6644c-25q64"
Oct 30 19:48:31.878585 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:31.878064    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kr7xl\" (UniqueName: \"kubernetes.io/projected/f86eae85-6b3a-4c69-a001-b186bed6dd84-kube-api-access-kr7xl\") pod \"payload-pre-processing-e2e-ipp-a-913351-5587d6644c-25q64\" (UID: \"f86eae85-6b3a-4c69-a001-b186bed6dd84\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-913351-5587d6644c-25q64"
Oct 30 19:48:31.891012 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:31.890968    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1956ab1-08ac-4c8c-ab41-91993c5ccd75-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-913351-64c79d9748-wzvj5\" (UID: \"a1956ab1-08ac-4c8c-ab41-91993c5ccd75\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-913351-64c79d9748-wzvj5"
Oct 30 19:48:31.891174 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:31.891123    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ld2x\" (UniqueName: \"kubernetes.io/projected/a1956ab1-08ac-4c8c-ab41-91993c5ccd75-kube-api-access-9ld2x\") pod \"payload-processing-e2e-ipp-a-913351-64c79d9748-wzvj5\" (UID: \"a1956ab1-08ac-4c8c-ab41-91993c5ccd75\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-913351-64c79d9748-wzvj5"
Oct 30 19:48:31.892008 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:31.891977    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1956ab1-08ac-4c8c-ab41-91993c5ccd75-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-913351-64c79d9748-wzvj5\" (UID: \"a1956ab1-08ac-4c8c-ab41-91993c5ccd75\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-913351-64c79d9748-wzvj5"
Oct 30 19:48:31.904487 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:31.904458    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ld2x\" (UniqueName: \"kubernetes.io/projected/a1956ab1-08ac-4c8c-ab41-91993c5ccd75-kube-api-access-9ld2x\") pod \"payload-processing-e2e-ipp-a-913351-64c79d9748-wzvj5\" (UID: \"a1956ab1-08ac-4c8c-ab41-91993c5ccd75\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-913351-64c79d9748-wzvj5"
Oct 30 19:48:31.906720 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:31.906692    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-913351-5587d6644c-25q64"
Oct 30 19:48:31.980223 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:31.980189    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-913351-64c79d9748-wzvj5"
Oct 30 19:48:32.221248 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:32.221146    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-913351-5587d6644c-25q64"]
Oct 30 19:48:32.234459 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:48:32.234422    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf86eae85_6b3a_4c69_a001_b186bed6dd84.slice/crio-44898a13a94abeb3cf98a0c7fa45b985074c1afe4f0793a7e54a19f35fa2063b WatchSource:0}: Error finding container 44898a13a94abeb3cf98a0c7fa45b985074c1afe4f0793a7e54a19f35fa2063b: Status 404 returned error can't find the container with id 44898a13a94abeb3cf98a0c7fa45b985074c1afe4f0793a7e54a19f35fa2063b
Oct 30 19:48:32.236288 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:32.236258    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-501ac8-768b76884b-8wrfj"]
Oct 30 19:48:32.250541 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:32.250514    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-501ac8-dd4dc7b8c-bnggs"]
Oct 30 19:48:32.317533 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:32.317461    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-913351-64c79d9748-wzvj5"]
Oct 30 19:48:32.320710 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:48:32.320680    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1956ab1_08ac_4c8c_ab41_91993c5ccd75.slice/crio-cb5b26e502635f77dc4924b777fa174fd7c63eb95efbe481fc28819d4224389e WatchSource:0}: Error finding container cb5b26e502635f77dc4924b777fa174fd7c63eb95efbe481fc28819d4224389e: Status 404 returned error can't find the container with id cb5b26e502635f77dc4924b777fa174fd7c63eb95efbe481fc28819d4224389e
Oct 30 19:48:32.599504 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:32.595778    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-477fd9c0-84cd565b7f-sggp9"]
Oct 30 19:48:33.239958 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:33.239919    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-913351-64c79d9748-wzvj5" event={"ID":"a1956ab1-08ac-4c8c-ab41-91993c5ccd75","Type":"ContainerStarted","Data":"6afbac3cb759c616366d5bcdd78f880a34caeee996d23fda8c84235a35b940bb"}
Oct 30 19:48:33.240578 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:33.240552    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-913351-64c79d9748-wzvj5" event={"ID":"a1956ab1-08ac-4c8c-ab41-91993c5ccd75","Type":"ContainerStarted","Data":"cb5b26e502635f77dc4924b777fa174fd7c63eb95efbe481fc28819d4224389e"}
Oct 30 19:48:33.241826 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:33.241802    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-913351-64c79d9748-wzvj5"
Oct 30 19:48:33.246605 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:33.246577    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-913351-5587d6644c-25q64" event={"ID":"f86eae85-6b3a-4c69-a001-b186bed6dd84","Type":"ContainerStarted","Data":"608de6f325cee9463578440d095199418de8f31b3bc6e846697a7bcb99577efe"}
Oct 30 19:48:33.246791 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:33.246770    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-913351-5587d6644c-25q64" event={"ID":"f86eae85-6b3a-4c69-a001-b186bed6dd84","Type":"ContainerStarted","Data":"44898a13a94abeb3cf98a0c7fa45b985074c1afe4f0793a7e54a19f35fa2063b"}
Oct 30 19:48:33.247739 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:33.247705    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-913351-5587d6644c-25q64"
Oct 30 19:48:33.274112 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:33.274058    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-913351-64c79d9748-wzvj5" podStartSLOduration=2.274038853 podStartE2EDuration="2.274038853s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:48:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:48:33.26800206 +0000 UTC m=+988.596120216" watchObservedRunningTime="2026-10-30 19:48:33.274038853 +0000 UTC m=+988.602157009"
Oct 30 19:48:37.260505 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:37.257539    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-501ac8-dd4dc7b8c-bnggs" podUID="26ef3405-8610-49bd-a79b-52279ca775b1" containerName="payload-pre-processing" containerID="cri-o://5d85770eb2346e590a05d3e0a7946e7a596e8c7ee944ed795e98f01f9a219d8f" gracePeriod=25
Oct 30 19:48:37.260505 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:37.258262    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-501ac8-768b76884b-8wrfj" podUID="af9d2807-c5e6-44a5-85d3-3ec0a5577602" containerName="payload-processing" containerID="cri-o://7d29cc1d8c6cbbd43b437fa4551dfbdebdff35be33492781cbe590925e5d6979" gracePeriod=25
Oct 30 19:48:37.603741 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:37.601604    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-477fd9c0-84cd565b7f-sggp9" podUID="6e6e973c-daab-4ccc-b552-ce2e718708a8" containerName="payload-pre-processing" containerID="cri-o://68814908d4a8411ecd5af3f354a6e65f86c9418d143e7ce045ac96fec337ae3d" gracePeriod=25
Oct 30 19:48:37.767774 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:37.767728    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-501ac8-dd4dc7b8c-bnggs"
Oct 30 19:48:37.775130 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:37.775103    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-501ac8-768b76884b-8wrfj"
Oct 30 19:48:37.798616 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:37.796740    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-913351-5587d6644c-25q64" podStartSLOduration=6.796722245 podStartE2EDuration="6.796722245s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:48:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:48:33.297958615 +0000 UTC m=+988.626076768" watchObservedRunningTime="2026-10-30 19:48:37.796722245 +0000 UTC m=+993.124840400"
Oct 30 19:48:37.883188 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:37.883103    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/af9d2807-c5e6-44a5-85d3-3ec0a5577602-kube-api-access-gw2s2\" (UniqueName: \"kubernetes.io/projected/af9d2807-c5e6-44a5-85d3-3ec0a5577602-kube-api-access-gw2s2\") pod \"af9d2807-c5e6-44a5-85d3-3ec0a5577602\" (UID: \"af9d2807-c5e6-44a5-85d3-3ec0a5577602\") "
Oct 30 19:48:37.883356 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:37.883235    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/26ef3405-8610-49bd-a79b-52279ca775b1-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26ef3405-8610-49bd-a79b-52279ca775b1-plugins-config-volume\") pod \"26ef3405-8610-49bd-a79b-52279ca775b1\" (UID: \"26ef3405-8610-49bd-a79b-52279ca775b1\") "
Oct 30 19:48:37.883729 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:37.883696    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26ef3405-8610-49bd-a79b-52279ca775b1-plugins-config-volume" pod "26ef3405-8610-49bd-a79b-52279ca775b1" (UID: "26ef3405-8610-49bd-a79b-52279ca775b1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:48:37.883841 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:37.883767    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/26ef3405-8610-49bd-a79b-52279ca775b1-kube-api-access-5w542\" (UniqueName: \"kubernetes.io/projected/26ef3405-8610-49bd-a79b-52279ca775b1-kube-api-access-5w542\") pod \"26ef3405-8610-49bd-a79b-52279ca775b1\" (UID: \"26ef3405-8610-49bd-a79b-52279ca775b1\") "
Oct 30 19:48:37.884672 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:37.883954    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/af9d2807-c5e6-44a5-85d3-3ec0a5577602-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af9d2807-c5e6-44a5-85d3-3ec0a5577602-plugins-config-volume\") pod \"af9d2807-c5e6-44a5-85d3-3ec0a5577602\" (UID: \"af9d2807-c5e6-44a5-85d3-3ec0a5577602\") "
Oct 30 19:48:37.885650 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:37.884876    6996 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26ef3405-8610-49bd-a79b-52279ca775b1-plugins-config-volume\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:37.885650 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:37.885211    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af9d2807-c5e6-44a5-85d3-3ec0a5577602-plugins-config-volume" pod "af9d2807-c5e6-44a5-85d3-3ec0a5577602" (UID: "af9d2807-c5e6-44a5-85d3-3ec0a5577602"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:48:37.888944 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:37.888906    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af9d2807-c5e6-44a5-85d3-3ec0a5577602-kube-api-access-gw2s2" pod "af9d2807-c5e6-44a5-85d3-3ec0a5577602" (UID: "af9d2807-c5e6-44a5-85d3-3ec0a5577602"). InnerVolumeSpecName "kube-api-access-gw2s2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:48:37.894525 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:37.890849    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26ef3405-8610-49bd-a79b-52279ca775b1-kube-api-access-5w542" pod "26ef3405-8610-49bd-a79b-52279ca775b1" (UID: "26ef3405-8610-49bd-a79b-52279ca775b1"). InnerVolumeSpecName "kube-api-access-5w542". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:48:37.986050 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:37.986012    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gw2s2\" (UniqueName: \"kubernetes.io/projected/af9d2807-c5e6-44a5-85d3-3ec0a5577602-kube-api-access-gw2s2\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:37.986050 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:37.986041    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5w542\" (UniqueName: \"kubernetes.io/projected/26ef3405-8610-49bd-a79b-52279ca775b1-kube-api-access-5w542\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:37.986050 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:37.986052    6996 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af9d2807-c5e6-44a5-85d3-3ec0a5577602-plugins-config-volume\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:37.990091 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:37.990070    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-477fd9c0-84cd565b7f-sggp9"
Oct 30 19:48:38.086994 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:38.086950    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6e6e973c-daab-4ccc-b552-ce2e718708a8-kube-api-access-n7xwf\" (UniqueName: \"kubernetes.io/projected/6e6e973c-daab-4ccc-b552-ce2e718708a8-kube-api-access-n7xwf\") pod \"6e6e973c-daab-4ccc-b552-ce2e718708a8\" (UID: \"6e6e973c-daab-4ccc-b552-ce2e718708a8\") "
Oct 30 19:48:38.087209 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:38.087193    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6e6e973c-daab-4ccc-b552-ce2e718708a8-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e6e973c-daab-4ccc-b552-ce2e718708a8-plugins-config-volume\") pod \"6e6e973c-daab-4ccc-b552-ce2e718708a8\" (UID: \"6e6e973c-daab-4ccc-b552-ce2e718708a8\") "
Oct 30 19:48:38.087899 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:38.087874    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e6e973c-daab-4ccc-b552-ce2e718708a8-plugins-config-volume" pod "6e6e973c-daab-4ccc-b552-ce2e718708a8" (UID: "6e6e973c-daab-4ccc-b552-ce2e718708a8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:48:38.089331 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:38.089311    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e6e973c-daab-4ccc-b552-ce2e718708a8-kube-api-access-n7xwf" pod "6e6e973c-daab-4ccc-b552-ce2e718708a8" (UID: "6e6e973c-daab-4ccc-b552-ce2e718708a8"). InnerVolumeSpecName "kube-api-access-n7xwf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:48:38.188776 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:38.188701    6996 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e6e973c-daab-4ccc-b552-ce2e718708a8-plugins-config-volume\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:38.188776 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:38.188727    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n7xwf\" (UniqueName: \"kubernetes.io/projected/6e6e973c-daab-4ccc-b552-ce2e718708a8-kube-api-access-n7xwf\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:38.311025 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:38.310990    6996 generic.go:356] "Generic (PLEG): container finished" podID="26ef3405-8610-49bd-a79b-52279ca775b1" containerID="5d85770eb2346e590a05d3e0a7946e7a596e8c7ee944ed795e98f01f9a219d8f" exitCode=0
Oct 30 19:48:38.311382 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:38.311127    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-501ac8-dd4dc7b8c-bnggs"
Oct 30 19:48:38.311622 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:38.311590    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-501ac8-dd4dc7b8c-bnggs" event={"ID":"26ef3405-8610-49bd-a79b-52279ca775b1","Type":"ContainerDied","Data":"5d85770eb2346e590a05d3e0a7946e7a596e8c7ee944ed795e98f01f9a219d8f"}
Oct 30 19:48:38.312320 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:38.312291    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-501ac8-dd4dc7b8c-bnggs" event={"ID":"26ef3405-8610-49bd-a79b-52279ca775b1","Type":"ContainerDied","Data":"4c66df7b7d563a68ac03ca4729084d466e27a63c48a75f56253eeaa6afbb793b"}
Oct 30 19:48:38.312407 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:38.312395    6996 scope.go:122] "RemoveContainer" containerID="5d85770eb2346e590a05d3e0a7946e7a596e8c7ee944ed795e98f01f9a219d8f"
Oct 30 19:48:38.313867 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:38.313624    6996 generic.go:356] "Generic (PLEG): container finished" podID="af9d2807-c5e6-44a5-85d3-3ec0a5577602" containerID="7d29cc1d8c6cbbd43b437fa4551dfbdebdff35be33492781cbe590925e5d6979" exitCode=0
Oct 30 19:48:38.313867 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:38.313707    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-501ac8-768b76884b-8wrfj" event={"ID":"af9d2807-c5e6-44a5-85d3-3ec0a5577602","Type":"ContainerDied","Data":"7d29cc1d8c6cbbd43b437fa4551dfbdebdff35be33492781cbe590925e5d6979"}
Oct 30 19:48:38.313867 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:38.313746    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-501ac8-768b76884b-8wrfj" event={"ID":"af9d2807-c5e6-44a5-85d3-3ec0a5577602","Type":"ContainerDied","Data":"30c5c134e4f863f6c4ecb41ad67f3683f27dcc6f8ec878a5622f61bb17f54ec5"}
Oct 30 19:48:38.313867 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:38.313811    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-501ac8-768b76884b-8wrfj"
Oct 30 19:48:38.315989 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:38.315960    6996 generic.go:356] "Generic (PLEG): container finished" podID="6e6e973c-daab-4ccc-b552-ce2e718708a8" containerID="68814908d4a8411ecd5af3f354a6e65f86c9418d143e7ce045ac96fec337ae3d" exitCode=0
Oct 30 19:48:38.316105 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:38.316004    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-477fd9c0-84cd565b7f-sggp9" event={"ID":"6e6e973c-daab-4ccc-b552-ce2e718708a8","Type":"ContainerDied","Data":"68814908d4a8411ecd5af3f354a6e65f86c9418d143e7ce045ac96fec337ae3d"}
Oct 30 19:48:38.316105 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:38.316040    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-477fd9c0-84cd565b7f-sggp9" event={"ID":"6e6e973c-daab-4ccc-b552-ce2e718708a8","Type":"ContainerDied","Data":"1bdd57500f71fe991c3e049e7d769dc205b31243604457f2fb55f5aad60184ab"}
Oct 30 19:48:38.316219 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:38.316111    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-477fd9c0-84cd565b7f-sggp9"
Oct 30 19:48:38.326937 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:38.326897    6996 scope.go:122] "RemoveContainer" containerID="5d85770eb2346e590a05d3e0a7946e7a596e8c7ee944ed795e98f01f9a219d8f"
Oct 30 19:48:38.327222 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:48:38.327193    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d85770eb2346e590a05d3e0a7946e7a596e8c7ee944ed795e98f01f9a219d8f\": container with ID starting with 5d85770eb2346e590a05d3e0a7946e7a596e8c7ee944ed795e98f01f9a219d8f not found: ID does not exist" containerID="5d85770eb2346e590a05d3e0a7946e7a596e8c7ee944ed795e98f01f9a219d8f"
Oct 30 19:48:38.327316 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:38.327231    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d85770eb2346e590a05d3e0a7946e7a596e8c7ee944ed795e98f01f9a219d8f"} err="failed to get container status \"5d85770eb2346e590a05d3e0a7946e7a596e8c7ee944ed795e98f01f9a219d8f\": rpc error: code = NotFound desc = could not find container \"5d85770eb2346e590a05d3e0a7946e7a596e8c7ee944ed795e98f01f9a219d8f\": container with ID starting with 5d85770eb2346e590a05d3e0a7946e7a596e8c7ee944ed795e98f01f9a219d8f not found: ID does not exist"
Oct 30 19:48:38.327316 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:38.327257    6996 scope.go:122] "RemoveContainer" containerID="7d29cc1d8c6cbbd43b437fa4551dfbdebdff35be33492781cbe590925e5d6979"
Oct 30 19:48:38.368553 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:38.368521    6996 scope.go:122] "RemoveContainer" containerID="7d29cc1d8c6cbbd43b437fa4551dfbdebdff35be33492781cbe590925e5d6979"
Oct 30 19:48:38.368940 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:38.368905    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-501ac8-dd4dc7b8c-bnggs"]
Oct 30 19:48:38.369063 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:48:38.369040    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d29cc1d8c6cbbd43b437fa4551dfbdebdff35be33492781cbe590925e5d6979\": container with ID starting with 7d29cc1d8c6cbbd43b437fa4551dfbdebdff35be33492781cbe590925e5d6979 not found: ID does not exist" containerID="7d29cc1d8c6cbbd43b437fa4551dfbdebdff35be33492781cbe590925e5d6979"
Oct 30 19:48:38.369133 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:38.369076    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d29cc1d8c6cbbd43b437fa4551dfbdebdff35be33492781cbe590925e5d6979"} err="failed to get container status \"7d29cc1d8c6cbbd43b437fa4551dfbdebdff35be33492781cbe590925e5d6979\": rpc error: code = NotFound desc = could not find container \"7d29cc1d8c6cbbd43b437fa4551dfbdebdff35be33492781cbe590925e5d6979\": container with ID starting with 7d29cc1d8c6cbbd43b437fa4551dfbdebdff35be33492781cbe590925e5d6979 not found: ID does not exist"
Oct 30 19:48:38.369133 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:38.369101    6996 scope.go:122] "RemoveContainer" containerID="68814908d4a8411ecd5af3f354a6e65f86c9418d143e7ce045ac96fec337ae3d"
Oct 30 19:48:38.374523 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:38.374502    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-501ac8-dd4dc7b8c-bnggs"]
Oct 30 19:48:38.381146 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:38.381129    6996 scope.go:122] "RemoveContainer" containerID="68814908d4a8411ecd5af3f354a6e65f86c9418d143e7ce045ac96fec337ae3d"
Oct 30 19:48:38.381548 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:48:38.381528    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68814908d4a8411ecd5af3f354a6e65f86c9418d143e7ce045ac96fec337ae3d\": container with ID starting with 68814908d4a8411ecd5af3f354a6e65f86c9418d143e7ce045ac96fec337ae3d not found: ID does not exist" containerID="68814908d4a8411ecd5af3f354a6e65f86c9418d143e7ce045ac96fec337ae3d"
Oct 30 19:48:38.381623 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:38.381556    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68814908d4a8411ecd5af3f354a6e65f86c9418d143e7ce045ac96fec337ae3d"} err="failed to get container status \"68814908d4a8411ecd5af3f354a6e65f86c9418d143e7ce045ac96fec337ae3d\": rpc error: code = NotFound desc = could not find container \"68814908d4a8411ecd5af3f354a6e65f86c9418d143e7ce045ac96fec337ae3d\": container with ID starting with 68814908d4a8411ecd5af3f354a6e65f86c9418d143e7ce045ac96fec337ae3d not found: ID does not exist"
Oct 30 19:48:38.397453 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:38.397428    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-501ac8-768b76884b-8wrfj"]
Oct 30 19:48:38.406382 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:38.406361    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-501ac8-768b76884b-8wrfj"]
Oct 30 19:48:38.433372 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:38.433350    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-477fd9c0-84cd565b7f-sggp9"]
Oct 30 19:48:38.448554 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:38.436584    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-477fd9c0-84cd565b7f-sggp9"]
Oct 30 19:48:38.563910 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:38.561893    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-913351-564dc8cdf7-cpcnr"]
Oct 30 19:48:38.563910 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:38.563180    6996 cpu_manager.go:414] "containerMap: removing container" podUID="6e6e973c-daab-4ccc-b552-ce2e718708a8" containerName="payload-pre-processing"
Oct 30 19:48:38.563910 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:38.563201    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6e6e973c-daab-4ccc-b552-ce2e718708a8" containerName="payload-pre-processing"
Oct 30 19:48:38.563910 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:38.563240    6996 cpu_manager.go:414] "containerMap: removing container" podUID="26ef3405-8610-49bd-a79b-52279ca775b1" containerName="payload-pre-processing"
Oct 30 19:48:38.563910 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:38.563250    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="26ef3405-8610-49bd-a79b-52279ca775b1" containerName="payload-pre-processing"
Oct 30 19:48:38.563910 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:38.563271    6996 cpu_manager.go:414] "containerMap: removing container" podUID="af9d2807-c5e6-44a5-85d3-3ec0a5577602" containerName="payload-processing"
Oct 30 19:48:38.563910 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:38.563279    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="af9d2807-c5e6-44a5-85d3-3ec0a5577602" containerName="payload-processing"
Oct 30 19:48:38.563910 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:38.563421    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="6e6e973c-daab-4ccc-b552-ce2e718708a8" containerName="payload-pre-processing"
Oct 30 19:48:38.563910 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:38.563440    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="26ef3405-8610-49bd-a79b-52279ca775b1" containerName="payload-pre-processing"
Oct 30 19:48:38.563910 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:38.563452    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="af9d2807-c5e6-44a5-85d3-3ec0a5577602" containerName="payload-processing"
Oct 30 19:48:38.584086 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:38.584036    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-913351-564dc8cdf7-cpcnr"
Oct 30 19:48:38.591290 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:38.591256    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-913351-564dc8cdf7-cpcnr"]
Oct 30 19:48:38.604183 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:38.604138    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-913351-serving-cert\""
Oct 30 19:48:38.709925 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:38.708993    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a440bbbf-403d-4f26-976a-b869c6df1635-metrics-certs\") pod \"maas-api-e2e-ipp-a-913351-564dc8cdf7-cpcnr\" (UID: \"a440bbbf-403d-4f26-976a-b869c6df1635\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-913351-564dc8cdf7-cpcnr"
Oct 30 19:48:38.709925 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:38.709145    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrpdk\" (UniqueName: \"kubernetes.io/projected/a440bbbf-403d-4f26-976a-b869c6df1635-kube-api-access-hrpdk\") pod \"maas-api-e2e-ipp-a-913351-564dc8cdf7-cpcnr\" (UID: \"a440bbbf-403d-4f26-976a-b869c6df1635\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-913351-564dc8cdf7-cpcnr"
Oct 30 19:48:38.709925 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:38.709270    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a440bbbf-403d-4f26-976a-b869c6df1635-maas-api-tls\") pod \"maas-api-e2e-ipp-a-913351-564dc8cdf7-cpcnr\" (UID: \"a440bbbf-403d-4f26-976a-b869c6df1635\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-913351-564dc8cdf7-cpcnr"
Oct 30 19:48:38.709925 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:38.709378    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a440bbbf-403d-4f26-976a-b869c6df1635-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-913351-564dc8cdf7-cpcnr\" (UID: \"a440bbbf-403d-4f26-976a-b869c6df1635\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-913351-564dc8cdf7-cpcnr"
Oct 30 19:48:38.794158 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:38.794121    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="26ef3405-8610-49bd-a79b-52279ca775b1" path="/var/lib/kubelet/pods/26ef3405-8610-49bd-a79b-52279ca775b1/volumes"
Oct 30 19:48:38.794719 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:38.794698    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6e6e973c-daab-4ccc-b552-ce2e718708a8" path="/var/lib/kubelet/pods/6e6e973c-daab-4ccc-b552-ce2e718708a8/volumes"
Oct 30 19:48:38.795176 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:38.795154    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="af9d2807-c5e6-44a5-85d3-3ec0a5577602" path="/var/lib/kubelet/pods/af9d2807-c5e6-44a5-85d3-3ec0a5577602/volumes"
Oct 30 19:48:38.810517 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:38.810491    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a440bbbf-403d-4f26-976a-b869c6df1635-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-913351-564dc8cdf7-cpcnr\" (UID: \"a440bbbf-403d-4f26-976a-b869c6df1635\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-913351-564dc8cdf7-cpcnr"
Oct 30 19:48:38.810666 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:38.810649    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a440bbbf-403d-4f26-976a-b869c6df1635-metrics-certs\") pod \"maas-api-e2e-ipp-a-913351-564dc8cdf7-cpcnr\" (UID: \"a440bbbf-403d-4f26-976a-b869c6df1635\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-913351-564dc8cdf7-cpcnr"
Oct 30 19:48:38.810742 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:38.810712    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrpdk\" (UniqueName: \"kubernetes.io/projected/a440bbbf-403d-4f26-976a-b869c6df1635-kube-api-access-hrpdk\") pod \"maas-api-e2e-ipp-a-913351-564dc8cdf7-cpcnr\" (UID: \"a440bbbf-403d-4f26-976a-b869c6df1635\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-913351-564dc8cdf7-cpcnr"
Oct 30 19:48:38.810809 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:38.810793    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a440bbbf-403d-4f26-976a-b869c6df1635-maas-api-tls\") pod \"maas-api-e2e-ipp-a-913351-564dc8cdf7-cpcnr\" (UID: \"a440bbbf-403d-4f26-976a-b869c6df1635\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-913351-564dc8cdf7-cpcnr"
Oct 30 19:48:38.810992 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:48:38.810965    6996 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-913351-serving-cert: secret "maas-api-e2e-ipp-a-913351-serving-cert" not found
Oct 30 19:48:38.811055 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:48:38.811042    6996 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a440bbbf-403d-4f26-976a-b869c6df1635-maas-api-tls podName:a440bbbf-403d-4f26-976a-b869c6df1635 nodeName:}" failed. No retries permitted until 2026-10-30 19:48:39.311022547 +0000 UTC m=+994.639140685 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a440bbbf-403d-4f26-976a-b869c6df1635-maas-api-tls") pod "maas-api-e2e-ipp-a-913351-564dc8cdf7-cpcnr" (UID: "a440bbbf-403d-4f26-976a-b869c6df1635") : secret "maas-api-e2e-ipp-a-913351-serving-cert" not found
Oct 30 19:48:38.811231 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:38.811213    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a440bbbf-403d-4f26-976a-b869c6df1635-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-913351-564dc8cdf7-cpcnr\" (UID: \"a440bbbf-403d-4f26-976a-b869c6df1635\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-913351-564dc8cdf7-cpcnr"
Oct 30 19:48:38.813344 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:38.813319    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a440bbbf-403d-4f26-976a-b869c6df1635-metrics-certs\") pod \"maas-api-e2e-ipp-a-913351-564dc8cdf7-cpcnr\" (UID: \"a440bbbf-403d-4f26-976a-b869c6df1635\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-913351-564dc8cdf7-cpcnr"
Oct 30 19:48:38.854609 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:38.854575    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrpdk\" (UniqueName: \"kubernetes.io/projected/a440bbbf-403d-4f26-976a-b869c6df1635-kube-api-access-hrpdk\") pod \"maas-api-e2e-ipp-a-913351-564dc8cdf7-cpcnr\" (UID: \"a440bbbf-403d-4f26-976a-b869c6df1635\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-913351-564dc8cdf7-cpcnr"
Oct 30 19:48:39.318996 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:39.318231    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a440bbbf-403d-4f26-976a-b869c6df1635-maas-api-tls\") pod \"maas-api-e2e-ipp-a-913351-564dc8cdf7-cpcnr\" (UID: \"a440bbbf-403d-4f26-976a-b869c6df1635\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-913351-564dc8cdf7-cpcnr"
Oct 30 19:48:39.322350 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:39.322294    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a440bbbf-403d-4f26-976a-b869c6df1635-maas-api-tls\") pod \"maas-api-e2e-ipp-a-913351-564dc8cdf7-cpcnr\" (UID: \"a440bbbf-403d-4f26-976a-b869c6df1635\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-913351-564dc8cdf7-cpcnr"
Oct 30 19:48:39.515993 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:39.515957    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-913351-564dc8cdf7-cpcnr"
Oct 30 19:48:39.802145 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:39.802117    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-913351-564dc8cdf7-cpcnr"]
Oct 30 19:48:39.805247 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:48:39.805218    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda440bbbf_403d_4f26_976a_b869c6df1635.slice/crio-9c8d3313287ca880fcfe15d1de0ed7409856678e63d246423afc61fe3e063bcc WatchSource:0}: Error finding container 9c8d3313287ca880fcfe15d1de0ed7409856678e63d246423afc61fe3e063bcc: Status 404 returned error can't find the container with id 9c8d3313287ca880fcfe15d1de0ed7409856678e63d246423afc61fe3e063bcc
Oct 30 19:48:40.345891 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:40.345856    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-913351-564dc8cdf7-cpcnr" event={"ID":"a440bbbf-403d-4f26-976a-b869c6df1635","Type":"ContainerStarted","Data":"9c8d3313287ca880fcfe15d1de0ed7409856678e63d246423afc61fe3e063bcc"}
Oct 30 19:48:41.355654 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:41.355613    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-913351-564dc8cdf7-cpcnr" event={"ID":"a440bbbf-403d-4f26-976a-b869c6df1635","Type":"ContainerStarted","Data":"ca017c2bdf5d993f0d00de6a51efb2721e304b101c0d6355646e4ef3e42d63f1"}
Oct 30 19:48:41.356143 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:41.355694    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-913351-564dc8cdf7-cpcnr"
Oct 30 19:48:41.377325 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:41.377266    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-913351-564dc8cdf7-cpcnr" podStartSLOduration=2.882434322 podStartE2EDuration="3.377250322s" totalImagesPullingTime="494.816ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:48:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:48:41.375458366 +0000 UTC m=+996.703576519" watchObservedRunningTime="2026-10-30 19:48:41.377250322 +0000 UTC m=+996.705368476"
Oct 30 19:48:43.236614 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:43.236583    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 19:48:43.372114 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:43.372083    6996 generic.go:356] "Generic (PLEG): container finished" podID="e5aaa21a-1a12-4b2b-b05e-f08504685cd8" containerID="2607a1ffd03e61d227ab1169ca7c998fceec6b47285852a37afcafe1e231c02c" exitCode=0
Oct 30 19:48:43.372296 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:43.372157    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r" event={"ID":"e5aaa21a-1a12-4b2b-b05e-f08504685cd8","Type":"ContainerDied","Data":"2607a1ffd03e61d227ab1169ca7c998fceec6b47285852a37afcafe1e231c02c"}
Oct 30 19:48:43.372661 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:43.372642    6996 scope.go:122] "RemoveContainer" containerID="2607a1ffd03e61d227ab1169ca7c998fceec6b47285852a37afcafe1e231c02c"
Oct 30 19:48:43.374165 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:43.374134    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:48:43.374261 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:43.374222    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:48:43.374329 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:43.374272    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:48:43.417127 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:43.417102    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r"
Oct 30 19:48:44.420455 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:44.419309    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r" event={"ID":"e5aaa21a-1a12-4b2b-b05e-f08504685cd8","Type":"ContainerStarted","Data":"f32b4fd714143e8bcef46c0c51985fe8b9b2765237d4cf55c5259b516c9af770"}
Oct 30 19:48:45.010445 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:45.010417    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r"
Oct 30 19:48:45.011040 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:45.011021    6996 patch_prober.go:29] interesting pod/e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused" start-of-body=
Oct 30 19:48:45.011181 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:45.011165    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r" podUID="e5aaa21a-1a12-4b2b-b05e-f08504685cd8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused"
Oct 30 19:48:45.298152 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:45.297932    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-913351-5587d6644c-25q64"
Oct 30 19:48:45.298455 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:45.298437    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-913351-64c79d9748-wzvj5"
Oct 30 19:48:46.058370 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:46.028778    6996 patch_prober.go:29] interesting pod/e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused" start-of-body=
Oct 30 19:48:46.058370 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:46.028870    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r" podUID="e5aaa21a-1a12-4b2b-b05e-f08504685cd8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused"
Oct 30 19:48:47.032499 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:47.032223    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-501ac8-5c4f9d85bb-pvmxr"]
Oct 30 19:48:47.085501 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:47.032470    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-501ac8-5c4f9d85bb-pvmxr" podUID="ce28e93b-5db7-448f-8c7b-445428b81a7d" containerName="maas-api" containerID="cri-o://825ed48e3209dd1468a29559b4ad593ffc42f3a76f3ad96053d4773915e2a786" gracePeriod=30
Oct 30 19:48:47.124512 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:47.110635    6996 patch_prober.go:29] interesting pod/e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused" start-of-body=
Oct 30 19:48:47.157647 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:47.157601    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r" podUID="e5aaa21a-1a12-4b2b-b05e-f08504685cd8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused"
Oct 30 19:48:47.247207 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:47.247155    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq" podUID="89009638-003d-4a5d-ab13-1f8c8f64d3b7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.88:8000/ready\": context deadline exceeded"
Oct 30 19:48:47.247675 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:47.247453    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s" podUID="ae905dfb-f30d-480a-899b-ea676abc1d8d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:48:47.273761 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:47.247858    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr" podUID="0e470010-625d-443f-81ec-9bf965988212" containerName="main" probeResult="failure" output="Get \"https://10.134.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:48:47.339579 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:47.275463    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b" podUID="19fe4fba-09ae-4620-b4d2-0ef8c2463d71" containerName="main" probeResult="failure" output="Get \"https://10.134.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:48:47.517500 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:47.516713    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-913351-564dc8cdf7-cpcnr"
Oct 30 19:48:47.902855 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:47.902810    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk" podUID="a1857d2d-3b40-4e63-be33-be4ba476f926" containerName="main" probeResult="failure" output="Get \"https://10.134.0.90:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:48:48.032247 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:48.032193    6996 patch_prober.go:29] interesting pod/e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused" start-of-body=
Oct 30 19:48:48.032574 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:48.032544    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r" podUID="e5aaa21a-1a12-4b2b-b05e-f08504685cd8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused"
Oct 30 19:48:48.141561 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:48.137618    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-501ac8-5c4f9d85bb-pvmxr" podUID="ce28e93b-5db7-448f-8c7b-445428b81a7d" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.102:8443/health\": dial tcp 10.134.0.102:8443: connect: connection refused"
Oct 30 19:48:48.287505 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:48.286578    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b" podUID="19fe4fba-09ae-4620-b4d2-0ef8c2463d71" containerName="main" probeResult="failure" output="Get \"https://10.134.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:48:48.288508 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:48.288067    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq" podUID="89009638-003d-4a5d-ab13-1f8c8f64d3b7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.88:8000/ready\": context deadline exceeded"
Oct 30 19:48:48.293500 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:48.290285    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s" podUID="ae905dfb-f30d-480a-899b-ea676abc1d8d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.73:8000/ready\": context deadline exceeded"
Oct 30 19:48:48.293500 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:48.291378    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr" podUID="0e470010-625d-443f-81ec-9bf965988212" containerName="main" probeResult="failure" output="Get \"https://10.134.0.72:8000/ready\": context deadline exceeded"
Oct 30 19:48:48.293500 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:48.292703    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b"
Oct 30 19:48:48.293500 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:48.292764    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq"
Oct 30 19:48:48.293500 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:48.292803    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s"
Oct 30 19:48:48.293500 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:48.292840    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr"
Oct 30 19:48:48.895945 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:48.895826    6996 generic.go:356] "Generic (PLEG): container finished" podID="ce28e93b-5db7-448f-8c7b-445428b81a7d" containerID="825ed48e3209dd1468a29559b4ad593ffc42f3a76f3ad96053d4773915e2a786" exitCode=0
Oct 30 19:48:48.897576 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:48.897551    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b"
Oct 30 19:48:48.897796 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:48.897777    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-501ac8-5c4f9d85bb-pvmxr" event={"ID":"ce28e93b-5db7-448f-8c7b-445428b81a7d","Type":"ContainerDied","Data":"825ed48e3209dd1468a29559b4ad593ffc42f3a76f3ad96053d4773915e2a786"}
Oct 30 19:48:48.897943 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:48.897931    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr"
Oct 30 19:48:48.898057 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:48.898048    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s"
Oct 30 19:48:48.898154 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:48.898145    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq"
Oct 30 19:48:48.900975 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:48.900877    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-501ac8-5c4f9d85bb-pvmxr"
Oct 30 19:48:49.009679 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:49.009647    6996 patch_prober.go:29] interesting pod/e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused" start-of-body=
Oct 30 19:48:49.009855 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:49.009706    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r" podUID="e5aaa21a-1a12-4b2b-b05e-f08504685cd8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused"
Oct 30 19:48:49.031759 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:49.031729    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ce28e93b-5db7-448f-8c7b-445428b81a7d-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce28e93b-5db7-448f-8c7b-445428b81a7d-maas-api-tls\") pod \"ce28e93b-5db7-448f-8c7b-445428b81a7d\" (UID: \"ce28e93b-5db7-448f-8c7b-445428b81a7d\") "
Oct 30 19:48:49.031759 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:49.031765    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ce28e93b-5db7-448f-8c7b-445428b81a7d-metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce28e93b-5db7-448f-8c7b-445428b81a7d-metrics-certs\") pod \"ce28e93b-5db7-448f-8c7b-445428b81a7d\" (UID: \"ce28e93b-5db7-448f-8c7b-445428b81a7d\") "
Oct 30 19:48:49.031981 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:49.031965    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ce28e93b-5db7-448f-8c7b-445428b81a7d-kube-api-access-9z989\" (UniqueName: \"kubernetes.io/projected/ce28e93b-5db7-448f-8c7b-445428b81a7d-kube-api-access-9z989\") pod \"ce28e93b-5db7-448f-8c7b-445428b81a7d\" (UID: \"ce28e93b-5db7-448f-8c7b-445428b81a7d\") "
Oct 30 19:48:49.032162 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:49.032141    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ce28e93b-5db7-448f-8c7b-445428b81a7d-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce28e93b-5db7-448f-8c7b-445428b81a7d-openshift-service-ca\") pod \"ce28e93b-5db7-448f-8c7b-445428b81a7d\" (UID: \"ce28e93b-5db7-448f-8c7b-445428b81a7d\") "
Oct 30 19:48:49.034049 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:49.032644    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce28e93b-5db7-448f-8c7b-445428b81a7d-openshift-service-ca" pod "ce28e93b-5db7-448f-8c7b-445428b81a7d" (UID: "ce28e93b-5db7-448f-8c7b-445428b81a7d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:48:49.034049 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:49.033115    6996 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce28e93b-5db7-448f-8c7b-445428b81a7d-openshift-service-ca\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:49.034049 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:49.033848    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce28e93b-5db7-448f-8c7b-445428b81a7d-maas-api-tls" pod "ce28e93b-5db7-448f-8c7b-445428b81a7d" (UID: "ce28e93b-5db7-448f-8c7b-445428b81a7d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:48:49.035280 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:49.035258    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce28e93b-5db7-448f-8c7b-445428b81a7d-metrics-certs" pod "ce28e93b-5db7-448f-8c7b-445428b81a7d" (UID: "ce28e93b-5db7-448f-8c7b-445428b81a7d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:48:49.035613 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:49.035585    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce28e93b-5db7-448f-8c7b-445428b81a7d-kube-api-access-9z989" pod "ce28e93b-5db7-448f-8c7b-445428b81a7d" (UID: "ce28e93b-5db7-448f-8c7b-445428b81a7d"). InnerVolumeSpecName "kube-api-access-9z989". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:48:49.135658 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:49.135628    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9z989\" (UniqueName: \"kubernetes.io/projected/ce28e93b-5db7-448f-8c7b-445428b81a7d-kube-api-access-9z989\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:49.135658 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:49.135655    6996 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce28e93b-5db7-448f-8c7b-445428b81a7d-maas-api-tls\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:49.135850 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:49.135667    6996 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce28e93b-5db7-448f-8c7b-445428b81a7d-metrics-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:49.907079 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:49.907040    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-501ac8-5c4f9d85bb-pvmxr" event={"ID":"ce28e93b-5db7-448f-8c7b-445428b81a7d","Type":"ContainerDied","Data":"72ca4c96f693c72d33ba85d6d6ce208be2fa1af4612b950ec96205a658154ae6"}
Oct 30 19:48:49.907527 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:49.907086    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-501ac8-5c4f9d85bb-pvmxr"
Oct 30 19:48:49.907527 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:49.907111    6996 scope.go:122] "RemoveContainer" containerID="825ed48e3209dd1468a29559b4ad593ffc42f3a76f3ad96053d4773915e2a786"
Oct 30 19:48:49.936314 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:49.936287    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-501ac8-5c4f9d85bb-pvmxr"]
Oct 30 19:48:49.964171 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:49.964136    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-501ac8-5c4f9d85bb-pvmxr"]
Oct 30 19:48:50.009780 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:50.009748    6996 patch_prober.go:29] interesting pod/e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused" start-of-body=
Oct 30 19:48:50.009969 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:50.009816    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r" podUID="e5aaa21a-1a12-4b2b-b05e-f08504685cd8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused"
Oct 30 19:48:50.793596 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:50.793552    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ce28e93b-5db7-448f-8c7b-445428b81a7d" path="/var/lib/kubelet/pods/ce28e93b-5db7-448f-8c7b-445428b81a7d/volumes"
Oct 30 19:48:51.014183 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:51.014152    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r"
Oct 30 19:48:51.014700 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:51.014674    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r"
Oct 30 19:48:51.015572 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:51.015551    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r"
Oct 30 19:48:52.997447 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:52.997413    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-c9d69c-fffd76955-lkr9q"]
Oct 30 19:48:52.998134 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:52.998118    6996 cpu_manager.go:414] "containerMap: removing container" podUID="ce28e93b-5db7-448f-8c7b-445428b81a7d" containerName="maas-api"
Oct 30 19:48:52.998181 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:52.998134    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ce28e93b-5db7-448f-8c7b-445428b81a7d" containerName="maas-api"
Oct 30 19:48:52.998269 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:52.998259    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="ce28e93b-5db7-448f-8c7b-445428b81a7d" containerName="maas-api"
Oct 30 19:48:53.001874 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:53.001856    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c9d69c-fffd76955-lkr9q"
Oct 30 19:48:53.007496 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:53.007462    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-c9d69c-dockercfg-xtjtf\""
Oct 30 19:48:53.008773 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:53.008744    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-c9d69c\""
Oct 30 19:48:53.013025 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:53.012999    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-c9d69c-fffd76955-lkr9q"]
Oct 30 19:48:53.031266 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:53.031235    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-c9d69c-844759994d-x4kmk"]
Oct 30 19:48:53.035543 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:53.035523    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-c9d69c-844759994d-x4kmk"
Oct 30 19:48:53.048442 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:53.048414    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-c9d69c-844759994d-x4kmk"]
Oct 30 19:48:53.141958 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:53.141922    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7601e4d-cc98-4f1d-8e46-331748368032-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-c9d69c-fffd76955-lkr9q\" (UID: \"b7601e4d-cc98-4f1d-8e46-331748368032\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c9d69c-fffd76955-lkr9q"
Oct 30 19:48:53.142654 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:53.142628    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8736332-22bc-48e0-a80e-9606ae27549a-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-c9d69c-844759994d-x4kmk\" (UID: \"a8736332-22bc-48e0-a80e-9606ae27549a\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-c9d69c-844759994d-x4kmk"
Oct 30 19:48:53.143098 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:53.143076    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxvg7\" (UniqueName: \"kubernetes.io/projected/a8736332-22bc-48e0-a80e-9606ae27549a-kube-api-access-nxvg7\") pod \"payload-processing-e2e-ipp-a-c9d69c-844759994d-x4kmk\" (UID: \"a8736332-22bc-48e0-a80e-9606ae27549a\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-c9d69c-844759994d-x4kmk"
Oct 30 19:48:53.143266 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:53.143248    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5klcj\" (UniqueName: \"kubernetes.io/projected/b7601e4d-cc98-4f1d-8e46-331748368032-kube-api-access-5klcj\") pod \"payload-pre-processing-e2e-ipp-a-c9d69c-fffd76955-lkr9q\" (UID: \"b7601e4d-cc98-4f1d-8e46-331748368032\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c9d69c-fffd76955-lkr9q"
Oct 30 19:48:53.247023 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:53.246961    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5klcj\" (UniqueName: \"kubernetes.io/projected/b7601e4d-cc98-4f1d-8e46-331748368032-kube-api-access-5klcj\") pod \"payload-pre-processing-e2e-ipp-a-c9d69c-fffd76955-lkr9q\" (UID: \"b7601e4d-cc98-4f1d-8e46-331748368032\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c9d69c-fffd76955-lkr9q"
Oct 30 19:48:53.247233 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:53.247143    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7601e4d-cc98-4f1d-8e46-331748368032-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-c9d69c-fffd76955-lkr9q\" (UID: \"b7601e4d-cc98-4f1d-8e46-331748368032\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c9d69c-fffd76955-lkr9q"
Oct 30 19:48:53.247233 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:53.247206    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8736332-22bc-48e0-a80e-9606ae27549a-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-c9d69c-844759994d-x4kmk\" (UID: \"a8736332-22bc-48e0-a80e-9606ae27549a\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-c9d69c-844759994d-x4kmk"
Oct 30 19:48:53.247350 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:53.247319    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxvg7\" (UniqueName: \"kubernetes.io/projected/a8736332-22bc-48e0-a80e-9606ae27549a-kube-api-access-nxvg7\") pod \"payload-processing-e2e-ipp-a-c9d69c-844759994d-x4kmk\" (UID: \"a8736332-22bc-48e0-a80e-9606ae27549a\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-c9d69c-844759994d-x4kmk"
Oct 30 19:48:53.249247 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:53.249157    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7601e4d-cc98-4f1d-8e46-331748368032-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-c9d69c-fffd76955-lkr9q\" (UID: \"b7601e4d-cc98-4f1d-8e46-331748368032\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c9d69c-fffd76955-lkr9q"
Oct 30 19:48:53.251714 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:53.250244    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8736332-22bc-48e0-a80e-9606ae27549a-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-c9d69c-844759994d-x4kmk\" (UID: \"a8736332-22bc-48e0-a80e-9606ae27549a\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-c9d69c-844759994d-x4kmk"
Oct 30 19:48:53.264767 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:53.264706    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxvg7\" (UniqueName: \"kubernetes.io/projected/a8736332-22bc-48e0-a80e-9606ae27549a-kube-api-access-nxvg7\") pod \"payload-processing-e2e-ipp-a-c9d69c-844759994d-x4kmk\" (UID: \"a8736332-22bc-48e0-a80e-9606ae27549a\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-c9d69c-844759994d-x4kmk"
Oct 30 19:48:53.272086 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:53.272022    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5klcj\" (UniqueName: \"kubernetes.io/projected/b7601e4d-cc98-4f1d-8e46-331748368032-kube-api-access-5klcj\") pod \"payload-pre-processing-e2e-ipp-a-c9d69c-fffd76955-lkr9q\" (UID: \"b7601e4d-cc98-4f1d-8e46-331748368032\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c9d69c-fffd76955-lkr9q"
Oct 30 19:48:53.318550 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:53.318502    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c9d69c-fffd76955-lkr9q"
Oct 30 19:48:53.353935 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:53.353905    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-c9d69c-844759994d-x4kmk"
Oct 30 19:48:53.612674 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:53.612645    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-c9d69c-fffd76955-lkr9q"]
Oct 30 19:48:53.613677 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:48:53.613650    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7601e4d_cc98_4f1d_8e46_331748368032.slice/crio-2cc2df9897ea48b8944ad1c0af2ca3fe2192efc5ef81bc5724f28fe1e5911356 WatchSource:0}: Error finding container 2cc2df9897ea48b8944ad1c0af2ca3fe2192efc5ef81bc5724f28fe1e5911356: Status 404 returned error can't find the container with id 2cc2df9897ea48b8944ad1c0af2ca3fe2192efc5ef81bc5724f28fe1e5911356
Oct 30 19:48:53.660351 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:53.660319    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-c9d69c-844759994d-x4kmk"]
Oct 30 19:48:53.660920 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:48:53.660892    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8736332_22bc_48e0_a80e_9606ae27549a.slice/crio-399671e18917ba09f327cb4984e8910472886501d73746a30a253dd66b00d5f7 WatchSource:0}: Error finding container 399671e18917ba09f327cb4984e8910472886501d73746a30a253dd66b00d5f7: Status 404 returned error can't find the container with id 399671e18917ba09f327cb4984e8910472886501d73746a30a253dd66b00d5f7
Oct 30 19:48:53.974433 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:53.974333    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c9d69c-fffd76955-lkr9q" event={"ID":"b7601e4d-cc98-4f1d-8e46-331748368032","Type":"ContainerStarted","Data":"e4f9013ea1f2b8d853b3b69b9c7dc44e99b8eec9f49cabde3867d146f25b26bd"}
Oct 30 19:48:53.974433 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:53.974400    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c9d69c-fffd76955-lkr9q" event={"ID":"b7601e4d-cc98-4f1d-8e46-331748368032","Type":"ContainerStarted","Data":"2cc2df9897ea48b8944ad1c0af2ca3fe2192efc5ef81bc5724f28fe1e5911356"}
Oct 30 19:48:53.974433 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:53.974434    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c9d69c-fffd76955-lkr9q"
Oct 30 19:48:53.976398 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:53.976368    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-c9d69c-844759994d-x4kmk" event={"ID":"a8736332-22bc-48e0-a80e-9606ae27549a","Type":"ContainerStarted","Data":"b6e56051300854b57751a9cd61b06a08108d3e047d59202551156209b71870f3"}
Oct 30 19:48:53.976594 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:53.976430    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-c9d69c-844759994d-x4kmk"
Oct 30 19:48:53.976594 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:53.976455    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-c9d69c-844759994d-x4kmk" event={"ID":"a8736332-22bc-48e0-a80e-9606ae27549a","Type":"ContainerStarted","Data":"399671e18917ba09f327cb4984e8910472886501d73746a30a253dd66b00d5f7"}
Oct 30 19:48:53.994193 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:53.994146    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c9d69c-fffd76955-lkr9q" podStartSLOduration=1.994134106 podStartE2EDuration="1.994134106s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:48:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:48:53.991899674 +0000 UTC m=+1009.320017828" watchObservedRunningTime="2026-10-30 19:48:53.994134106 +0000 UTC m=+1009.322252260"
Oct 30 19:48:54.011365 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:54.011319    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-c9d69c-844759994d-x4kmk" podStartSLOduration=1.011303376 podStartE2EDuration="1.011303376s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:48:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:48:54.009768436 +0000 UTC m=+1009.337886589" watchObservedRunningTime="2026-10-30 19:48:54.011303376 +0000 UTC m=+1009.339421529"
Oct 30 19:48:55.388661 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:55.388625    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-913351-64c79d9748-wzvj5"]
Oct 30 19:48:55.399166 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:55.399135    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-913351-5587d6644c-25q64"]
Oct 30 19:48:56.851627 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:56.851590    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7"]
Oct 30 19:48:56.852101 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:56.851879    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7" podUID="a61de104-8144-4107-a532-61cba1cf53e1" containerName="istio-proxy" containerID="cri-o://f5a0a42afdb4b9a595e1d4f596fd0061d8c85f4a9d6186e9108bc96b5e05f9c6" gracePeriod=30
Oct 30 19:48:59.273039 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:59.273005    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-c9d69c-6d96fc497c-p25r9"]
Oct 30 19:48:59.278920 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:59.278894    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c9d69c-6d96fc497c-p25r9"
Oct 30 19:48:59.287452 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:59.287427    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-c9d69c-serving-cert\""
Oct 30 19:48:59.294052 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:59.294028    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-c9d69c-6d96fc497c-p25r9"]
Oct 30 19:48:59.449958 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:59.449925    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e65ecbf-2aba-4a2c-abfa-c5d6c789152a-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-c9d69c-6d96fc497c-p25r9\" (UID: \"5e65ecbf-2aba-4a2c-abfa-c5d6c789152a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c9d69c-6d96fc497c-p25r9"
Oct 30 19:48:59.450116 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:59.449972    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5e65ecbf-2aba-4a2c-abfa-c5d6c789152a-maas-api-tls\") pod \"maas-api-e2e-ipp-a-c9d69c-6d96fc497c-p25r9\" (UID: \"5e65ecbf-2aba-4a2c-abfa-c5d6c789152a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c9d69c-6d96fc497c-p25r9"
Oct 30 19:48:59.450116 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:59.450077    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zcts9\" (UniqueName: \"kubernetes.io/projected/5e65ecbf-2aba-4a2c-abfa-c5d6c789152a-kube-api-access-zcts9\") pod \"maas-api-e2e-ipp-a-c9d69c-6d96fc497c-p25r9\" (UID: \"5e65ecbf-2aba-4a2c-abfa-c5d6c789152a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c9d69c-6d96fc497c-p25r9"
Oct 30 19:48:59.450240 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:59.450219    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e65ecbf-2aba-4a2c-abfa-c5d6c789152a-metrics-certs\") pod \"maas-api-e2e-ipp-a-c9d69c-6d96fc497c-p25r9\" (UID: \"5e65ecbf-2aba-4a2c-abfa-c5d6c789152a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c9d69c-6d96fc497c-p25r9"
Oct 30 19:48:59.551828 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:59.551743    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e65ecbf-2aba-4a2c-abfa-c5d6c789152a-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-c9d69c-6d96fc497c-p25r9\" (UID: \"5e65ecbf-2aba-4a2c-abfa-c5d6c789152a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c9d69c-6d96fc497c-p25r9"
Oct 30 19:48:59.551828 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:59.551805    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5e65ecbf-2aba-4a2c-abfa-c5d6c789152a-maas-api-tls\") pod \"maas-api-e2e-ipp-a-c9d69c-6d96fc497c-p25r9\" (UID: \"5e65ecbf-2aba-4a2c-abfa-c5d6c789152a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c9d69c-6d96fc497c-p25r9"
Oct 30 19:48:59.552056 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:59.551870    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcts9\" (UniqueName: \"kubernetes.io/projected/5e65ecbf-2aba-4a2c-abfa-c5d6c789152a-kube-api-access-zcts9\") pod \"maas-api-e2e-ipp-a-c9d69c-6d96fc497c-p25r9\" (UID: \"5e65ecbf-2aba-4a2c-abfa-c5d6c789152a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c9d69c-6d96fc497c-p25r9"
Oct 30 19:48:59.552113 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:59.552058    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e65ecbf-2aba-4a2c-abfa-c5d6c789152a-metrics-certs\") pod \"maas-api-e2e-ipp-a-c9d69c-6d96fc497c-p25r9\" (UID: \"5e65ecbf-2aba-4a2c-abfa-c5d6c789152a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c9d69c-6d96fc497c-p25r9"
Oct 30 19:48:59.552612 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:59.552578    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e65ecbf-2aba-4a2c-abfa-c5d6c789152a-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-c9d69c-6d96fc497c-p25r9\" (UID: \"5e65ecbf-2aba-4a2c-abfa-c5d6c789152a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c9d69c-6d96fc497c-p25r9"
Oct 30 19:48:59.554391 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:59.554370    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5e65ecbf-2aba-4a2c-abfa-c5d6c789152a-maas-api-tls\") pod \"maas-api-e2e-ipp-a-c9d69c-6d96fc497c-p25r9\" (UID: \"5e65ecbf-2aba-4a2c-abfa-c5d6c789152a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c9d69c-6d96fc497c-p25r9"
Oct 30 19:48:59.554642 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:59.554625    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e65ecbf-2aba-4a2c-abfa-c5d6c789152a-metrics-certs\") pod \"maas-api-e2e-ipp-a-c9d69c-6d96fc497c-p25r9\" (UID: \"5e65ecbf-2aba-4a2c-abfa-c5d6c789152a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c9d69c-6d96fc497c-p25r9"
Oct 30 19:48:59.563367 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:59.563347    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcts9\" (UniqueName: \"kubernetes.io/projected/5e65ecbf-2aba-4a2c-abfa-c5d6c789152a-kube-api-access-zcts9\") pod \"maas-api-e2e-ipp-a-c9d69c-6d96fc497c-p25r9\" (UID: \"5e65ecbf-2aba-4a2c-abfa-c5d6c789152a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c9d69c-6d96fc497c-p25r9"
Oct 30 19:48:59.601567 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:59.601534    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c9d69c-6d96fc497c-p25r9"
Oct 30 19:48:59.891741 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:48:59.891708    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-c9d69c-6d96fc497c-p25r9"]
Oct 30 19:48:59.892864 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:48:59.892835    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e65ecbf_2aba_4a2c_abfa_c5d6c789152a.slice/crio-b8656cca4871fd974ef128d31485c1223282f22a67427e6a2729dbda9cce04e8 WatchSource:0}: Error finding container b8656cca4871fd974ef128d31485c1223282f22a67427e6a2729dbda9cce04e8: Status 404 returned error can't find the container with id b8656cca4871fd974ef128d31485c1223282f22a67427e6a2729dbda9cce04e8
Oct 30 19:49:00.052434 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:00.052400    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c9d69c-6d96fc497c-p25r9" event={"ID":"5e65ecbf-2aba-4a2c-abfa-c5d6c789152a","Type":"ContainerStarted","Data":"b8656cca4871fd974ef128d31485c1223282f22a67427e6a2729dbda9cce04e8"}
Oct 30 19:49:00.389030 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:00.388888    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-913351-64c79d9748-wzvj5" podUID="a1956ab1-08ac-4c8c-ab41-91993c5ccd75" containerName="payload-processing" containerID="cri-o://6afbac3cb759c616366d5bcdd78f880a34caeee996d23fda8c84235a35b940bb" gracePeriod=25
Oct 30 19:49:00.399711 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:00.399658    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-913351-5587d6644c-25q64" podUID="f86eae85-6b3a-4c69-a001-b186bed6dd84" containerName="payload-pre-processing" containerID="cri-o://608de6f325cee9463578440d095199418de8f31b3bc6e846697a7bcb99577efe" gracePeriod=25
Oct 30 19:49:00.808381 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:00.808348    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-913351-64c79d9748-wzvj5"
Oct 30 19:49:00.869819 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:00.869459    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-913351-5587d6644c-25q64"
Oct 30 19:49:01.003167 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:01.003070    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a1956ab1-08ac-4c8c-ab41-91993c5ccd75-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1956ab1-08ac-4c8c-ab41-91993c5ccd75-plugins-config-volume\") pod \"a1956ab1-08ac-4c8c-ab41-91993c5ccd75\" (UID: \"a1956ab1-08ac-4c8c-ab41-91993c5ccd75\") "
Oct 30 19:49:01.003362 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:01.003235    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f86eae85-6b3a-4c69-a001-b186bed6dd84-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f86eae85-6b3a-4c69-a001-b186bed6dd84-plugins-config-volume\") pod \"f86eae85-6b3a-4c69-a001-b186bed6dd84\" (UID: \"f86eae85-6b3a-4c69-a001-b186bed6dd84\") "
Oct 30 19:49:01.003465 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:01.003426    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1956ab1-08ac-4c8c-ab41-91993c5ccd75-plugins-config-volume" pod "a1956ab1-08ac-4c8c-ab41-91993c5ccd75" (UID: "a1956ab1-08ac-4c8c-ab41-91993c5ccd75"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:49:01.003600 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:01.003437    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f86eae85-6b3a-4c69-a001-b186bed6dd84-kube-api-access-kr7xl\" (UniqueName: \"kubernetes.io/projected/f86eae85-6b3a-4c69-a001-b186bed6dd84-kube-api-access-kr7xl\") pod \"f86eae85-6b3a-4c69-a001-b186bed6dd84\" (UID: \"f86eae85-6b3a-4c69-a001-b186bed6dd84\") "
Oct 30 19:49:01.003672 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:01.003656    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f86eae85-6b3a-4c69-a001-b186bed6dd84-plugins-config-volume" pod "f86eae85-6b3a-4c69-a001-b186bed6dd84" (UID: "f86eae85-6b3a-4c69-a001-b186bed6dd84"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:49:01.003732 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:01.003673    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a1956ab1-08ac-4c8c-ab41-91993c5ccd75-kube-api-access-9ld2x\" (UniqueName: \"kubernetes.io/projected/a1956ab1-08ac-4c8c-ab41-91993c5ccd75-kube-api-access-9ld2x\") pod \"a1956ab1-08ac-4c8c-ab41-91993c5ccd75\" (UID: \"a1956ab1-08ac-4c8c-ab41-91993c5ccd75\") "
Oct 30 19:49:01.004336 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:01.004310    6996 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f86eae85-6b3a-4c69-a001-b186bed6dd84-plugins-config-volume\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:01.004460 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:01.004342    6996 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1956ab1-08ac-4c8c-ab41-91993c5ccd75-plugins-config-volume\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:01.006076 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:01.006048    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f86eae85-6b3a-4c69-a001-b186bed6dd84-kube-api-access-kr7xl" pod "f86eae85-6b3a-4c69-a001-b186bed6dd84" (UID: "f86eae85-6b3a-4c69-a001-b186bed6dd84"). InnerVolumeSpecName "kube-api-access-kr7xl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:49:01.006163 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:01.006092    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1956ab1-08ac-4c8c-ab41-91993c5ccd75-kube-api-access-9ld2x" pod "a1956ab1-08ac-4c8c-ab41-91993c5ccd75" (UID: "a1956ab1-08ac-4c8c-ab41-91993c5ccd75"). InnerVolumeSpecName "kube-api-access-9ld2x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:49:01.158312 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:01.157542    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kr7xl\" (UniqueName: \"kubernetes.io/projected/f86eae85-6b3a-4c69-a001-b186bed6dd84-kube-api-access-kr7xl\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:01.158312 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:01.157579    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9ld2x\" (UniqueName: \"kubernetes.io/projected/a1956ab1-08ac-4c8c-ab41-91993c5ccd75-kube-api-access-9ld2x\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:01.160340 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:01.160295    6996 generic.go:356] "Generic (PLEG): container finished" podID="a1956ab1-08ac-4c8c-ab41-91993c5ccd75" containerID="6afbac3cb759c616366d5bcdd78f880a34caeee996d23fda8c84235a35b940bb" exitCode=0
Oct 30 19:49:01.160574 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:01.160403    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-913351-64c79d9748-wzvj5" event={"ID":"a1956ab1-08ac-4c8c-ab41-91993c5ccd75","Type":"ContainerDied","Data":"6afbac3cb759c616366d5bcdd78f880a34caeee996d23fda8c84235a35b940bb"}
Oct 30 19:49:01.160574 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:01.160455    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-913351-64c79d9748-wzvj5" event={"ID":"a1956ab1-08ac-4c8c-ab41-91993c5ccd75","Type":"ContainerDied","Data":"cb5b26e502635f77dc4924b777fa174fd7c63eb95efbe481fc28819d4224389e"}
Oct 30 19:49:01.160574 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:01.160502    6996 scope.go:122] "RemoveContainer" containerID="6afbac3cb759c616366d5bcdd78f880a34caeee996d23fda8c84235a35b940bb"
Oct 30 19:49:01.160574 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:01.160539    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-913351-64c79d9748-wzvj5"
Oct 30 19:49:01.165465 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:01.165433    6996 generic.go:356] "Generic (PLEG): container finished" podID="f86eae85-6b3a-4c69-a001-b186bed6dd84" containerID="608de6f325cee9463578440d095199418de8f31b3bc6e846697a7bcb99577efe" exitCode=0
Oct 30 19:49:01.167192 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:01.165994    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-913351-5587d6644c-25q64" event={"ID":"f86eae85-6b3a-4c69-a001-b186bed6dd84","Type":"ContainerDied","Data":"608de6f325cee9463578440d095199418de8f31b3bc6e846697a7bcb99577efe"}
Oct 30 19:49:01.167192 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:01.166059    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-913351-5587d6644c-25q64" event={"ID":"f86eae85-6b3a-4c69-a001-b186bed6dd84","Type":"ContainerDied","Data":"44898a13a94abeb3cf98a0c7fa45b985074c1afe4f0793a7e54a19f35fa2063b"}
Oct 30 19:49:01.167192 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:01.166147    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-913351-5587d6644c-25q64"
Oct 30 19:49:01.195852 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:01.195813    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c9d69c-6d96fc497c-p25r9" event={"ID":"5e65ecbf-2aba-4a2c-abfa-c5d6c789152a","Type":"ContainerStarted","Data":"e75b1fb062e28ba30d7d534d208e98bfa059a8ef3898788460813a0043a578fe"}
Oct 30 19:49:01.196859 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:01.196820    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c9d69c-6d96fc497c-p25r9"
Oct 30 19:49:01.198743 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:01.198655    6996 scope.go:122] "RemoveContainer" containerID="6afbac3cb759c616366d5bcdd78f880a34caeee996d23fda8c84235a35b940bb"
Oct 30 19:49:01.200466 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:49:01.200428    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6afbac3cb759c616366d5bcdd78f880a34caeee996d23fda8c84235a35b940bb\": container with ID starting with 6afbac3cb759c616366d5bcdd78f880a34caeee996d23fda8c84235a35b940bb not found: ID does not exist" containerID="6afbac3cb759c616366d5bcdd78f880a34caeee996d23fda8c84235a35b940bb"
Oct 30 19:49:01.200589 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:01.200474    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6afbac3cb759c616366d5bcdd78f880a34caeee996d23fda8c84235a35b940bb"} err="failed to get container status \"6afbac3cb759c616366d5bcdd78f880a34caeee996d23fda8c84235a35b940bb\": rpc error: code = NotFound desc = could not find container \"6afbac3cb759c616366d5bcdd78f880a34caeee996d23fda8c84235a35b940bb\": container with ID starting with 6afbac3cb759c616366d5bcdd78f880a34caeee996d23fda8c84235a35b940bb not found: ID does not exist"
Oct 30 19:49:01.200589 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:01.200532    6996 scope.go:122] "RemoveContainer" containerID="608de6f325cee9463578440d095199418de8f31b3bc6e846697a7bcb99577efe"
Oct 30 19:49:01.209722 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:49:01.209694    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 19:49:01.214181 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:01.214164    6996 scope.go:122] "RemoveContainer" containerID="608de6f325cee9463578440d095199418de8f31b3bc6e846697a7bcb99577efe"
Oct 30 19:49:01.214461 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:49:01.214442    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"608de6f325cee9463578440d095199418de8f31b3bc6e846697a7bcb99577efe\": container with ID starting with 608de6f325cee9463578440d095199418de8f31b3bc6e846697a7bcb99577efe not found: ID does not exist" containerID="608de6f325cee9463578440d095199418de8f31b3bc6e846697a7bcb99577efe"
Oct 30 19:49:01.214544 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:01.214474    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"608de6f325cee9463578440d095199418de8f31b3bc6e846697a7bcb99577efe"} err="failed to get container status \"608de6f325cee9463578440d095199418de8f31b3bc6e846697a7bcb99577efe\": rpc error: code = NotFound desc = could not find container \"608de6f325cee9463578440d095199418de8f31b3bc6e846697a7bcb99577efe\": container with ID starting with 608de6f325cee9463578440d095199418de8f31b3bc6e846697a7bcb99577efe not found: ID does not exist"
Oct 30 19:49:01.221537 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:01.221474    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c9d69c-6d96fc497c-p25r9" podStartSLOduration=1.694033336 podStartE2EDuration="2.221461732s" totalImagesPullingTime="527.428396ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:48:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:49:01.217894045 +0000 UTC m=+1016.546012200" watchObservedRunningTime="2026-10-30 19:49:01.221461732 +0000 UTC m=+1016.549579912"
Oct 30 19:49:01.236458 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:01.236435    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-913351-5587d6644c-25q64"]
Oct 30 19:49:01.244412 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:01.244292    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-913351-5587d6644c-25q64"]
Oct 30 19:49:01.256885 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:01.256824    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-913351-64c79d9748-wzvj5"]
Oct 30 19:49:01.272034 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:01.272005    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-913351-64c79d9748-wzvj5"]
Oct 30 19:49:01.523269 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:01.523175    6996 patch_prober.go:29] interesting pod/e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:49:01.523863 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:01.523829    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7" podUID="a61de104-8144-4107-a532-61cba1cf53e1" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:49:02.271574 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:02.270671    6996 generic.go:356] "Generic (PLEG): container finished" podID="a61de104-8144-4107-a532-61cba1cf53e1" containerID="f5a0a42afdb4b9a595e1d4f596fd0061d8c85f4a9d6186e9108bc96b5e05f9c6" exitCode=0
Oct 30 19:49:02.274539 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:02.271868    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7" event={"ID":"a61de104-8144-4107-a532-61cba1cf53e1","Type":"ContainerDied","Data":"f5a0a42afdb4b9a595e1d4f596fd0061d8c85f4a9d6186e9108bc96b5e05f9c6"}
Oct 30 19:49:02.274539 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:02.271933    6996 scope.go:122] "RemoveContainer" containerID="b1a3f3d7b4883cad7a223d26a2d85141b389c8b54535927589932316607b26ec"
Oct 30 19:49:03.317066 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:03.317029    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a1956ab1-08ac-4c8c-ab41-91993c5ccd75" path="/var/lib/kubelet/pods/a1956ab1-08ac-4c8c-ab41-91993c5ccd75/volumes"
Oct 30 19:49:03.320342 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:03.320313    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f86eae85-6b3a-4c69-a001-b186bed6dd84" path="/var/lib/kubelet/pods/f86eae85-6b3a-4c69-a001-b186bed6dd84/volumes"
Oct 30 19:49:03.560026 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:03.559986    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7"
Oct 30 19:49:03.588608 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:03.578918    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a61de104-8144-4107-a532-61cba1cf53e1-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a61de104-8144-4107-a532-61cba1cf53e1-istio-podinfo\") pod \"a61de104-8144-4107-a532-61cba1cf53e1\" (UID: \"a61de104-8144-4107-a532-61cba1cf53e1\") "
Oct 30 19:49:03.588608 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:03.579005    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a61de104-8144-4107-a532-61cba1cf53e1-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a61de104-8144-4107-a532-61cba1cf53e1-istiod-ca-cert\") pod \"a61de104-8144-4107-a532-61cba1cf53e1\" (UID: \"a61de104-8144-4107-a532-61cba1cf53e1\") "
Oct 30 19:49:03.588608 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:03.579053    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a61de104-8144-4107-a532-61cba1cf53e1-istio-token\" (UniqueName: \"kubernetes.io/projected/a61de104-8144-4107-a532-61cba1cf53e1-istio-token\") pod \"a61de104-8144-4107-a532-61cba1cf53e1\" (UID: \"a61de104-8144-4107-a532-61cba1cf53e1\") "
Oct 30 19:49:03.588608 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:03.579128    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a61de104-8144-4107-a532-61cba1cf53e1-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a61de104-8144-4107-a532-61cba1cf53e1-istio-envoy\") pod \"a61de104-8144-4107-a532-61cba1cf53e1\" (UID: \"a61de104-8144-4107-a532-61cba1cf53e1\") "
Oct 30 19:49:03.588608 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:03.579165    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a61de104-8144-4107-a532-61cba1cf53e1-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a61de104-8144-4107-a532-61cba1cf53e1-istio-data\") pod \"a61de104-8144-4107-a532-61cba1cf53e1\" (UID: \"a61de104-8144-4107-a532-61cba1cf53e1\") "
Oct 30 19:49:03.588608 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:03.579193    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a61de104-8144-4107-a532-61cba1cf53e1-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a61de104-8144-4107-a532-61cba1cf53e1-credential-socket\") pod \"a61de104-8144-4107-a532-61cba1cf53e1\" (UID: \"a61de104-8144-4107-a532-61cba1cf53e1\") "
Oct 30 19:49:03.588608 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:03.579228    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a61de104-8144-4107-a532-61cba1cf53e1-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a61de104-8144-4107-a532-61cba1cf53e1-workload-socket\") pod \"a61de104-8144-4107-a532-61cba1cf53e1\" (UID: \"a61de104-8144-4107-a532-61cba1cf53e1\") "
Oct 30 19:49:03.588608 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:03.579311    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a61de104-8144-4107-a532-61cba1cf53e1-kube-api-access-wglbw\" (UniqueName: \"kubernetes.io/projected/a61de104-8144-4107-a532-61cba1cf53e1-kube-api-access-wglbw\") pod \"a61de104-8144-4107-a532-61cba1cf53e1\" (UID: \"a61de104-8144-4107-a532-61cba1cf53e1\") "
Oct 30 19:49:03.588608 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:03.579377    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a61de104-8144-4107-a532-61cba1cf53e1-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a61de104-8144-4107-a532-61cba1cf53e1-workload-certs\") pod \"a61de104-8144-4107-a532-61cba1cf53e1\" (UID: \"a61de104-8144-4107-a532-61cba1cf53e1\") "
Oct 30 19:49:03.608508 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:03.592216    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a61de104-8144-4107-a532-61cba1cf53e1-credential-socket" pod "a61de104-8144-4107-a532-61cba1cf53e1" (UID: "a61de104-8144-4107-a532-61cba1cf53e1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:49:03.608508 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:03.592432    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a61de104-8144-4107-a532-61cba1cf53e1-istio-data" pod "a61de104-8144-4107-a532-61cba1cf53e1" (UID: "a61de104-8144-4107-a532-61cba1cf53e1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:49:03.608508 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:03.599447    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a61de104-8144-4107-a532-61cba1cf53e1-istiod-ca-cert" pod "a61de104-8144-4107-a532-61cba1cf53e1" (UID: "a61de104-8144-4107-a532-61cba1cf53e1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:49:03.608508 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:03.605821    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a61de104-8144-4107-a532-61cba1cf53e1-istio-envoy" pod "a61de104-8144-4107-a532-61cba1cf53e1" (UID: "a61de104-8144-4107-a532-61cba1cf53e1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:49:03.656893 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:03.651984    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a61de104-8144-4107-a532-61cba1cf53e1-istio-token" pod "a61de104-8144-4107-a532-61cba1cf53e1" (UID: "a61de104-8144-4107-a532-61cba1cf53e1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:49:03.656893 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:03.654871    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a61de104-8144-4107-a532-61cba1cf53e1-istio-podinfo" pod "a61de104-8144-4107-a532-61cba1cf53e1" (UID: "a61de104-8144-4107-a532-61cba1cf53e1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:49:03.658630 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:03.657524    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a61de104-8144-4107-a532-61cba1cf53e1-workload-socket" pod "a61de104-8144-4107-a532-61cba1cf53e1" (UID: "a61de104-8144-4107-a532-61cba1cf53e1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:49:03.660540 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:03.659023    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a61de104-8144-4107-a532-61cba1cf53e1-workload-certs" pod "a61de104-8144-4107-a532-61cba1cf53e1" (UID: "a61de104-8144-4107-a532-61cba1cf53e1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:49:03.663432 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:03.663390    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a61de104-8144-4107-a532-61cba1cf53e1-kube-api-access-wglbw" pod "a61de104-8144-4107-a532-61cba1cf53e1" (UID: "a61de104-8144-4107-a532-61cba1cf53e1"). InnerVolumeSpecName "kube-api-access-wglbw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:49:03.689880 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:03.689838    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wglbw\" (UniqueName: \"kubernetes.io/projected/a61de104-8144-4107-a532-61cba1cf53e1-kube-api-access-wglbw\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:03.689880 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:03.689884    6996 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a61de104-8144-4107-a532-61cba1cf53e1-workload-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:03.690079 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:03.689902    6996 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a61de104-8144-4107-a532-61cba1cf53e1-istio-podinfo\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:03.690079 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:03.689916    6996 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a61de104-8144-4107-a532-61cba1cf53e1-istiod-ca-cert\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:03.690079 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:03.689932    6996 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a61de104-8144-4107-a532-61cba1cf53e1-istio-token\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:03.690079 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:03.689945    6996 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a61de104-8144-4107-a532-61cba1cf53e1-istio-envoy\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:03.690079 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:03.689959    6996 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a61de104-8144-4107-a532-61cba1cf53e1-istio-data\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:03.690079 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:03.689973    6996 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a61de104-8144-4107-a532-61cba1cf53e1-credential-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:03.690079 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:03.689989    6996 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a61de104-8144-4107-a532-61cba1cf53e1-workload-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:04.409827 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:04.409795    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7"
Oct 30 19:49:04.410198 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:04.409797    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7" event={"ID":"a61de104-8144-4107-a532-61cba1cf53e1","Type":"ContainerDied","Data":"ff4895635fe9cc4306dff8af408ef9713af6630385f698d461239d44f7b2485d"}
Oct 30 19:49:04.410198 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:04.409925    6996 scope.go:122] "RemoveContainer" containerID="f5a0a42afdb4b9a595e1d4f596fd0061d8c85f4a9d6186e9108bc96b5e05f9c6"
Oct 30 19:49:04.436705 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:04.436676    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7"]
Oct 30 19:49:04.442006 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:04.441982    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-501ac8-openshift-default-6d65bf456f-vptb7"]
Oct 30 19:49:04.792659 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:04.792588    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a61de104-8144-4107-a532-61cba1cf53e1" path="/var/lib/kubelet/pods/a61de104-8144-4107-a532-61cba1cf53e1/volumes"
Oct 30 19:49:04.985494 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:04.985449    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c9d69c-fffd76955-lkr9q"
Oct 30 19:49:04.985657 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:04.985543    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-c9d69c-844759994d-x4kmk"
Oct 30 19:49:07.553458 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:07.553423    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-2761c7-776cb78fff-k5tqc"]
Oct 30 19:49:08.407658 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:08.407625    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c9d69c-6d96fc497c-p25r9"
Oct 30 19:49:11.484771 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:11.484736    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-c9d69c-844759994d-x4kmk"]
Oct 30 19:49:11.533002 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:11.532964    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-c9d69c-fffd76955-lkr9q"]
Oct 30 19:49:12.554229 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:12.554184    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-2761c7-776cb78fff-k5tqc" podUID="7eefebe9-9a98-49fc-a091-151f8719e3f2" containerName="payload-pre-processing" containerID="cri-o://2b598e119697484f88330b4a49baa3f9e430a7c1ab9b0a87ee8c0a2f9c388599" gracePeriod=25
Oct 30 19:49:12.829430 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:12.829405    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-2761c7-776cb78fff-k5tqc"
Oct 30 19:49:12.961155 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:12.961109    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7eefebe9-9a98-49fc-a091-151f8719e3f2-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7eefebe9-9a98-49fc-a091-151f8719e3f2-plugins-config-volume\") pod \"7eefebe9-9a98-49fc-a091-151f8719e3f2\" (UID: \"7eefebe9-9a98-49fc-a091-151f8719e3f2\") "
Oct 30 19:49:12.961369 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:12.961328    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7eefebe9-9a98-49fc-a091-151f8719e3f2-kube-api-access-5t2s8\" (UniqueName: \"kubernetes.io/projected/7eefebe9-9a98-49fc-a091-151f8719e3f2-kube-api-access-5t2s8\") pod \"7eefebe9-9a98-49fc-a091-151f8719e3f2\" (UID: \"7eefebe9-9a98-49fc-a091-151f8719e3f2\") "
Oct 30 19:49:12.961580 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:12.961556    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7eefebe9-9a98-49fc-a091-151f8719e3f2-plugins-config-volume" pod "7eefebe9-9a98-49fc-a091-151f8719e3f2" (UID: "7eefebe9-9a98-49fc-a091-151f8719e3f2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:49:12.961850 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:12.961830    6996 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7eefebe9-9a98-49fc-a091-151f8719e3f2-plugins-config-volume\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:12.963438 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:12.963418    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7eefebe9-9a98-49fc-a091-151f8719e3f2-kube-api-access-5t2s8" pod "7eefebe9-9a98-49fc-a091-151f8719e3f2" (UID: "7eefebe9-9a98-49fc-a091-151f8719e3f2"). InnerVolumeSpecName "kube-api-access-5t2s8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:49:13.063625 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.063547    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5t2s8\" (UniqueName: \"kubernetes.io/projected/7eefebe9-9a98-49fc-a091-151f8719e3f2-kube-api-access-5t2s8\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:13.490270 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.490237    6996 generic.go:356] "Generic (PLEG): container finished" podID="7eefebe9-9a98-49fc-a091-151f8719e3f2" containerID="2b598e119697484f88330b4a49baa3f9e430a7c1ab9b0a87ee8c0a2f9c388599" exitCode=0
Oct 30 19:49:13.490548 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.490304    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-2761c7-776cb78fff-k5tqc"
Oct 30 19:49:13.490548 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.490327    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-2761c7-776cb78fff-k5tqc" event={"ID":"7eefebe9-9a98-49fc-a091-151f8719e3f2","Type":"ContainerDied","Data":"2b598e119697484f88330b4a49baa3f9e430a7c1ab9b0a87ee8c0a2f9c388599"}
Oct 30 19:49:13.490548 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.490385    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-2761c7-776cb78fff-k5tqc" event={"ID":"7eefebe9-9a98-49fc-a091-151f8719e3f2","Type":"ContainerDied","Data":"216b921ae9c9406d6f54b316466a20806355bb7a174843c78d5d3bbea2768cbd"}
Oct 30 19:49:13.490548 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.490410    6996 scope.go:122] "RemoveContainer" containerID="2b598e119697484f88330b4a49baa3f9e430a7c1ab9b0a87ee8c0a2f9c388599"
Oct 30 19:49:13.508259 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.508227    6996 scope.go:122] "RemoveContainer" containerID="2b598e119697484f88330b4a49baa3f9e430a7c1ab9b0a87ee8c0a2f9c388599"
Oct 30 19:49:13.508624 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:49:13.508603    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b598e119697484f88330b4a49baa3f9e430a7c1ab9b0a87ee8c0a2f9c388599\": container with ID starting with 2b598e119697484f88330b4a49baa3f9e430a7c1ab9b0a87ee8c0a2f9c388599 not found: ID does not exist" containerID="2b598e119697484f88330b4a49baa3f9e430a7c1ab9b0a87ee8c0a2f9c388599"
Oct 30 19:49:13.508693 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.508636    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b598e119697484f88330b4a49baa3f9e430a7c1ab9b0a87ee8c0a2f9c388599"} err="failed to get container status \"2b598e119697484f88330b4a49baa3f9e430a7c1ab9b0a87ee8c0a2f9c388599\": rpc error: code = NotFound desc = could not find container \"2b598e119697484f88330b4a49baa3f9e430a7c1ab9b0a87ee8c0a2f9c388599\": container with ID starting with 2b598e119697484f88330b4a49baa3f9e430a7c1ab9b0a87ee8c0a2f9c388599 not found: ID does not exist"
Oct 30 19:49:13.526458 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.526422    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-2761c7-776cb78fff-k5tqc"]
Oct 30 19:49:13.531162 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.531137    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-2761c7-776cb78fff-k5tqc"]
Oct 30 19:49:13.682780 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.682744    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc"]
Oct 30 19:49:13.683791 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.683766    6996 cpu_manager.go:414] "containerMap: removing container" podUID="a1956ab1-08ac-4c8c-ab41-91993c5ccd75" containerName="payload-processing"
Oct 30 19:49:13.683791 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.683789    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a1956ab1-08ac-4c8c-ab41-91993c5ccd75" containerName="payload-processing"
Oct 30 19:49:13.683982 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.683818    6996 cpu_manager.go:414] "containerMap: removing container" podUID="a61de104-8144-4107-a532-61cba1cf53e1" containerName="istio-proxy"
Oct 30 19:49:13.683982 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.683827    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a61de104-8144-4107-a532-61cba1cf53e1" containerName="istio-proxy"
Oct 30 19:49:13.683982 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.683850    6996 cpu_manager.go:414] "containerMap: removing container" podUID="a61de104-8144-4107-a532-61cba1cf53e1" containerName="istio-proxy"
Oct 30 19:49:13.683982 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.683855    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a61de104-8144-4107-a532-61cba1cf53e1" containerName="istio-proxy"
Oct 30 19:49:13.683982 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.683865    6996 cpu_manager.go:414] "containerMap: removing container" podUID="f86eae85-6b3a-4c69-a001-b186bed6dd84" containerName="payload-pre-processing"
Oct 30 19:49:13.683982 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.683869    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f86eae85-6b3a-4c69-a001-b186bed6dd84" containerName="payload-pre-processing"
Oct 30 19:49:13.683982 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.683882    6996 cpu_manager.go:414] "containerMap: removing container" podUID="7eefebe9-9a98-49fc-a091-151f8719e3f2" containerName="payload-pre-processing"
Oct 30 19:49:13.683982 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.683886    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7eefebe9-9a98-49fc-a091-151f8719e3f2" containerName="payload-pre-processing"
Oct 30 19:49:13.683982 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.683983    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="a1956ab1-08ac-4c8c-ab41-91993c5ccd75" containerName="payload-processing"
Oct 30 19:49:13.684430 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.683993    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="a61de104-8144-4107-a532-61cba1cf53e1" containerName="istio-proxy"
Oct 30 19:49:13.684430 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.684004    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="a61de104-8144-4107-a532-61cba1cf53e1" containerName="istio-proxy"
Oct 30 19:49:13.684430 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.684011    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="f86eae85-6b3a-4c69-a001-b186bed6dd84" containerName="payload-pre-processing"
Oct 30 19:49:13.684430 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.684020    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="7eefebe9-9a98-49fc-a091-151f8719e3f2" containerName="payload-pre-processing"
Oct 30 19:49:13.688946 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.688925    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lhxgc"]
Oct 30 19:49:13.689065 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.689045    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc"
Oct 30 19:49:13.693120 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.693102    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lhxgc"
Oct 30 19:49:13.702518 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.702466    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lhxgc"]
Oct 30 19:49:13.703650 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.703631    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Oct 30 19:49:13.704017 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.703997    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-gmjw8\""
Oct 30 19:49:13.704168 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.704152    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Oct 30 19:49:13.704303 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.704288    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Oct 30 19:49:13.704539 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.704518    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-1e3723-openshift-default-dockercfg-vxwnb\""
Oct 30 19:49:13.709728 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.709706    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc"]
Oct 30 19:49:13.888121 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.888080    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6d36c63-b64c-49e6-8c9e-7e939e777275-istio-token\") pod \"e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc\" (UID: \"e6d36c63-b64c-49e6-8c9e-7e939e777275\") " pod="openshift-ingress/e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc"
Oct 30 19:49:13.888767 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.888740    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/95f15479-e511-4755-9380-a8f23af90ac6-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-lhxgc\" (UID: \"95f15479-e511-4755-9380-a8f23af90ac6\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lhxgc"
Oct 30 19:49:13.889059 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.889042    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6d36c63-b64c-49e6-8c9e-7e939e777275-istio-data\") pod \"e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc\" (UID: \"e6d36c63-b64c-49e6-8c9e-7e939e777275\") " pod="openshift-ingress/e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc"
Oct 30 19:49:13.889229 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.889212    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/95f15479-e511-4755-9380-a8f23af90ac6-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-lhxgc\" (UID: \"95f15479-e511-4755-9380-a8f23af90ac6\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lhxgc"
Oct 30 19:49:13.889379 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.889363    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6d36c63-b64c-49e6-8c9e-7e939e777275-istiod-ca-cert\") pod \"e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc\" (UID: \"e6d36c63-b64c-49e6-8c9e-7e939e777275\") " pod="openshift-ingress/e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc"
Oct 30 19:49:13.889542 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.889525    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6d36c63-b64c-49e6-8c9e-7e939e777275-istio-podinfo\") pod \"e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc\" (UID: \"e6d36c63-b64c-49e6-8c9e-7e939e777275\") " pod="openshift-ingress/e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc"
Oct 30 19:49:13.889705 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.889691    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/95f15479-e511-4755-9380-a8f23af90ac6-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-lhxgc\" (UID: \"95f15479-e511-4755-9380-a8f23af90ac6\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lhxgc"
Oct 30 19:49:13.889855 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.889839    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6d36c63-b64c-49e6-8c9e-7e939e777275-istio-envoy\") pod \"e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc\" (UID: \"e6d36c63-b64c-49e6-8c9e-7e939e777275\") " pod="openshift-ingress/e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc"
Oct 30 19:49:13.890081 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.890065    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hnvc\" (UniqueName: \"kubernetes.io/projected/95f15479-e511-4755-9380-a8f23af90ac6-kube-api-access-4hnvc\") pod \"istiod-openshift-gateway-7ddd9cfdc8-lhxgc\" (UID: \"95f15479-e511-4755-9380-a8f23af90ac6\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lhxgc"
Oct 30 19:49:13.890262 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.890245    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6d36c63-b64c-49e6-8c9e-7e939e777275-credential-socket\") pod \"e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc\" (UID: \"e6d36c63-b64c-49e6-8c9e-7e939e777275\") " pod="openshift-ingress/e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc"
Oct 30 19:49:13.890415 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.890400    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/95f15479-e511-4755-9380-a8f23af90ac6-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-lhxgc\" (UID: \"95f15479-e511-4755-9380-a8f23af90ac6\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lhxgc"
Oct 30 19:49:13.890590 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.890573    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/95f15479-e511-4755-9380-a8f23af90ac6-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-lhxgc\" (UID: \"95f15479-e511-4755-9380-a8f23af90ac6\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lhxgc"
Oct 30 19:49:13.890768 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.890752    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6d36c63-b64c-49e6-8c9e-7e939e777275-workload-certs\") pod \"e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc\" (UID: \"e6d36c63-b64c-49e6-8c9e-7e939e777275\") " pod="openshift-ingress/e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc"
Oct 30 19:49:13.890920 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.890904    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95f15479-e511-4755-9380-a8f23af90ac6-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-lhxgc\" (UID: \"95f15479-e511-4755-9380-a8f23af90ac6\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lhxgc"
Oct 30 19:49:13.891090 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.891075    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gg9vp\" (UniqueName: \"kubernetes.io/projected/e6d36c63-b64c-49e6-8c9e-7e939e777275-kube-api-access-gg9vp\") pod \"e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc\" (UID: \"e6d36c63-b64c-49e6-8c9e-7e939e777275\") " pod="openshift-ingress/e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc"
Oct 30 19:49:13.891229 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.891215    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6d36c63-b64c-49e6-8c9e-7e939e777275-workload-socket\") pod \"e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc\" (UID: \"e6d36c63-b64c-49e6-8c9e-7e939e777275\") " pod="openshift-ingress/e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc"
Oct 30 19:49:13.992987 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.992951    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6d36c63-b64c-49e6-8c9e-7e939e777275-istio-token\") pod \"e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc\" (UID: \"e6d36c63-b64c-49e6-8c9e-7e939e777275\") " pod="openshift-ingress/e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc"
Oct 30 19:49:13.992987 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.992993    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/95f15479-e511-4755-9380-a8f23af90ac6-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-lhxgc\" (UID: \"95f15479-e511-4755-9380-a8f23af90ac6\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lhxgc"
Oct 30 19:49:13.993218 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.993066    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6d36c63-b64c-49e6-8c9e-7e939e777275-istio-data\") pod \"e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc\" (UID: \"e6d36c63-b64c-49e6-8c9e-7e939e777275\") " pod="openshift-ingress/e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc"
Oct 30 19:49:13.993218 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.993103    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/95f15479-e511-4755-9380-a8f23af90ac6-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-lhxgc\" (UID: \"95f15479-e511-4755-9380-a8f23af90ac6\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lhxgc"
Oct 30 19:49:13.993218 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.993136    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6d36c63-b64c-49e6-8c9e-7e939e777275-istiod-ca-cert\") pod \"e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc\" (UID: \"e6d36c63-b64c-49e6-8c9e-7e939e777275\") " pod="openshift-ingress/e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc"
Oct 30 19:49:13.993218 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.993162    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6d36c63-b64c-49e6-8c9e-7e939e777275-istio-podinfo\") pod \"e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc\" (UID: \"e6d36c63-b64c-49e6-8c9e-7e939e777275\") " pod="openshift-ingress/e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc"
Oct 30 19:49:13.993218 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.993185    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/95f15479-e511-4755-9380-a8f23af90ac6-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-lhxgc\" (UID: \"95f15479-e511-4755-9380-a8f23af90ac6\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lhxgc"
Oct 30 19:49:13.993218 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.993215    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6d36c63-b64c-49e6-8c9e-7e939e777275-istio-envoy\") pod \"e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc\" (UID: \"e6d36c63-b64c-49e6-8c9e-7e939e777275\") " pod="openshift-ingress/e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc"
Oct 30 19:49:13.993572 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.993325    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hnvc\" (UniqueName: \"kubernetes.io/projected/95f15479-e511-4755-9380-a8f23af90ac6-kube-api-access-4hnvc\") pod \"istiod-openshift-gateway-7ddd9cfdc8-lhxgc\" (UID: \"95f15479-e511-4755-9380-a8f23af90ac6\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lhxgc"
Oct 30 19:49:13.993572 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.993393    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6d36c63-b64c-49e6-8c9e-7e939e777275-credential-socket\") pod \"e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc\" (UID: \"e6d36c63-b64c-49e6-8c9e-7e939e777275\") " pod="openshift-ingress/e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc"
Oct 30 19:49:13.993572 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.993425    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/95f15479-e511-4755-9380-a8f23af90ac6-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-lhxgc\" (UID: \"95f15479-e511-4755-9380-a8f23af90ac6\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lhxgc"
Oct 30 19:49:13.993572 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.993473    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/95f15479-e511-4755-9380-a8f23af90ac6-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-lhxgc\" (UID: \"95f15479-e511-4755-9380-a8f23af90ac6\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lhxgc"
Oct 30 19:49:13.993572 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.993526    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6d36c63-b64c-49e6-8c9e-7e939e777275-istio-data\") pod \"e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc\" (UID: \"e6d36c63-b64c-49e6-8c9e-7e939e777275\") " pod="openshift-ingress/e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc"
Oct 30 19:49:13.993836 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.993771    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6d36c63-b64c-49e6-8c9e-7e939e777275-workload-certs\") pod \"e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc\" (UID: \"e6d36c63-b64c-49e6-8c9e-7e939e777275\") " pod="openshift-ingress/e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc"
Oct 30 19:49:13.993836 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.993538    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6d36c63-b64c-49e6-8c9e-7e939e777275-workload-certs\") pod \"e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc\" (UID: \"e6d36c63-b64c-49e6-8c9e-7e939e777275\") " pod="openshift-ingress/e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc"
Oct 30 19:49:13.993836 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.993826    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95f15479-e511-4755-9380-a8f23af90ac6-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-lhxgc\" (UID: \"95f15479-e511-4755-9380-a8f23af90ac6\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lhxgc"
Oct 30 19:49:13.993992 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.993882    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gg9vp\" (UniqueName: \"kubernetes.io/projected/e6d36c63-b64c-49e6-8c9e-7e939e777275-kube-api-access-gg9vp\") pod \"e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc\" (UID: \"e6d36c63-b64c-49e6-8c9e-7e939e777275\") " pod="openshift-ingress/e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc"
Oct 30 19:49:13.993992 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.993914    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6d36c63-b64c-49e6-8c9e-7e939e777275-workload-socket\") pod \"e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc\" (UID: \"e6d36c63-b64c-49e6-8c9e-7e939e777275\") " pod="openshift-ingress/e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc"
Oct 30 19:49:13.993992 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.993929    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6d36c63-b64c-49e6-8c9e-7e939e777275-istiod-ca-cert\") pod \"e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc\" (UID: \"e6d36c63-b64c-49e6-8c9e-7e939e777275\") " pod="openshift-ingress/e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc"
Oct 30 19:49:13.994139 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.994049    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6d36c63-b64c-49e6-8c9e-7e939e777275-credential-socket\") pod \"e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc\" (UID: \"e6d36c63-b64c-49e6-8c9e-7e939e777275\") " pod="openshift-ingress/e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc"
Oct 30 19:49:13.994375 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.994348    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6d36c63-b64c-49e6-8c9e-7e939e777275-workload-socket\") pod \"e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc\" (UID: \"e6d36c63-b64c-49e6-8c9e-7e939e777275\") " pod="openshift-ingress/e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc"
Oct 30 19:49:13.995091 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.995062    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/95f15479-e511-4755-9380-a8f23af90ac6-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-lhxgc\" (UID: \"95f15479-e511-4755-9380-a8f23af90ac6\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lhxgc"
Oct 30 19:49:13.995960 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.995916    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/95f15479-e511-4755-9380-a8f23af90ac6-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-lhxgc\" (UID: \"95f15479-e511-4755-9380-a8f23af90ac6\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lhxgc"
Oct 30 19:49:13.996128 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.996103    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/95f15479-e511-4755-9380-a8f23af90ac6-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-lhxgc\" (UID: \"95f15479-e511-4755-9380-a8f23af90ac6\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lhxgc"
Oct 30 19:49:13.996251 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.996230    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6d36c63-b64c-49e6-8c9e-7e939e777275-istio-envoy\") pod \"e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc\" (UID: \"e6d36c63-b64c-49e6-8c9e-7e939e777275\") " pod="openshift-ingress/e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc"
Oct 30 19:49:13.996330 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.996308    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/95f15479-e511-4755-9380-a8f23af90ac6-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-lhxgc\" (UID: \"95f15479-e511-4755-9380-a8f23af90ac6\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lhxgc"
Oct 30 19:49:13.996411 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.996395    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/95f15479-e511-4755-9380-a8f23af90ac6-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-lhxgc\" (UID: \"95f15479-e511-4755-9380-a8f23af90ac6\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lhxgc"
Oct 30 19:49:13.997235 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:13.997211    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6d36c63-b64c-49e6-8c9e-7e939e777275-istio-podinfo\") pod \"e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc\" (UID: \"e6d36c63-b64c-49e6-8c9e-7e939e777275\") " pod="openshift-ingress/e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc"
Oct 30 19:49:14.054304 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:14.054275    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95f15479-e511-4755-9380-a8f23af90ac6-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-lhxgc\" (UID: \"95f15479-e511-4755-9380-a8f23af90ac6\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lhxgc"
Oct 30 19:49:14.054444 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:14.054273    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6d36c63-b64c-49e6-8c9e-7e939e777275-istio-token\") pod \"e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc\" (UID: \"e6d36c63-b64c-49e6-8c9e-7e939e777275\") " pod="openshift-ingress/e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc"
Oct 30 19:49:14.054566 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:14.054539    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gg9vp\" (UniqueName: \"kubernetes.io/projected/e6d36c63-b64c-49e6-8c9e-7e939e777275-kube-api-access-gg9vp\") pod \"e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc\" (UID: \"e6d36c63-b64c-49e6-8c9e-7e939e777275\") " pod="openshift-ingress/e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc"
Oct 30 19:49:14.055438 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:14.055417    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hnvc\" (UniqueName: \"kubernetes.io/projected/95f15479-e511-4755-9380-a8f23af90ac6-kube-api-access-4hnvc\") pod \"istiod-openshift-gateway-7ddd9cfdc8-lhxgc\" (UID: \"95f15479-e511-4755-9380-a8f23af90ac6\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lhxgc"
Oct 30 19:49:14.308451 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:14.308357    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc"
Oct 30 19:49:14.316717 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:14.316695    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lhxgc"
Oct 30 19:49:14.643230 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:14.643197    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lhxgc"]
Oct 30 19:49:14.643966 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:49:14.643936    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95f15479_e511_4755_9380_a8f23af90ac6.slice/crio-92c67ce0553b18addf91a564687e1ba937f92145daae6dda61b01297f797539d WatchSource:0}: Error finding container 92c67ce0553b18addf91a564687e1ba937f92145daae6dda61b01297f797539d: Status 404 returned error can't find the container with id 92c67ce0553b18addf91a564687e1ba937f92145daae6dda61b01297f797539d
Oct 30 19:49:14.654065 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:14.654034    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc"]
Oct 30 19:49:14.662547 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:14.661194    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:49:14.662547 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:14.662094    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:49:14.662547 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:14.662158    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:49:14.796615 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:14.796576    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7eefebe9-9a98-49fc-a091-151f8719e3f2" path="/var/lib/kubelet/pods/7eefebe9-9a98-49fc-a091-151f8719e3f2/volumes"
Oct 30 19:49:15.508302 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:15.508265    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lhxgc" event={"ID":"95f15479-e511-4755-9380-a8f23af90ac6","Type":"ContainerStarted","Data":"92c67ce0553b18addf91a564687e1ba937f92145daae6dda61b01297f797539d"}
Oct 30 19:49:15.509890 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:15.509861    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc" event={"ID":"e6d36c63-b64c-49e6-8c9e-7e939e777275","Type":"ContainerStarted","Data":"c673f266600df1d3edc7064e0f0b7a22751b4e27d117c7e3f16a5b1e64c2ab80"}
Oct 30 19:49:15.510002 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:15.509911    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc" event={"ID":"e6d36c63-b64c-49e6-8c9e-7e939e777275","Type":"ContainerStarted","Data":"07879f2e5fa731757fd7f574ea34b3fb2c701c68327455bf6ab9bae528bbef5a"}
Oct 30 19:49:15.535940 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:15.535893    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc" podStartSLOduration=2.535879424 podStartE2EDuration="2.535879424s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:49:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:49:15.531656288 +0000 UTC m=+1030.859774442" watchObservedRunningTime="2026-10-30 19:49:15.535879424 +0000 UTC m=+1030.863997577"
Oct 30 19:49:16.308785 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:16.308711    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc"
Oct 30 19:49:16.315756 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:16.315725    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc"
Oct 30 19:49:16.485431 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:16.485391    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-c9d69c-844759994d-x4kmk" podUID="a8736332-22bc-48e0-a80e-9606ae27549a" containerName="payload-processing" containerID="cri-o://b6e56051300854b57751a9cd61b06a08108d3e047d59202551156209b71870f3" gracePeriod=25
Oct 30 19:49:16.519115 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:16.519086    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc"
Oct 30 19:49:16.520127 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:16.520107    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc"
Oct 30 19:49:16.533681 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:16.533650    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c9d69c-fffd76955-lkr9q" podUID="b7601e4d-cc98-4f1d-8e46-331748368032" containerName="payload-pre-processing" containerID="cri-o://e4f9013ea1f2b8d853b3b69b9c7dc44e99b8eec9f49cabde3867d146f25b26bd" gracePeriod=25
Oct 30 19:49:16.893192 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:16.893165    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-c9d69c-844759994d-x4kmk"
Oct 30 19:49:16.945098 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:16.945070    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c9d69c-fffd76955-lkr9q"
Oct 30 19:49:16.949285 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:16.948441    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a8736332-22bc-48e0-a80e-9606ae27549a-kube-api-access-nxvg7\" (UniqueName: \"kubernetes.io/projected/a8736332-22bc-48e0-a80e-9606ae27549a-kube-api-access-nxvg7\") pod \"a8736332-22bc-48e0-a80e-9606ae27549a\" (UID: \"a8736332-22bc-48e0-a80e-9606ae27549a\") "
Oct 30 19:49:16.949285 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:16.948637    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a8736332-22bc-48e0-a80e-9606ae27549a-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8736332-22bc-48e0-a80e-9606ae27549a-plugins-config-volume\") pod \"a8736332-22bc-48e0-a80e-9606ae27549a\" (UID: \"a8736332-22bc-48e0-a80e-9606ae27549a\") "
Oct 30 19:49:16.949470 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:16.949393    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8736332-22bc-48e0-a80e-9606ae27549a-plugins-config-volume" pod "a8736332-22bc-48e0-a80e-9606ae27549a" (UID: "a8736332-22bc-48e0-a80e-9606ae27549a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:49:16.954366 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:16.954339    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8736332-22bc-48e0-a80e-9606ae27549a-kube-api-access-nxvg7" pod "a8736332-22bc-48e0-a80e-9606ae27549a" (UID: "a8736332-22bc-48e0-a80e-9606ae27549a"). InnerVolumeSpecName "kube-api-access-nxvg7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:49:17.050049 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:17.050013    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b7601e4d-cc98-4f1d-8e46-331748368032-kube-api-access-5klcj\" (UniqueName: \"kubernetes.io/projected/b7601e4d-cc98-4f1d-8e46-331748368032-kube-api-access-5klcj\") pod \"b7601e4d-cc98-4f1d-8e46-331748368032\" (UID: \"b7601e4d-cc98-4f1d-8e46-331748368032\") "
Oct 30 19:49:17.050232 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:17.050100    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b7601e4d-cc98-4f1d-8e46-331748368032-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7601e4d-cc98-4f1d-8e46-331748368032-plugins-config-volume\") pod \"b7601e4d-cc98-4f1d-8e46-331748368032\" (UID: \"b7601e4d-cc98-4f1d-8e46-331748368032\") "
Oct 30 19:49:17.051112 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:17.051082    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7601e4d-cc98-4f1d-8e46-331748368032-plugins-config-volume" pod "b7601e4d-cc98-4f1d-8e46-331748368032" (UID: "b7601e4d-cc98-4f1d-8e46-331748368032"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:49:17.051348 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:17.051327    6996 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7601e4d-cc98-4f1d-8e46-331748368032-plugins-config-volume\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:17.051423 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:17.051355    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nxvg7\" (UniqueName: \"kubernetes.io/projected/a8736332-22bc-48e0-a80e-9606ae27549a-kube-api-access-nxvg7\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:17.051423 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:17.051372    6996 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8736332-22bc-48e0-a80e-9606ae27549a-plugins-config-volume\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:17.052947 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:17.052918    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7601e4d-cc98-4f1d-8e46-331748368032-kube-api-access-5klcj" pod "b7601e4d-cc98-4f1d-8e46-331748368032" (UID: "b7601e4d-cc98-4f1d-8e46-331748368032"). InnerVolumeSpecName "kube-api-access-5klcj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:49:17.153289 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:17.153210    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5klcj\" (UniqueName: \"kubernetes.io/projected/b7601e4d-cc98-4f1d-8e46-331748368032-kube-api-access-5klcj\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:17.530185 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:17.530154    6996 generic.go:356] "Generic (PLEG): container finished" podID="a8736332-22bc-48e0-a80e-9606ae27549a" containerID="b6e56051300854b57751a9cd61b06a08108d3e047d59202551156209b71870f3" exitCode=0
Oct 30 19:49:17.530606 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:17.530217    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-c9d69c-844759994d-x4kmk"
Oct 30 19:49:17.530606 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:17.530240    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-c9d69c-844759994d-x4kmk" event={"ID":"a8736332-22bc-48e0-a80e-9606ae27549a","Type":"ContainerDied","Data":"b6e56051300854b57751a9cd61b06a08108d3e047d59202551156209b71870f3"}
Oct 30 19:49:17.530606 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:17.530301    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-c9d69c-844759994d-x4kmk" event={"ID":"a8736332-22bc-48e0-a80e-9606ae27549a","Type":"ContainerDied","Data":"399671e18917ba09f327cb4984e8910472886501d73746a30a253dd66b00d5f7"}
Oct 30 19:49:17.530606 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:17.530325    6996 scope.go:122] "RemoveContainer" containerID="b6e56051300854b57751a9cd61b06a08108d3e047d59202551156209b71870f3"
Oct 30 19:49:17.532042 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:17.532019    6996 generic.go:356] "Generic (PLEG): container finished" podID="b7601e4d-cc98-4f1d-8e46-331748368032" containerID="e4f9013ea1f2b8d853b3b69b9c7dc44e99b8eec9f49cabde3867d146f25b26bd" exitCode=0
Oct 30 19:49:17.532132 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:17.532062    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c9d69c-fffd76955-lkr9q"
Oct 30 19:49:17.532132 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:17.532099    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c9d69c-fffd76955-lkr9q" event={"ID":"b7601e4d-cc98-4f1d-8e46-331748368032","Type":"ContainerDied","Data":"e4f9013ea1f2b8d853b3b69b9c7dc44e99b8eec9f49cabde3867d146f25b26bd"}
Oct 30 19:49:17.532230 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:17.532152    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c9d69c-fffd76955-lkr9q" event={"ID":"b7601e4d-cc98-4f1d-8e46-331748368032","Type":"ContainerDied","Data":"2cc2df9897ea48b8944ad1c0af2ca3fe2192efc5ef81bc5724f28fe1e5911356"}
Oct 30 19:49:17.542919 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:17.542889    6996 scope.go:122] "RemoveContainer" containerID="b6e56051300854b57751a9cd61b06a08108d3e047d59202551156209b71870f3"
Oct 30 19:49:17.543274 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:49:17.543252    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6e56051300854b57751a9cd61b06a08108d3e047d59202551156209b71870f3\": container with ID starting with b6e56051300854b57751a9cd61b06a08108d3e047d59202551156209b71870f3 not found: ID does not exist" containerID="b6e56051300854b57751a9cd61b06a08108d3e047d59202551156209b71870f3"
Oct 30 19:49:17.543384 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:17.543285    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6e56051300854b57751a9cd61b06a08108d3e047d59202551156209b71870f3"} err="failed to get container status \"b6e56051300854b57751a9cd61b06a08108d3e047d59202551156209b71870f3\": rpc error: code = NotFound desc = could not find container \"b6e56051300854b57751a9cd61b06a08108d3e047d59202551156209b71870f3\": container with ID starting with b6e56051300854b57751a9cd61b06a08108d3e047d59202551156209b71870f3 not found: ID does not exist"
Oct 30 19:49:17.543384 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:17.543307    6996 scope.go:122] "RemoveContainer" containerID="e4f9013ea1f2b8d853b3b69b9c7dc44e99b8eec9f49cabde3867d146f25b26bd"
Oct 30 19:49:17.557179 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:17.557152    6996 scope.go:122] "RemoveContainer" containerID="e4f9013ea1f2b8d853b3b69b9c7dc44e99b8eec9f49cabde3867d146f25b26bd"
Oct 30 19:49:17.557648 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:49:17.557628    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4f9013ea1f2b8d853b3b69b9c7dc44e99b8eec9f49cabde3867d146f25b26bd\": container with ID starting with e4f9013ea1f2b8d853b3b69b9c7dc44e99b8eec9f49cabde3867d146f25b26bd not found: ID does not exist" containerID="e4f9013ea1f2b8d853b3b69b9c7dc44e99b8eec9f49cabde3867d146f25b26bd"
Oct 30 19:49:17.557745 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:17.557655    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4f9013ea1f2b8d853b3b69b9c7dc44e99b8eec9f49cabde3867d146f25b26bd"} err="failed to get container status \"e4f9013ea1f2b8d853b3b69b9c7dc44e99b8eec9f49cabde3867d146f25b26bd\": rpc error: code = NotFound desc = could not find container \"e4f9013ea1f2b8d853b3b69b9c7dc44e99b8eec9f49cabde3867d146f25b26bd\": container with ID starting with e4f9013ea1f2b8d853b3b69b9c7dc44e99b8eec9f49cabde3867d146f25b26bd not found: ID does not exist"
Oct 30 19:49:17.571067 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:17.571037    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-c9d69c-fffd76955-lkr9q"]
Oct 30 19:49:17.583809 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:17.583782    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-c9d69c-fffd76955-lkr9q"]
Oct 30 19:49:17.600565 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:17.600536    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-c9d69c-844759994d-x4kmk"]
Oct 30 19:49:17.604294 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:17.604266    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-c9d69c-844759994d-x4kmk"]
Oct 30 19:49:18.245975 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:18.245926    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:49:18.246116 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:18.246028    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:49:18.543768 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:18.543609    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lhxgc" event={"ID":"95f15479-e511-4755-9380-a8f23af90ac6","Type":"ContainerStarted","Data":"efefd1fc5b872df9ed6e084a2a092561fcf9347645ab4e37282a5a02ab43f92c"}
Oct 30 19:49:18.544592 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:18.543888    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lhxgc"
Oct 30 19:49:18.572185 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:18.572121    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lhxgc" podStartSLOduration=1.973459259 podStartE2EDuration="5.572101065s" totalImagesPullingTime="3.598641806s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:49:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:49:18.568753935 +0000 UTC m=+1033.896872090" watchObservedRunningTime="2026-10-30 19:49:18.572101065 +0000 UTC m=+1033.900219220"
Oct 30 19:49:18.795581 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:18.795491    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a8736332-22bc-48e0-a80e-9606ae27549a" path="/var/lib/kubelet/pods/a8736332-22bc-48e0-a80e-9606ae27549a/volumes"
Oct 30 19:49:18.796145 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:18.796125    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b7601e4d-cc98-4f1d-8e46-331748368032" path="/var/lib/kubelet/pods/b7601e4d-cc98-4f1d-8e46-331748368032/volumes"
Oct 30 19:49:19.559023 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:19.558987    6996 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-lhxgc container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:49:19.559561 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:19.559045    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lhxgc" podUID="95f15479-e511-4755-9380-a8f23af90ac6" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:49:22.556938 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:22.556901    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lhxgc"
Oct 30 19:49:24.697156 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:24.697119    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 19:49:31.192086 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:49:31.192047    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 19:49:36.134843 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:36.134806    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r"]
Oct 30 19:49:36.135272 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:36.135011    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r" podUID="e5aaa21a-1a12-4b2b-b05e-f08504685cd8" containerName="istio-proxy" containerID="cri-o://f32b4fd714143e8bcef46c0c51985fe8b9b2765237d4cf55c5259b516c9af770" gracePeriod=30
Oct 30 19:49:39.268534 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:39.268465    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-040623-577cc456d-sf9nq"]
Oct 30 19:49:39.269141 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:39.269127    6996 cpu_manager.go:414] "containerMap: removing container" podUID="a8736332-22bc-48e0-a80e-9606ae27549a" containerName="payload-processing"
Oct 30 19:49:39.269193 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:39.269142    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a8736332-22bc-48e0-a80e-9606ae27549a" containerName="payload-processing"
Oct 30 19:49:39.269193 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:39.269175    6996 cpu_manager.go:414] "containerMap: removing container" podUID="b7601e4d-cc98-4f1d-8e46-331748368032" containerName="payload-pre-processing"
Oct 30 19:49:39.269193 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:39.269180    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b7601e4d-cc98-4f1d-8e46-331748368032" containerName="payload-pre-processing"
Oct 30 19:49:39.269305 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:39.269296    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="b7601e4d-cc98-4f1d-8e46-331748368032" containerName="payload-pre-processing"
Oct 30 19:49:39.269338 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:39.269310    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="a8736332-22bc-48e0-a80e-9606ae27549a" containerName="payload-processing"
Oct 30 19:49:39.274107 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:39.274081    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-040623-577cc456d-sf9nq"
Oct 30 19:49:39.295807 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:39.295776    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-040623-577cc456d-sf9nq"]
Oct 30 19:49:39.362710 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:39.358801    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-040623\""
Oct 30 19:49:39.362710 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:39.358991    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-040623-dockercfg-t626x\""
Oct 30 19:49:39.407369 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:39.399539    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p54vl\" (UniqueName: \"kubernetes.io/projected/cc327913-37b2-489d-8916-8707d6b5561a-kube-api-access-p54vl\") pod \"payload-processing-e2e-ipp-b-040623-577cc456d-sf9nq\" (UID: \"cc327913-37b2-489d-8916-8707d6b5561a\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-040623-577cc456d-sf9nq"
Oct 30 19:49:39.407369 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:39.399869    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cc327913-37b2-489d-8916-8707d6b5561a-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-040623-577cc456d-sf9nq\" (UID: \"cc327913-37b2-489d-8916-8707d6b5561a\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-040623-577cc456d-sf9nq"
Oct 30 19:49:39.501151 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:39.501115    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cc327913-37b2-489d-8916-8707d6b5561a-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-040623-577cc456d-sf9nq\" (UID: \"cc327913-37b2-489d-8916-8707d6b5561a\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-040623-577cc456d-sf9nq"
Oct 30 19:49:39.501444 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:39.501424    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p54vl\" (UniqueName: \"kubernetes.io/projected/cc327913-37b2-489d-8916-8707d6b5561a-kube-api-access-p54vl\") pod \"payload-processing-e2e-ipp-b-040623-577cc456d-sf9nq\" (UID: \"cc327913-37b2-489d-8916-8707d6b5561a\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-040623-577cc456d-sf9nq"
Oct 30 19:49:39.502244 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:39.502219    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cc327913-37b2-489d-8916-8707d6b5561a-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-040623-577cc456d-sf9nq\" (UID: \"cc327913-37b2-489d-8916-8707d6b5561a\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-040623-577cc456d-sf9nq"
Oct 30 19:49:39.515400 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:39.515373    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p54vl\" (UniqueName: \"kubernetes.io/projected/cc327913-37b2-489d-8916-8707d6b5561a-kube-api-access-p54vl\") pod \"payload-processing-e2e-ipp-b-040623-577cc456d-sf9nq\" (UID: \"cc327913-37b2-489d-8916-8707d6b5561a\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-040623-577cc456d-sf9nq"
Oct 30 19:49:39.593175 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:39.593146    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-040623-577cc456d-sf9nq"
Oct 30 19:49:39.884244 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:39.884212    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-040623-577cc456d-sf9nq"]
Oct 30 19:49:39.885692 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:49:39.885668    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc327913_37b2_489d_8916_8707d6b5561a.slice/crio-e481d22b61c0eb4e17e796b3c6e92bb66509018909959e24c55b7da1bef8a8af WatchSource:0}: Error finding container e481d22b61c0eb4e17e796b3c6e92bb66509018909959e24c55b7da1bef8a8af: Status 404 returned error can't find the container with id e481d22b61c0eb4e17e796b3c6e92bb66509018909959e24c55b7da1bef8a8af
Oct 30 19:49:40.816940 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:40.816906    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-040623-577cc456d-sf9nq" event={"ID":"cc327913-37b2-489d-8916-8707d6b5561a","Type":"ContainerStarted","Data":"bfa586a236566615d400d6172cec16daefcf5e8b985f990e61659735cf3de186"}
Oct 30 19:49:40.817409 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:40.816960    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-040623-577cc456d-sf9nq" event={"ID":"cc327913-37b2-489d-8916-8707d6b5561a","Type":"ContainerStarted","Data":"e481d22b61c0eb4e17e796b3c6e92bb66509018909959e24c55b7da1bef8a8af"}
Oct 30 19:49:40.817409 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:40.816988    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-040623-577cc456d-sf9nq"
Oct 30 19:49:40.857676 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:40.857619    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-040623-577cc456d-sf9nq" podStartSLOduration=1.857605008 podStartE2EDuration="1.857605008s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:49:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:49:40.854841265 +0000 UTC m=+1056.182959421" watchObservedRunningTime="2026-10-30 19:49:40.857605008 +0000 UTC m=+1056.185723159"
Oct 30 19:49:41.413668 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:41.413612    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r"
Oct 30 19:49:41.534217 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:41.534175    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e5aaa21a-1a12-4b2b-b05e-f08504685cd8-kube-api-access-xddnz\" (UniqueName: \"kubernetes.io/projected/e5aaa21a-1a12-4b2b-b05e-f08504685cd8-kube-api-access-xddnz\") pod \"e5aaa21a-1a12-4b2b-b05e-f08504685cd8\" (UID: \"e5aaa21a-1a12-4b2b-b05e-f08504685cd8\") "
Oct 30 19:49:41.534369 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:41.534314    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e5aaa21a-1a12-4b2b-b05e-f08504685cd8-istio-token\" (UniqueName: \"kubernetes.io/projected/e5aaa21a-1a12-4b2b-b05e-f08504685cd8-istio-token\") pod \"e5aaa21a-1a12-4b2b-b05e-f08504685cd8\" (UID: \"e5aaa21a-1a12-4b2b-b05e-f08504685cd8\") "
Oct 30 19:49:41.534369 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:41.534356    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e5aaa21a-1a12-4b2b-b05e-f08504685cd8-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5aaa21a-1a12-4b2b-b05e-f08504685cd8-credential-socket\") pod \"e5aaa21a-1a12-4b2b-b05e-f08504685cd8\" (UID: \"e5aaa21a-1a12-4b2b-b05e-f08504685cd8\") "
Oct 30 19:49:41.534538 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:41.534392    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e5aaa21a-1a12-4b2b-b05e-f08504685cd8-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5aaa21a-1a12-4b2b-b05e-f08504685cd8-workload-socket\") pod \"e5aaa21a-1a12-4b2b-b05e-f08504685cd8\" (UID: \"e5aaa21a-1a12-4b2b-b05e-f08504685cd8\") "
Oct 30 19:49:41.534538 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:41.534421    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e5aaa21a-1a12-4b2b-b05e-f08504685cd8-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5aaa21a-1a12-4b2b-b05e-f08504685cd8-workload-certs\") pod \"e5aaa21a-1a12-4b2b-b05e-f08504685cd8\" (UID: \"e5aaa21a-1a12-4b2b-b05e-f08504685cd8\") "
Oct 30 19:49:41.534538 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:41.534523    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e5aaa21a-1a12-4b2b-b05e-f08504685cd8-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5aaa21a-1a12-4b2b-b05e-f08504685cd8-istio-envoy\") pod \"e5aaa21a-1a12-4b2b-b05e-f08504685cd8\" (UID: \"e5aaa21a-1a12-4b2b-b05e-f08504685cd8\") "
Oct 30 19:49:41.534686 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:41.534580    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e5aaa21a-1a12-4b2b-b05e-f08504685cd8-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5aaa21a-1a12-4b2b-b05e-f08504685cd8-istio-data\") pod \"e5aaa21a-1a12-4b2b-b05e-f08504685cd8\" (UID: \"e5aaa21a-1a12-4b2b-b05e-f08504685cd8\") "
Oct 30 19:49:41.534686 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:41.534654    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e5aaa21a-1a12-4b2b-b05e-f08504685cd8-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5aaa21a-1a12-4b2b-b05e-f08504685cd8-istio-podinfo\") pod \"e5aaa21a-1a12-4b2b-b05e-f08504685cd8\" (UID: \"e5aaa21a-1a12-4b2b-b05e-f08504685cd8\") "
Oct 30 19:49:41.534792 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:41.534710    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e5aaa21a-1a12-4b2b-b05e-f08504685cd8-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5aaa21a-1a12-4b2b-b05e-f08504685cd8-istiod-ca-cert\") pod \"e5aaa21a-1a12-4b2b-b05e-f08504685cd8\" (UID: \"e5aaa21a-1a12-4b2b-b05e-f08504685cd8\") "
Oct 30 19:49:41.534792 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:41.534757    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5aaa21a-1a12-4b2b-b05e-f08504685cd8-credential-socket" pod "e5aaa21a-1a12-4b2b-b05e-f08504685cd8" (UID: "e5aaa21a-1a12-4b2b-b05e-f08504685cd8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:49:41.534902 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:41.534801    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5aaa21a-1a12-4b2b-b05e-f08504685cd8-workload-socket" pod "e5aaa21a-1a12-4b2b-b05e-f08504685cd8" (UID: "e5aaa21a-1a12-4b2b-b05e-f08504685cd8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:49:41.534902 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:41.534821    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5aaa21a-1a12-4b2b-b05e-f08504685cd8-workload-certs" pod "e5aaa21a-1a12-4b2b-b05e-f08504685cd8" (UID: "e5aaa21a-1a12-4b2b-b05e-f08504685cd8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:49:41.534902 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:41.534864    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5aaa21a-1a12-4b2b-b05e-f08504685cd8-istio-data" pod "e5aaa21a-1a12-4b2b-b05e-f08504685cd8" (UID: "e5aaa21a-1a12-4b2b-b05e-f08504685cd8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:49:41.535272 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:41.535246    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5aaa21a-1a12-4b2b-b05e-f08504685cd8-istiod-ca-cert" pod "e5aaa21a-1a12-4b2b-b05e-f08504685cd8" (UID: "e5aaa21a-1a12-4b2b-b05e-f08504685cd8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:49:41.535640 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:41.535615    6996 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5aaa21a-1a12-4b2b-b05e-f08504685cd8-credential-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:41.535780 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:41.535645    6996 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5aaa21a-1a12-4b2b-b05e-f08504685cd8-workload-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:41.535780 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:41.535660    6996 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5aaa21a-1a12-4b2b-b05e-f08504685cd8-workload-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:41.535780 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:41.535674    6996 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5aaa21a-1a12-4b2b-b05e-f08504685cd8-istio-data\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:41.535780 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:41.535689    6996 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5aaa21a-1a12-4b2b-b05e-f08504685cd8-istiod-ca-cert\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:41.536771 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:41.536745    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5aaa21a-1a12-4b2b-b05e-f08504685cd8-istio-token" pod "e5aaa21a-1a12-4b2b-b05e-f08504685cd8" (UID: "e5aaa21a-1a12-4b2b-b05e-f08504685cd8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:49:41.536895 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:41.536784    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5aaa21a-1a12-4b2b-b05e-f08504685cd8-kube-api-access-xddnz" pod "e5aaa21a-1a12-4b2b-b05e-f08504685cd8" (UID: "e5aaa21a-1a12-4b2b-b05e-f08504685cd8"). InnerVolumeSpecName "kube-api-access-xddnz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:49:41.536955 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:41.536891    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e5aaa21a-1a12-4b2b-b05e-f08504685cd8-istio-podinfo" pod "e5aaa21a-1a12-4b2b-b05e-f08504685cd8" (UID: "e5aaa21a-1a12-4b2b-b05e-f08504685cd8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:49:41.537063 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:41.537035    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5aaa21a-1a12-4b2b-b05e-f08504685cd8-istio-envoy" pod "e5aaa21a-1a12-4b2b-b05e-f08504685cd8" (UID: "e5aaa21a-1a12-4b2b-b05e-f08504685cd8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:49:41.637149 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:41.637118    6996 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5aaa21a-1a12-4b2b-b05e-f08504685cd8-istio-envoy\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:41.637149 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:41.637146    6996 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5aaa21a-1a12-4b2b-b05e-f08504685cd8-istio-podinfo\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:41.637149 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:41.637156    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xddnz\" (UniqueName: \"kubernetes.io/projected/e5aaa21a-1a12-4b2b-b05e-f08504685cd8-kube-api-access-xddnz\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:41.637360 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:41.637167    6996 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5aaa21a-1a12-4b2b-b05e-f08504685cd8-istio-token\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:41.826731 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:41.826699    6996 generic.go:356] "Generic (PLEG): container finished" podID="e5aaa21a-1a12-4b2b-b05e-f08504685cd8" containerID="f32b4fd714143e8bcef46c0c51985fe8b9b2765237d4cf55c5259b516c9af770" exitCode=0
Oct 30 19:49:41.827257 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:41.826787    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r"
Oct 30 19:49:41.827257 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:41.826917    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r" event={"ID":"e5aaa21a-1a12-4b2b-b05e-f08504685cd8","Type":"ContainerDied","Data":"f32b4fd714143e8bcef46c0c51985fe8b9b2765237d4cf55c5259b516c9af770"}
Oct 30 19:49:41.827257 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:41.826955    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r" event={"ID":"e5aaa21a-1a12-4b2b-b05e-f08504685cd8","Type":"ContainerDied","Data":"98e380ddc9a7677fc6bf5cdebfc44ed6107506f63ad0bed3268bd8df9c37b103"}
Oct 30 19:49:41.827257 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:41.826977    6996 scope.go:122] "RemoveContainer" containerID="f32b4fd714143e8bcef46c0c51985fe8b9b2765237d4cf55c5259b516c9af770"
Oct 30 19:49:41.843831 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:41.843809    6996 scope.go:122] "RemoveContainer" containerID="2607a1ffd03e61d227ab1169ca7c998fceec6b47285852a37afcafe1e231c02c"
Oct 30 19:49:41.854073 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:41.854051    6996 scope.go:122] "RemoveContainer" containerID="f32b4fd714143e8bcef46c0c51985fe8b9b2765237d4cf55c5259b516c9af770"
Oct 30 19:49:41.854378 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:49:41.854355    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f32b4fd714143e8bcef46c0c51985fe8b9b2765237d4cf55c5259b516c9af770\": container with ID starting with f32b4fd714143e8bcef46c0c51985fe8b9b2765237d4cf55c5259b516c9af770 not found: ID does not exist" containerID="f32b4fd714143e8bcef46c0c51985fe8b9b2765237d4cf55c5259b516c9af770"
Oct 30 19:49:41.854444 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:41.854387    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f32b4fd714143e8bcef46c0c51985fe8b9b2765237d4cf55c5259b516c9af770"} err="failed to get container status \"f32b4fd714143e8bcef46c0c51985fe8b9b2765237d4cf55c5259b516c9af770\": rpc error: code = NotFound desc = could not find container \"f32b4fd714143e8bcef46c0c51985fe8b9b2765237d4cf55c5259b516c9af770\": container with ID starting with f32b4fd714143e8bcef46c0c51985fe8b9b2765237d4cf55c5259b516c9af770 not found: ID does not exist"
Oct 30 19:49:41.854444 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:41.854406    6996 scope.go:122] "RemoveContainer" containerID="2607a1ffd03e61d227ab1169ca7c998fceec6b47285852a37afcafe1e231c02c"
Oct 30 19:49:41.854693 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:49:41.854677    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2607a1ffd03e61d227ab1169ca7c998fceec6b47285852a37afcafe1e231c02c\": container with ID starting with 2607a1ffd03e61d227ab1169ca7c998fceec6b47285852a37afcafe1e231c02c not found: ID does not exist" containerID="2607a1ffd03e61d227ab1169ca7c998fceec6b47285852a37afcafe1e231c02c"
Oct 30 19:49:41.854745 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:41.854698    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2607a1ffd03e61d227ab1169ca7c998fceec6b47285852a37afcafe1e231c02c"} err="failed to get container status \"2607a1ffd03e61d227ab1169ca7c998fceec6b47285852a37afcafe1e231c02c\": rpc error: code = NotFound desc = could not find container \"2607a1ffd03e61d227ab1169ca7c998fceec6b47285852a37afcafe1e231c02c\": container with ID starting with 2607a1ffd03e61d227ab1169ca7c998fceec6b47285852a37afcafe1e231c02c not found: ID does not exist"
Oct 30 19:49:41.856609 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:41.856588    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r"]
Oct 30 19:49:41.866427 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:41.866401    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-2761c7-openshift-default-5dc5fd95b7-x6m5r"]
Oct 30 19:49:42.793769 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:42.793734    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e5aaa21a-1a12-4b2b-b05e-f08504685cd8" path="/var/lib/kubelet/pods/e5aaa21a-1a12-4b2b-b05e-f08504685cd8/volumes"
Oct 30 19:49:49.611819 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:49.611782    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk"]
Oct 30 19:49:49.645104 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:49.645072    6996 cpu_manager.go:414] "containerMap: removing container" podUID="e5aaa21a-1a12-4b2b-b05e-f08504685cd8" containerName="istio-proxy"
Oct 30 19:49:49.645317 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:49.645301    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e5aaa21a-1a12-4b2b-b05e-f08504685cd8" containerName="istio-proxy"
Oct 30 19:49:49.645442 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:49.645431    6996 cpu_manager.go:414] "containerMap: removing container" podUID="e5aaa21a-1a12-4b2b-b05e-f08504685cd8" containerName="istio-proxy"
Oct 30 19:49:49.646654 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:49.646632    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e5aaa21a-1a12-4b2b-b05e-f08504685cd8" containerName="istio-proxy"
Oct 30 19:49:49.647101 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:49.647082    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="e5aaa21a-1a12-4b2b-b05e-f08504685cd8" containerName="istio-proxy"
Oct 30 19:49:49.656903 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:49.656844    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk"]
Oct 30 19:49:49.657037 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:49.657008    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk"
Oct 30 19:49:49.663921 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:49.663894    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-2beb7b-openshift-default-dockercfg-fxfnk\""
Oct 30 19:49:49.742840 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:49.742797    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9ce7283-f1bf-4afe-97dd-ad5f5ce14786-credential-socket\") pod \"e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk\" (UID: \"d9ce7283-f1bf-4afe-97dd-ad5f5ce14786\") " pod="openshift-ingress/e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk"
Oct 30 19:49:49.743055 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:49.742883    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9ce7283-f1bf-4afe-97dd-ad5f5ce14786-workload-certs\") pod \"e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk\" (UID: \"d9ce7283-f1bf-4afe-97dd-ad5f5ce14786\") " pod="openshift-ingress/e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk"
Oct 30 19:49:49.743055 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:49.742913    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9ce7283-f1bf-4afe-97dd-ad5f5ce14786-istio-podinfo\") pod \"e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk\" (UID: \"d9ce7283-f1bf-4afe-97dd-ad5f5ce14786\") " pod="openshift-ingress/e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk"
Oct 30 19:49:49.743055 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:49.742968    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9ce7283-f1bf-4afe-97dd-ad5f5ce14786-istio-token\") pod \"e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk\" (UID: \"d9ce7283-f1bf-4afe-97dd-ad5f5ce14786\") " pod="openshift-ingress/e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk"
Oct 30 19:49:49.743055 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:49.743017    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w76wx\" (UniqueName: \"kubernetes.io/projected/d9ce7283-f1bf-4afe-97dd-ad5f5ce14786-kube-api-access-w76wx\") pod \"e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk\" (UID: \"d9ce7283-f1bf-4afe-97dd-ad5f5ce14786\") " pod="openshift-ingress/e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk"
Oct 30 19:49:49.743290 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:49.743145    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9ce7283-f1bf-4afe-97dd-ad5f5ce14786-istio-envoy\") pod \"e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk\" (UID: \"d9ce7283-f1bf-4afe-97dd-ad5f5ce14786\") " pod="openshift-ingress/e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk"
Oct 30 19:49:49.743290 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:49.743251    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9ce7283-f1bf-4afe-97dd-ad5f5ce14786-istio-data\") pod \"e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk\" (UID: \"d9ce7283-f1bf-4afe-97dd-ad5f5ce14786\") " pod="openshift-ingress/e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk"
Oct 30 19:49:49.743290 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:49.743280    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9ce7283-f1bf-4afe-97dd-ad5f5ce14786-istiod-ca-cert\") pod \"e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk\" (UID: \"d9ce7283-f1bf-4afe-97dd-ad5f5ce14786\") " pod="openshift-ingress/e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk"
Oct 30 19:49:49.743449 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:49.743312    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9ce7283-f1bf-4afe-97dd-ad5f5ce14786-workload-socket\") pod \"e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk\" (UID: \"d9ce7283-f1bf-4afe-97dd-ad5f5ce14786\") " pod="openshift-ingress/e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk"
Oct 30 19:49:49.845313 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:49.844991    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9ce7283-f1bf-4afe-97dd-ad5f5ce14786-credential-socket\") pod \"e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk\" (UID: \"d9ce7283-f1bf-4afe-97dd-ad5f5ce14786\") " pod="openshift-ingress/e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk"
Oct 30 19:49:49.845313 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:49.845065    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9ce7283-f1bf-4afe-97dd-ad5f5ce14786-workload-certs\") pod \"e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk\" (UID: \"d9ce7283-f1bf-4afe-97dd-ad5f5ce14786\") " pod="openshift-ingress/e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk"
Oct 30 19:49:49.845313 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:49.845091    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9ce7283-f1bf-4afe-97dd-ad5f5ce14786-istio-podinfo\") pod \"e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk\" (UID: \"d9ce7283-f1bf-4afe-97dd-ad5f5ce14786\") " pod="openshift-ingress/e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk"
Oct 30 19:49:49.845313 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:49.845142    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9ce7283-f1bf-4afe-97dd-ad5f5ce14786-istio-token\") pod \"e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk\" (UID: \"d9ce7283-f1bf-4afe-97dd-ad5f5ce14786\") " pod="openshift-ingress/e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk"
Oct 30 19:49:49.845313 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:49.845181    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w76wx\" (UniqueName: \"kubernetes.io/projected/d9ce7283-f1bf-4afe-97dd-ad5f5ce14786-kube-api-access-w76wx\") pod \"e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk\" (UID: \"d9ce7283-f1bf-4afe-97dd-ad5f5ce14786\") " pod="openshift-ingress/e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk"
Oct 30 19:49:49.845313 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:49.845265    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9ce7283-f1bf-4afe-97dd-ad5f5ce14786-istio-envoy\") pod \"e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk\" (UID: \"d9ce7283-f1bf-4afe-97dd-ad5f5ce14786\") " pod="openshift-ingress/e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk"
Oct 30 19:49:49.845866 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:49.845370    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9ce7283-f1bf-4afe-97dd-ad5f5ce14786-istio-data\") pod \"e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk\" (UID: \"d9ce7283-f1bf-4afe-97dd-ad5f5ce14786\") " pod="openshift-ingress/e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk"
Oct 30 19:49:49.845866 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:49.845404    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9ce7283-f1bf-4afe-97dd-ad5f5ce14786-istiod-ca-cert\") pod \"e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk\" (UID: \"d9ce7283-f1bf-4afe-97dd-ad5f5ce14786\") " pod="openshift-ingress/e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk"
Oct 30 19:49:49.845866 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:49.845435    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9ce7283-f1bf-4afe-97dd-ad5f5ce14786-workload-socket\") pod \"e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk\" (UID: \"d9ce7283-f1bf-4afe-97dd-ad5f5ce14786\") " pod="openshift-ingress/e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk"
Oct 30 19:49:49.845866 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:49.845542    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9ce7283-f1bf-4afe-97dd-ad5f5ce14786-credential-socket\") pod \"e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk\" (UID: \"d9ce7283-f1bf-4afe-97dd-ad5f5ce14786\") " pod="openshift-ingress/e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk"
Oct 30 19:49:49.845866 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:49.845596    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9ce7283-f1bf-4afe-97dd-ad5f5ce14786-workload-certs\") pod \"e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk\" (UID: \"d9ce7283-f1bf-4afe-97dd-ad5f5ce14786\") " pod="openshift-ingress/e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk"
Oct 30 19:49:49.846389 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:49.846313    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9ce7283-f1bf-4afe-97dd-ad5f5ce14786-workload-socket\") pod \"e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk\" (UID: \"d9ce7283-f1bf-4afe-97dd-ad5f5ce14786\") " pod="openshift-ingress/e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk"
Oct 30 19:49:49.846880 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:49.846815    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9ce7283-f1bf-4afe-97dd-ad5f5ce14786-istiod-ca-cert\") pod \"e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk\" (UID: \"d9ce7283-f1bf-4afe-97dd-ad5f5ce14786\") " pod="openshift-ingress/e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk"
Oct 30 19:49:49.846980 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:49.846917    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9ce7283-f1bf-4afe-97dd-ad5f5ce14786-istio-data\") pod \"e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk\" (UID: \"d9ce7283-f1bf-4afe-97dd-ad5f5ce14786\") " pod="openshift-ingress/e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk"
Oct 30 19:49:49.850900 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:49.848841    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9ce7283-f1bf-4afe-97dd-ad5f5ce14786-istio-podinfo\") pod \"e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk\" (UID: \"d9ce7283-f1bf-4afe-97dd-ad5f5ce14786\") " pod="openshift-ingress/e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk"
Oct 30 19:49:49.850900 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:49.850223    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9ce7283-f1bf-4afe-97dd-ad5f5ce14786-istio-envoy\") pod \"e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk\" (UID: \"d9ce7283-f1bf-4afe-97dd-ad5f5ce14786\") " pod="openshift-ingress/e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk"
Oct 30 19:49:49.872146 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:49.872030    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9ce7283-f1bf-4afe-97dd-ad5f5ce14786-istio-token\") pod \"e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk\" (UID: \"d9ce7283-f1bf-4afe-97dd-ad5f5ce14786\") " pod="openshift-ingress/e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk"
Oct 30 19:49:49.876153 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:49.876093    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w76wx\" (UniqueName: \"kubernetes.io/projected/d9ce7283-f1bf-4afe-97dd-ad5f5ce14786-kube-api-access-w76wx\") pod \"e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk\" (UID: \"d9ce7283-f1bf-4afe-97dd-ad5f5ce14786\") " pod="openshift-ingress/e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk"
Oct 30 19:49:49.999807 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:49.999763    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk"
Oct 30 19:49:50.335399 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:50.335371    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk"]
Oct 30 19:49:50.336427 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:49:50.336403    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9ce7283_f1bf_4afe_97dd_ad5f5ce14786.slice/crio-f57f34124057e4afc50f7226930088784ada4b5984521a3ed8947013f783d3e0 WatchSource:0}: Error finding container f57f34124057e4afc50f7226930088784ada4b5984521a3ed8947013f783d3e0: Status 404 returned error can't find the container with id f57f34124057e4afc50f7226930088784ada4b5984521a3ed8947013f783d3e0
Oct 30 19:49:50.339446 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:50.339395    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:49:50.339590 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:50.339507    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:49:50.339590 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:50.339558    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:49:50.912559 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:50.912517    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk" event={"ID":"d9ce7283-f1bf-4afe-97dd-ad5f5ce14786","Type":"ContainerStarted","Data":"c1399355ad0d85b6f237444d6ddce78a326cf9c5da2d7bb70520f69f038fe8ee"}
Oct 30 19:49:50.913005 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:50.912581    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk" event={"ID":"d9ce7283-f1bf-4afe-97dd-ad5f5ce14786","Type":"ContainerStarted","Data":"f57f34124057e4afc50f7226930088784ada4b5984521a3ed8947013f783d3e0"}
Oct 30 19:49:50.950904 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:50.950847    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk" podStartSLOduration=1.950834418 podStartE2EDuration="1.950834418s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:49:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:49:50.94770499 +0000 UTC m=+1066.275823144" watchObservedRunningTime="2026-10-30 19:49:50.950834418 +0000 UTC m=+1066.278952572"
Oct 30 19:49:51.001072 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:51.001047    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk"
Oct 30 19:49:51.007141 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:51.007118    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk"
Oct 30 19:49:51.837503 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:51.834402    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-040623-577cc456d-sf9nq"
Oct 30 19:49:51.922584 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:51.922534    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk"
Oct 30 19:49:51.923603 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:51.923583    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk"
Oct 30 19:49:55.103900 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:55.103859    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-2beb7b-6f9c4fb479-5cpxb"]
Oct 30 19:49:55.104865 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:55.104840    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="e5aaa21a-1a12-4b2b-b05e-f08504685cd8" containerName="istio-proxy"
Oct 30 19:49:55.108767 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:55.108747    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-2beb7b-6f9c4fb479-5cpxb"
Oct 30 19:49:55.118546 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:55.118521    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-2beb7b\""
Oct 30 19:49:55.124030 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:55.124005    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-2beb7b-6f9c4fb479-5cpxb"]
Oct 30 19:49:55.187604 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:55.187566    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8937dc08-19b1-4247-a174-e80fc16b1fe3-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-2beb7b-6f9c4fb479-5cpxb\" (UID: \"8937dc08-19b1-4247-a174-e80fc16b1fe3\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-2beb7b-6f9c4fb479-5cpxb"
Oct 30 19:49:55.187786 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:55.187622    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97lz8\" (UniqueName: \"kubernetes.io/projected/8937dc08-19b1-4247-a174-e80fc16b1fe3-kube-api-access-97lz8\") pod \"payload-pre-processing-e2e-api-b-2beb7b-6f9c4fb479-5cpxb\" (UID: \"8937dc08-19b1-4247-a174-e80fc16b1fe3\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-2beb7b-6f9c4fb479-5cpxb"
Oct 30 19:49:55.292504 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:55.291839    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8937dc08-19b1-4247-a174-e80fc16b1fe3-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-2beb7b-6f9c4fb479-5cpxb\" (UID: \"8937dc08-19b1-4247-a174-e80fc16b1fe3\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-2beb7b-6f9c4fb479-5cpxb"
Oct 30 19:49:55.292504 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:55.291910    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97lz8\" (UniqueName: \"kubernetes.io/projected/8937dc08-19b1-4247-a174-e80fc16b1fe3-kube-api-access-97lz8\") pod \"payload-pre-processing-e2e-api-b-2beb7b-6f9c4fb479-5cpxb\" (UID: \"8937dc08-19b1-4247-a174-e80fc16b1fe3\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-2beb7b-6f9c4fb479-5cpxb"
Oct 30 19:49:55.298531 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:55.293423    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8937dc08-19b1-4247-a174-e80fc16b1fe3-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-2beb7b-6f9c4fb479-5cpxb\" (UID: \"8937dc08-19b1-4247-a174-e80fc16b1fe3\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-2beb7b-6f9c4fb479-5cpxb"
Oct 30 19:49:55.359993 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:55.359907    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-97lz8\" (UniqueName: \"kubernetes.io/projected/8937dc08-19b1-4247-a174-e80fc16b1fe3-kube-api-access-97lz8\") pod \"payload-pre-processing-e2e-api-b-2beb7b-6f9c4fb479-5cpxb\" (UID: \"8937dc08-19b1-4247-a174-e80fc16b1fe3\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-2beb7b-6f9c4fb479-5cpxb"
Oct 30 19:49:55.430792 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:55.430748    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-2beb7b-6f9c4fb479-5cpxb"
Oct 30 19:49:55.773257 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:55.773228    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-2beb7b-6f9c4fb479-5cpxb"]
Oct 30 19:49:55.775079 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:49:55.775048    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8937dc08_19b1_4247_a174_e80fc16b1fe3.slice/crio-7b7ce6e3ee86cd24c774a4aaac4374e6db9f3025d81e7dad31b18ed9cc935794 WatchSource:0}: Error finding container 7b7ce6e3ee86cd24c774a4aaac4374e6db9f3025d81e7dad31b18ed9cc935794: Status 404 returned error can't find the container with id 7b7ce6e3ee86cd24c774a4aaac4374e6db9f3025d81e7dad31b18ed9cc935794
Oct 30 19:49:55.997805 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:55.997762    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-2beb7b-6f9c4fb479-5cpxb" event={"ID":"8937dc08-19b1-4247-a174-e80fc16b1fe3","Type":"ContainerStarted","Data":"a450e880007c12730f558c92a8d335bec4d00045b0aee58679a426ec58517f83"}
Oct 30 19:49:55.997970 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:55.997828    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-2beb7b-6f9c4fb479-5cpxb" event={"ID":"8937dc08-19b1-4247-a174-e80fc16b1fe3","Type":"ContainerStarted","Data":"7b7ce6e3ee86cd24c774a4aaac4374e6db9f3025d81e7dad31b18ed9cc935794"}
Oct 30 19:49:55.997970 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:55.997851    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-2beb7b-6f9c4fb479-5cpxb"
Oct 30 19:49:56.022309 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:56.022205    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-2beb7b-6f9c4fb479-5cpxb" podStartSLOduration=1.022192132 podStartE2EDuration="1.022192132s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:49:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:49:56.020057568 +0000 UTC m=+1071.348175722" watchObservedRunningTime="2026-10-30 19:49:56.022192132 +0000 UTC m=+1071.350310357"
Oct 30 19:49:56.074353 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:56.074316    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-e907e4-7547657bb4-s8x4l"]
Oct 30 19:49:56.078915 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:56.078894    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e907e4-7547657bb4-s8x4l"
Oct 30 19:49:56.091915 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:56.091875    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-e907e4-serving-cert\""
Oct 30 19:49:56.098125 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:56.098101    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-e907e4-7547657bb4-s8x4l"]
Oct 30 19:49:56.210035 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:56.210002    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ffc6067-3cf9-4a43-8a7c-3b210debc122-openshift-service-ca\") pod \"maas-api-e2e-api-a-e907e4-7547657bb4-s8x4l\" (UID: \"8ffc6067-3cf9-4a43-8a7c-3b210debc122\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e907e4-7547657bb4-s8x4l"
Oct 30 19:49:56.210455 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:56.210163    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gf4cn\" (UniqueName: \"kubernetes.io/projected/8ffc6067-3cf9-4a43-8a7c-3b210debc122-kube-api-access-gf4cn\") pod \"maas-api-e2e-api-a-e907e4-7547657bb4-s8x4l\" (UID: \"8ffc6067-3cf9-4a43-8a7c-3b210debc122\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e907e4-7547657bb4-s8x4l"
Oct 30 19:49:56.210455 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:56.210293    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8ffc6067-3cf9-4a43-8a7c-3b210debc122-metrics-certs\") pod \"maas-api-e2e-api-a-e907e4-7547657bb4-s8x4l\" (UID: \"8ffc6067-3cf9-4a43-8a7c-3b210debc122\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e907e4-7547657bb4-s8x4l"
Oct 30 19:49:56.210455 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:56.210339    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ffc6067-3cf9-4a43-8a7c-3b210debc122-maas-api-tls\") pod \"maas-api-e2e-api-a-e907e4-7547657bb4-s8x4l\" (UID: \"8ffc6067-3cf9-4a43-8a7c-3b210debc122\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e907e4-7547657bb4-s8x4l"
Oct 30 19:49:56.311903 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:56.311805    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8ffc6067-3cf9-4a43-8a7c-3b210debc122-metrics-certs\") pod \"maas-api-e2e-api-a-e907e4-7547657bb4-s8x4l\" (UID: \"8ffc6067-3cf9-4a43-8a7c-3b210debc122\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e907e4-7547657bb4-s8x4l"
Oct 30 19:49:56.311903 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:56.311889    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ffc6067-3cf9-4a43-8a7c-3b210debc122-maas-api-tls\") pod \"maas-api-e2e-api-a-e907e4-7547657bb4-s8x4l\" (UID: \"8ffc6067-3cf9-4a43-8a7c-3b210debc122\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e907e4-7547657bb4-s8x4l"
Oct 30 19:49:56.312142 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:56.312004    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ffc6067-3cf9-4a43-8a7c-3b210debc122-openshift-service-ca\") pod \"maas-api-e2e-api-a-e907e4-7547657bb4-s8x4l\" (UID: \"8ffc6067-3cf9-4a43-8a7c-3b210debc122\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e907e4-7547657bb4-s8x4l"
Oct 30 19:49:56.312202 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:56.312150    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gf4cn\" (UniqueName: \"kubernetes.io/projected/8ffc6067-3cf9-4a43-8a7c-3b210debc122-kube-api-access-gf4cn\") pod \"maas-api-e2e-api-a-e907e4-7547657bb4-s8x4l\" (UID: \"8ffc6067-3cf9-4a43-8a7c-3b210debc122\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e907e4-7547657bb4-s8x4l"
Oct 30 19:49:56.312727 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:56.312702    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ffc6067-3cf9-4a43-8a7c-3b210debc122-openshift-service-ca\") pod \"maas-api-e2e-api-a-e907e4-7547657bb4-s8x4l\" (UID: \"8ffc6067-3cf9-4a43-8a7c-3b210debc122\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e907e4-7547657bb4-s8x4l"
Oct 30 19:49:56.315133 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:56.315103    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8ffc6067-3cf9-4a43-8a7c-3b210debc122-metrics-certs\") pod \"maas-api-e2e-api-a-e907e4-7547657bb4-s8x4l\" (UID: \"8ffc6067-3cf9-4a43-8a7c-3b210debc122\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e907e4-7547657bb4-s8x4l"
Oct 30 19:49:56.318035 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:56.318012    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ffc6067-3cf9-4a43-8a7c-3b210debc122-maas-api-tls\") pod \"maas-api-e2e-api-a-e907e4-7547657bb4-s8x4l\" (UID: \"8ffc6067-3cf9-4a43-8a7c-3b210debc122\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e907e4-7547657bb4-s8x4l"
Oct 30 19:49:56.326090 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:56.326065    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gf4cn\" (UniqueName: \"kubernetes.io/projected/8ffc6067-3cf9-4a43-8a7c-3b210debc122-kube-api-access-gf4cn\") pod \"maas-api-e2e-api-a-e907e4-7547657bb4-s8x4l\" (UID: \"8ffc6067-3cf9-4a43-8a7c-3b210debc122\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e907e4-7547657bb4-s8x4l"
Oct 30 19:49:56.400627 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:56.400589    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e907e4-7547657bb4-s8x4l"
Oct 30 19:49:56.710522 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:56.710447    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-e907e4-7547657bb4-s8x4l"]
Oct 30 19:49:56.714657 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:49:56.714616    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ffc6067_3cf9_4a43_8a7c_3b210debc122.slice/crio-6bc36468981adc59929f8e8ed5da11c6bcb865cac181d4fcebf2147d51494c08 WatchSource:0}: Error finding container 6bc36468981adc59929f8e8ed5da11c6bcb865cac181d4fcebf2147d51494c08: Status 404 returned error can't find the container with id 6bc36468981adc59929f8e8ed5da11c6bcb865cac181d4fcebf2147d51494c08
Oct 30 19:49:57.010722 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:57.010626    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e907e4-7547657bb4-s8x4l" event={"ID":"8ffc6067-3cf9-4a43-8a7c-3b210debc122","Type":"ContainerStarted","Data":"6bc36468981adc59929f8e8ed5da11c6bcb865cac181d4fcebf2147d51494c08"}
Oct 30 19:49:58.021326 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:58.021245    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e907e4-7547657bb4-s8x4l" event={"ID":"8ffc6067-3cf9-4a43-8a7c-3b210debc122","Type":"ContainerStarted","Data":"aa24a7c186e4b726b9f5ef6ae231ee1be6dd0c40bd4ec1d6305fab391f6cc480"}
Oct 30 19:49:58.021789 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:58.021420    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e907e4-7547657bb4-s8x4l"
Oct 30 19:49:58.043270 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:49:58.043214    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e907e4-7547657bb4-s8x4l" podStartSLOduration=1.537607431 podStartE2EDuration="2.043202015s" totalImagesPullingTime="505.594584ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:49:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:49:58.040851256 +0000 UTC m=+1073.368969410" watchObservedRunningTime="2026-10-30 19:49:58.043202015 +0000 UTC m=+1073.371320168"
Oct 30 19:50:01.047652 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:01.047617    6996 generic.go:356] "Generic (PLEG): container finished" podID="e6d36c63-b64c-49e6-8c9e-7e939e777275" containerID="c673f266600df1d3edc7064e0f0b7a22751b4e27d117c7e3f16a5b1e64c2ab80" exitCode=0
Oct 30 19:50:01.048088 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:01.047691    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc" event={"ID":"e6d36c63-b64c-49e6-8c9e-7e939e777275","Type":"ContainerDied","Data":"c673f266600df1d3edc7064e0f0b7a22751b4e27d117c7e3f16a5b1e64c2ab80"}
Oct 30 19:50:01.048175 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:01.048161    6996 scope.go:122] "RemoveContainer" containerID="c673f266600df1d3edc7064e0f0b7a22751b4e27d117c7e3f16a5b1e64c2ab80"
Oct 30 19:50:01.049931 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:01.049899    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:50:01.050044 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:01.049996    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:50:01.050108 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:01.050054    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:50:01.205339 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:50:01.205147    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 19:50:01.520613 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:01.520574    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc"
Oct 30 19:50:02.070749 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:02.070706    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc" event={"ID":"e6d36c63-b64c-49e6-8c9e-7e939e777275","Type":"ContainerStarted","Data":"0d6ae7a1560e697ae216988b305c6caf28670c098d7d263200b1925c2ffdd222"}
Oct 30 19:50:02.309419 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:02.309375    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc"
Oct 30 19:50:02.311523 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:02.311446    6996 patch_prober.go:29] interesting pod/e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.115:15021/healthz/ready\": dial tcp 10.134.0.115:15021: connect: connection refused" start-of-body=
Oct 30 19:50:02.311692 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:02.311581    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc" podUID="e6d36c63-b64c-49e6-8c9e-7e939e777275" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.115:15021/healthz/ready\": dial tcp 10.134.0.115:15021: connect: connection refused"
Oct 30 19:50:03.309583 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:03.309551    6996 patch_prober.go:29] interesting pod/e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.115:15021/healthz/ready\": dial tcp 10.134.0.115:15021: connect: connection refused" start-of-body=
Oct 30 19:50:03.309973 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:03.309606    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc" podUID="e6d36c63-b64c-49e6-8c9e-7e939e777275" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.115:15021/healthz/ready\": dial tcp 10.134.0.115:15021: connect: connection refused"
Oct 30 19:50:04.035250 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:04.035213    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e907e4-7547657bb4-s8x4l"
Oct 30 19:50:04.310225 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:04.310138    6996 patch_prober.go:29] interesting pod/e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.115:15021/healthz/ready\": dial tcp 10.134.0.115:15021: connect: connection refused" start-of-body=
Oct 30 19:50:04.310831 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:04.310799    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc" podUID="e6d36c63-b64c-49e6-8c9e-7e939e777275" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.115:15021/healthz/ready\": dial tcp 10.134.0.115:15021: connect: connection refused"
Oct 30 19:50:05.309441 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:05.309405    6996 patch_prober.go:29] interesting pod/e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.115:15021/healthz/ready\": dial tcp 10.134.0.115:15021: connect: connection refused" start-of-body=
Oct 30 19:50:05.309643 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:05.309468    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc" podUID="e6d36c63-b64c-49e6-8c9e-7e939e777275" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.115:15021/healthz/ready\": dial tcp 10.134.0.115:15021: connect: connection refused"
Oct 30 19:50:06.309038 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:06.308994    6996 patch_prober.go:29] interesting pod/e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.115:15021/healthz/ready\": dial tcp 10.134.0.115:15021: connect: connection refused" start-of-body=
Oct 30 19:50:06.309578 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:06.309071    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc" podUID="e6d36c63-b64c-49e6-8c9e-7e939e777275" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.115:15021/healthz/ready\": dial tcp 10.134.0.115:15021: connect: connection refused"
Oct 30 19:50:07.014340 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:07.014308    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-2beb7b-6f9c4fb479-5cpxb"
Oct 30 19:50:07.313911 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:07.313831    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc"
Oct 30 19:50:07.314361 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:07.314340    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc"
Oct 30 19:50:07.314892 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:07.314876    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc"
Oct 30 19:50:09.256362 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:09.256321    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-2beb7b-59c9b6c955-qjwcz"]
Oct 30 19:50:09.260996 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:09.260968    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2beb7b-59c9b6c955-qjwcz"
Oct 30 19:50:09.264220 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:09.264192    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-2beb7b-serving-cert\""
Oct 30 19:50:09.273096 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:09.273071    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-2beb7b-59c9b6c955-qjwcz"]
Oct 30 19:50:09.316843 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:09.316811    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5988db4-cc38-4cc9-ac41-cc3abbf865bb-openshift-service-ca\") pod \"maas-api-e2e-api-b-2beb7b-59c9b6c955-qjwcz\" (UID: \"e5988db4-cc38-4cc9-ac41-cc3abbf865bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2beb7b-59c9b6c955-qjwcz"
Oct 30 19:50:09.317056 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:09.317029    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5988db4-cc38-4cc9-ac41-cc3abbf865bb-maas-api-tls\") pod \"maas-api-e2e-api-b-2beb7b-59c9b6c955-qjwcz\" (UID: \"e5988db4-cc38-4cc9-ac41-cc3abbf865bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2beb7b-59c9b6c955-qjwcz"
Oct 30 19:50:09.317144 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:09.317123    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e5988db4-cc38-4cc9-ac41-cc3abbf865bb-metrics-certs\") pod \"maas-api-e2e-api-b-2beb7b-59c9b6c955-qjwcz\" (UID: \"e5988db4-cc38-4cc9-ac41-cc3abbf865bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2beb7b-59c9b6c955-qjwcz"
Oct 30 19:50:09.317256 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:09.317240    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bvg2\" (UniqueName: \"kubernetes.io/projected/e5988db4-cc38-4cc9-ac41-cc3abbf865bb-kube-api-access-2bvg2\") pod \"maas-api-e2e-api-b-2beb7b-59c9b6c955-qjwcz\" (UID: \"e5988db4-cc38-4cc9-ac41-cc3abbf865bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2beb7b-59c9b6c955-qjwcz"
Oct 30 19:50:09.418138 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:09.418087    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5988db4-cc38-4cc9-ac41-cc3abbf865bb-maas-api-tls\") pod \"maas-api-e2e-api-b-2beb7b-59c9b6c955-qjwcz\" (UID: \"e5988db4-cc38-4cc9-ac41-cc3abbf865bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2beb7b-59c9b6c955-qjwcz"
Oct 30 19:50:09.418138 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:09.418147    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e5988db4-cc38-4cc9-ac41-cc3abbf865bb-metrics-certs\") pod \"maas-api-e2e-api-b-2beb7b-59c9b6c955-qjwcz\" (UID: \"e5988db4-cc38-4cc9-ac41-cc3abbf865bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2beb7b-59c9b6c955-qjwcz"
Oct 30 19:50:09.418354 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:09.418210    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bvg2\" (UniqueName: \"kubernetes.io/projected/e5988db4-cc38-4cc9-ac41-cc3abbf865bb-kube-api-access-2bvg2\") pod \"maas-api-e2e-api-b-2beb7b-59c9b6c955-qjwcz\" (UID: \"e5988db4-cc38-4cc9-ac41-cc3abbf865bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2beb7b-59c9b6c955-qjwcz"
Oct 30 19:50:09.418354 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:50:09.418248    6996 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-2beb7b-serving-cert: secret "maas-api-e2e-api-b-2beb7b-serving-cert" not found
Oct 30 19:50:09.418354 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:50:09.418307    6996 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5988db4-cc38-4cc9-ac41-cc3abbf865bb-maas-api-tls podName:e5988db4-cc38-4cc9-ac41-cc3abbf865bb nodeName:}" failed. No retries permitted until 2026-10-30 19:50:09.918291839 +0000 UTC m=+1085.246409987 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e5988db4-cc38-4cc9-ac41-cc3abbf865bb-maas-api-tls") pod "maas-api-e2e-api-b-2beb7b-59c9b6c955-qjwcz" (UID: "e5988db4-cc38-4cc9-ac41-cc3abbf865bb") : secret "maas-api-e2e-api-b-2beb7b-serving-cert" not found
Oct 30 19:50:09.418584 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:09.418375    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5988db4-cc38-4cc9-ac41-cc3abbf865bb-openshift-service-ca\") pod \"maas-api-e2e-api-b-2beb7b-59c9b6c955-qjwcz\" (UID: \"e5988db4-cc38-4cc9-ac41-cc3abbf865bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2beb7b-59c9b6c955-qjwcz"
Oct 30 19:50:09.419072 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:09.419046    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5988db4-cc38-4cc9-ac41-cc3abbf865bb-openshift-service-ca\") pod \"maas-api-e2e-api-b-2beb7b-59c9b6c955-qjwcz\" (UID: \"e5988db4-cc38-4cc9-ac41-cc3abbf865bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2beb7b-59c9b6c955-qjwcz"
Oct 30 19:50:09.420625 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:09.420607    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e5988db4-cc38-4cc9-ac41-cc3abbf865bb-metrics-certs\") pod \"maas-api-e2e-api-b-2beb7b-59c9b6c955-qjwcz\" (UID: \"e5988db4-cc38-4cc9-ac41-cc3abbf865bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2beb7b-59c9b6c955-qjwcz"
Oct 30 19:50:09.433752 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:09.433726    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bvg2\" (UniqueName: \"kubernetes.io/projected/e5988db4-cc38-4cc9-ac41-cc3abbf865bb-kube-api-access-2bvg2\") pod \"maas-api-e2e-api-b-2beb7b-59c9b6c955-qjwcz\" (UID: \"e5988db4-cc38-4cc9-ac41-cc3abbf865bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2beb7b-59c9b6c955-qjwcz"
Oct 30 19:50:09.932619 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:09.932585    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5988db4-cc38-4cc9-ac41-cc3abbf865bb-maas-api-tls\") pod \"maas-api-e2e-api-b-2beb7b-59c9b6c955-qjwcz\" (UID: \"e5988db4-cc38-4cc9-ac41-cc3abbf865bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2beb7b-59c9b6c955-qjwcz"
Oct 30 19:50:09.935088 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:09.935065    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5988db4-cc38-4cc9-ac41-cc3abbf865bb-maas-api-tls\") pod \"maas-api-e2e-api-b-2beb7b-59c9b6c955-qjwcz\" (UID: \"e5988db4-cc38-4cc9-ac41-cc3abbf865bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2beb7b-59c9b6c955-qjwcz"
Oct 30 19:50:10.176596 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:10.176565    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2beb7b-59c9b6c955-qjwcz"
Oct 30 19:50:10.498073 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:10.497947    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-2beb7b-59c9b6c955-qjwcz"]
Oct 30 19:50:10.511550 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:50:10.503762    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5988db4_cc38_4cc9_ac41_cc3abbf865bb.slice/crio-a02ae05329f9f4bf2fd37bbd94c0bca20e0bd200677cef06112c85c417a7db14 WatchSource:0}: Error finding container a02ae05329f9f4bf2fd37bbd94c0bca20e0bd200677cef06112c85c417a7db14: Status 404 returned error can't find the container with id a02ae05329f9f4bf2fd37bbd94c0bca20e0bd200677cef06112c85c417a7db14
Oct 30 19:50:11.246158 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:11.246074    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2beb7b-59c9b6c955-qjwcz" event={"ID":"e5988db4-cc38-4cc9-ac41-cc3abbf865bb","Type":"ContainerStarted","Data":"9aea1f57d8f5bed49197e1c037ed5a36b3316df2d200ed0e940e25c476410c8a"}
Oct 30 19:50:11.246158 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:11.246126    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2beb7b-59c9b6c955-qjwcz" event={"ID":"e5988db4-cc38-4cc9-ac41-cc3abbf865bb","Type":"ContainerStarted","Data":"a02ae05329f9f4bf2fd37bbd94c0bca20e0bd200677cef06112c85c417a7db14"}
Oct 30 19:50:11.246384 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:11.246210    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2beb7b-59c9b6c955-qjwcz"
Oct 30 19:50:11.280159 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:11.280113    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2beb7b-59c9b6c955-qjwcz" podStartSLOduration=1.860223596 podStartE2EDuration="2.280101588s" totalImagesPullingTime="419.877992ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:50:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:50:11.276227377 +0000 UTC m=+1086.604345530" watchObservedRunningTime="2026-10-30 19:50:11.280101588 +0000 UTC m=+1086.608219742"
Oct 30 19:50:12.568822 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:12.568786    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 19:50:13.378502 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:13.378313    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-040623-577cc456d-sf9nq"]
Oct 30 19:50:17.259364 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:17.259263    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2beb7b-59c9b6c955-qjwcz"
Oct 30 19:50:18.379583 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:18.379520    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-040623-577cc456d-sf9nq" podUID="cc327913-37b2-489d-8916-8707d6b5561a" containerName="payload-processing" containerID="cri-o://bfa586a236566615d400d6172cec16daefcf5e8b985f990e61659735cf3de186" gracePeriod=25
Oct 30 19:50:18.663703 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:18.663673    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-040623-577cc456d-sf9nq"
Oct 30 19:50:18.790354 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:18.790316    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cc327913-37b2-489d-8916-8707d6b5561a-kube-api-access-p54vl\" (UniqueName: \"kubernetes.io/projected/cc327913-37b2-489d-8916-8707d6b5561a-kube-api-access-p54vl\") pod \"cc327913-37b2-489d-8916-8707d6b5561a\" (UID: \"cc327913-37b2-489d-8916-8707d6b5561a\") "
Oct 30 19:50:18.790739 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:18.790717    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cc327913-37b2-489d-8916-8707d6b5561a-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cc327913-37b2-489d-8916-8707d6b5561a-plugins-config-volume\") pod \"cc327913-37b2-489d-8916-8707d6b5561a\" (UID: \"cc327913-37b2-489d-8916-8707d6b5561a\") "
Oct 30 19:50:18.791162 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:18.791126    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc327913-37b2-489d-8916-8707d6b5561a-plugins-config-volume" pod "cc327913-37b2-489d-8916-8707d6b5561a" (UID: "cc327913-37b2-489d-8916-8707d6b5561a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:50:18.791553 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:18.791510    6996 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cc327913-37b2-489d-8916-8707d6b5561a-plugins-config-volume\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:50:18.793075 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:18.793031    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc327913-37b2-489d-8916-8707d6b5561a-kube-api-access-p54vl" pod "cc327913-37b2-489d-8916-8707d6b5561a" (UID: "cc327913-37b2-489d-8916-8707d6b5561a"). InnerVolumeSpecName "kube-api-access-p54vl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:50:18.894554 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:18.893764    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p54vl\" (UniqueName: \"kubernetes.io/projected/cc327913-37b2-489d-8916-8707d6b5561a-kube-api-access-p54vl\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:50:19.315146 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:19.315058    6996 generic.go:356] "Generic (PLEG): container finished" podID="cc327913-37b2-489d-8916-8707d6b5561a" containerID="bfa586a236566615d400d6172cec16daefcf5e8b985f990e61659735cf3de186" exitCode=0
Oct 30 19:50:19.315146 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:19.315109    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-040623-577cc456d-sf9nq"
Oct 30 19:50:19.315332 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:19.315142    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-040623-577cc456d-sf9nq" event={"ID":"cc327913-37b2-489d-8916-8707d6b5561a","Type":"ContainerDied","Data":"bfa586a236566615d400d6172cec16daefcf5e8b985f990e61659735cf3de186"}
Oct 30 19:50:19.315332 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:19.315200    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-040623-577cc456d-sf9nq" event={"ID":"cc327913-37b2-489d-8916-8707d6b5561a","Type":"ContainerDied","Data":"e481d22b61c0eb4e17e796b3c6e92bb66509018909959e24c55b7da1bef8a8af"}
Oct 30 19:50:19.315332 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:19.315226    6996 scope.go:122] "RemoveContainer" containerID="bfa586a236566615d400d6172cec16daefcf5e8b985f990e61659735cf3de186"
Oct 30 19:50:19.328423 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:19.328401    6996 scope.go:122] "RemoveContainer" containerID="bfa586a236566615d400d6172cec16daefcf5e8b985f990e61659735cf3de186"
Oct 30 19:50:19.328957 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:50:19.328930    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bfa586a236566615d400d6172cec16daefcf5e8b985f990e61659735cf3de186\": container with ID starting with bfa586a236566615d400d6172cec16daefcf5e8b985f990e61659735cf3de186 not found: ID does not exist" containerID="bfa586a236566615d400d6172cec16daefcf5e8b985f990e61659735cf3de186"
Oct 30 19:50:19.329073 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:19.328964    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bfa586a236566615d400d6172cec16daefcf5e8b985f990e61659735cf3de186"} err="failed to get container status \"bfa586a236566615d400d6172cec16daefcf5e8b985f990e61659735cf3de186\": rpc error: code = NotFound desc = could not find container \"bfa586a236566615d400d6172cec16daefcf5e8b985f990e61659735cf3de186\": container with ID starting with bfa586a236566615d400d6172cec16daefcf5e8b985f990e61659735cf3de186 not found: ID does not exist"
Oct 30 19:50:19.352030 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:19.351994    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-040623-577cc456d-sf9nq"]
Oct 30 19:50:19.373078 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:19.373054    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-040623-577cc456d-sf9nq"]
Oct 30 19:50:20.793377 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:20.793339    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cc327913-37b2-489d-8916-8707d6b5561a" path="/var/lib/kubelet/pods/cc327913-37b2-489d-8916-8707d6b5561a/volumes"
Oct 30 19:50:26.051144 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:26.051110    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-f61e4b-7c54c4f9f5-gv9x9"]
Oct 30 19:50:26.051817 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:26.051802    6996 cpu_manager.go:414] "containerMap: removing container" podUID="cc327913-37b2-489d-8916-8707d6b5561a" containerName="payload-processing"
Oct 30 19:50:26.051877 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:26.051818    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cc327913-37b2-489d-8916-8707d6b5561a" containerName="payload-processing"
Oct 30 19:50:26.051930 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:26.051921    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="cc327913-37b2-489d-8916-8707d6b5561a" containerName="payload-processing"
Oct 30 19:50:26.055467 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:26.055448    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f61e4b-7c54c4f9f5-gv9x9"
Oct 30 19:50:26.060788 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:26.060759    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-f61e4b-dockercfg-kfpxn\""
Oct 30 19:50:26.067096 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:26.067078    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-f61e4b\""
Oct 30 19:50:26.077765 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:26.077738    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-f61e4b-7c54c4f9f5-gv9x9"]
Oct 30 19:50:26.118690 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:26.118647    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-f61e4b-66959bc884-hmnz9"]
Oct 30 19:50:26.128464 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:26.127591    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-f61e4b-66959bc884-hmnz9"
Oct 30 19:50:26.131777 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:26.131735    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-f61e4b-66959bc884-hmnz9"]
Oct 30 19:50:26.237540 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:26.237182    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3eafcd7e-b8d7-4f3b-a483-c2c7dc9b764a-plugins-config-volume\") pod \"payload-processing-e2e-api-b-f61e4b-66959bc884-hmnz9\" (UID: \"3eafcd7e-b8d7-4f3b-a483-c2c7dc9b764a\") " pod="openshift-ingress/payload-processing-e2e-api-b-f61e4b-66959bc884-hmnz9"
Oct 30 19:50:26.237540 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:26.237254    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zcg99\" (UniqueName: \"kubernetes.io/projected/7e524e27-0d0d-42db-b376-0c488d6a0622-kube-api-access-zcg99\") pod \"payload-pre-processing-e2e-api-b-f61e4b-7c54c4f9f5-gv9x9\" (UID: \"7e524e27-0d0d-42db-b376-0c488d6a0622\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-f61e4b-7c54c4f9f5-gv9x9"
Oct 30 19:50:26.237540 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:26.237290    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgnw6\" (UniqueName: \"kubernetes.io/projected/3eafcd7e-b8d7-4f3b-a483-c2c7dc9b764a-kube-api-access-dgnw6\") pod \"payload-processing-e2e-api-b-f61e4b-66959bc884-hmnz9\" (UID: \"3eafcd7e-b8d7-4f3b-a483-c2c7dc9b764a\") " pod="openshift-ingress/payload-processing-e2e-api-b-f61e4b-66959bc884-hmnz9"
Oct 30 19:50:26.237540 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:26.237511    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7e524e27-0d0d-42db-b376-0c488d6a0622-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-f61e4b-7c54c4f9f5-gv9x9\" (UID: \"7e524e27-0d0d-42db-b376-0c488d6a0622\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-f61e4b-7c54c4f9f5-gv9x9"
Oct 30 19:50:26.353502 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:26.347971    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7e524e27-0d0d-42db-b376-0c488d6a0622-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-f61e4b-7c54c4f9f5-gv9x9\" (UID: \"7e524e27-0d0d-42db-b376-0c488d6a0622\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-f61e4b-7c54c4f9f5-gv9x9"
Oct 30 19:50:26.353502 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:26.348216    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3eafcd7e-b8d7-4f3b-a483-c2c7dc9b764a-plugins-config-volume\") pod \"payload-processing-e2e-api-b-f61e4b-66959bc884-hmnz9\" (UID: \"3eafcd7e-b8d7-4f3b-a483-c2c7dc9b764a\") " pod="openshift-ingress/payload-processing-e2e-api-b-f61e4b-66959bc884-hmnz9"
Oct 30 19:50:26.353502 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:26.348275    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcg99\" (UniqueName: \"kubernetes.io/projected/7e524e27-0d0d-42db-b376-0c488d6a0622-kube-api-access-zcg99\") pod \"payload-pre-processing-e2e-api-b-f61e4b-7c54c4f9f5-gv9x9\" (UID: \"7e524e27-0d0d-42db-b376-0c488d6a0622\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-f61e4b-7c54c4f9f5-gv9x9"
Oct 30 19:50:26.353502 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:26.348313    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgnw6\" (UniqueName: \"kubernetes.io/projected/3eafcd7e-b8d7-4f3b-a483-c2c7dc9b764a-kube-api-access-dgnw6\") pod \"payload-processing-e2e-api-b-f61e4b-66959bc884-hmnz9\" (UID: \"3eafcd7e-b8d7-4f3b-a483-c2c7dc9b764a\") " pod="openshift-ingress/payload-processing-e2e-api-b-f61e4b-66959bc884-hmnz9"
Oct 30 19:50:26.353502 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:26.349758    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7e524e27-0d0d-42db-b376-0c488d6a0622-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-f61e4b-7c54c4f9f5-gv9x9\" (UID: \"7e524e27-0d0d-42db-b376-0c488d6a0622\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-f61e4b-7c54c4f9f5-gv9x9"
Oct 30 19:50:26.353502 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:26.350497    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3eafcd7e-b8d7-4f3b-a483-c2c7dc9b764a-plugins-config-volume\") pod \"payload-processing-e2e-api-b-f61e4b-66959bc884-hmnz9\" (UID: \"3eafcd7e-b8d7-4f3b-a483-c2c7dc9b764a\") " pod="openshift-ingress/payload-processing-e2e-api-b-f61e4b-66959bc884-hmnz9"
Oct 30 19:50:26.366834 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:26.366800    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgnw6\" (UniqueName: \"kubernetes.io/projected/3eafcd7e-b8d7-4f3b-a483-c2c7dc9b764a-kube-api-access-dgnw6\") pod \"payload-processing-e2e-api-b-f61e4b-66959bc884-hmnz9\" (UID: \"3eafcd7e-b8d7-4f3b-a483-c2c7dc9b764a\") " pod="openshift-ingress/payload-processing-e2e-api-b-f61e4b-66959bc884-hmnz9"
Oct 30 19:50:26.369931 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:26.369898    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcg99\" (UniqueName: \"kubernetes.io/projected/7e524e27-0d0d-42db-b376-0c488d6a0622-kube-api-access-zcg99\") pod \"payload-pre-processing-e2e-api-b-f61e4b-7c54c4f9f5-gv9x9\" (UID: \"7e524e27-0d0d-42db-b376-0c488d6a0622\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-f61e4b-7c54c4f9f5-gv9x9"
Oct 30 19:50:26.370750 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:26.370726    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f61e4b-7c54c4f9f5-gv9x9"
Oct 30 19:50:26.608032 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:26.607949    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-f61e4b-66959bc884-hmnz9"
Oct 30 19:50:26.690935 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:26.681198    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-f61e4b-7c54c4f9f5-gv9x9"]
Oct 30 19:50:26.690935 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:50:26.683660    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e524e27_0d0d_42db_b376_0c488d6a0622.slice/crio-3de60ea18e8279992d601c2157ced8b3e154d297be2b97b2772164f8979e7565 WatchSource:0}: Error finding container 3de60ea18e8279992d601c2157ced8b3e154d297be2b97b2772164f8979e7565: Status 404 returned error can't find the container with id 3de60ea18e8279992d601c2157ced8b3e154d297be2b97b2772164f8979e7565
Oct 30 19:50:26.898898 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:50:26.898857    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3eafcd7e_b8d7_4f3b_a483_c2c7dc9b764a.slice/crio-6ecfee7b4e27669ffd74a9f1b227ce693ee1cd6104434cd79ace08009198afae WatchSource:0}: Error finding container 6ecfee7b4e27669ffd74a9f1b227ce693ee1cd6104434cd79ace08009198afae: Status 404 returned error can't find the container with id 6ecfee7b4e27669ffd74a9f1b227ce693ee1cd6104434cd79ace08009198afae
Oct 30 19:50:26.899545 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:26.899430    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-f61e4b-66959bc884-hmnz9"]
Oct 30 19:50:27.385423 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:27.385386    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f61e4b-7c54c4f9f5-gv9x9" event={"ID":"7e524e27-0d0d-42db-b376-0c488d6a0622","Type":"ContainerStarted","Data":"dff798a149b1dc5b0f44cdda21d56089b87d4fe291ec7eb1bb53808bb57b04d7"}
Oct 30 19:50:27.385899 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:27.385452    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f61e4b-7c54c4f9f5-gv9x9" event={"ID":"7e524e27-0d0d-42db-b376-0c488d6a0622","Type":"ContainerStarted","Data":"3de60ea18e8279992d601c2157ced8b3e154d297be2b97b2772164f8979e7565"}
Oct 30 19:50:27.385899 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:27.385603    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f61e4b-7c54c4f9f5-gv9x9"
Oct 30 19:50:27.387522 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:27.387469    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-f61e4b-66959bc884-hmnz9" event={"ID":"3eafcd7e-b8d7-4f3b-a483-c2c7dc9b764a","Type":"ContainerStarted","Data":"040a37c9c0d37e4b3e663de4afce594e700f256d82adb5605a52744a552bd1a9"}
Oct 30 19:50:27.387653 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:27.387540    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-f61e4b-66959bc884-hmnz9" event={"ID":"3eafcd7e-b8d7-4f3b-a483-c2c7dc9b764a","Type":"ContainerStarted","Data":"6ecfee7b4e27669ffd74a9f1b227ce693ee1cd6104434cd79ace08009198afae"}
Oct 30 19:50:27.387653 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:27.387569    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-f61e4b-66959bc884-hmnz9"
Oct 30 19:50:27.406846 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:27.406797    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f61e4b-7c54c4f9f5-gv9x9" podStartSLOduration=1.406785335 podStartE2EDuration="1.406785335s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:50:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:50:27.404422686 +0000 UTC m=+1102.732540840" watchObservedRunningTime="2026-10-30 19:50:27.406785335 +0000 UTC m=+1102.734903530"
Oct 30 19:50:27.425862 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:27.425819    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-f61e4b-66959bc884-hmnz9" podStartSLOduration=1.4258068769999999 podStartE2EDuration="1.425806877s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:50:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:50:27.423154884 +0000 UTC m=+1102.751273038" watchObservedRunningTime="2026-10-30 19:50:27.425806877 +0000 UTC m=+1102.753925025"
Oct 30 19:50:31.192202 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:50:31.192158    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 19:50:38.397850 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:38.397819    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-f61e4b-66959bc884-hmnz9"
Oct 30 19:50:38.398311 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:38.397884    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f61e4b-7c54c4f9f5-gv9x9"
Oct 30 19:50:40.543545 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:40.543455    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f61e4b-57cccbd8f-j8kw2"]
Oct 30 19:50:40.549647 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:40.549620    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f61e4b-57cccbd8f-j8kw2"
Oct 30 19:50:40.561897 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:40.561391    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-f61e4b-serving-cert\""
Oct 30 19:50:40.571830 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:40.571805    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f61e4b-57cccbd8f-j8kw2"]
Oct 30 19:50:40.622584 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:40.622543    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c19eefde-3ee9-4c3f-b551-cdc4d30a96c2-openshift-service-ca\") pod \"maas-api-e2e-api-b-f61e4b-57cccbd8f-j8kw2\" (UID: \"c19eefde-3ee9-4c3f-b551-cdc4d30a96c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f61e4b-57cccbd8f-j8kw2"
Oct 30 19:50:40.622737 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:40.622670    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c19eefde-3ee9-4c3f-b551-cdc4d30a96c2-metrics-certs\") pod \"maas-api-e2e-api-b-f61e4b-57cccbd8f-j8kw2\" (UID: \"c19eefde-3ee9-4c3f-b551-cdc4d30a96c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f61e4b-57cccbd8f-j8kw2"
Oct 30 19:50:40.622861 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:40.622831    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c19eefde-3ee9-4c3f-b551-cdc4d30a96c2-maas-api-tls\") pod \"maas-api-e2e-api-b-f61e4b-57cccbd8f-j8kw2\" (UID: \"c19eefde-3ee9-4c3f-b551-cdc4d30a96c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f61e4b-57cccbd8f-j8kw2"
Oct 30 19:50:40.622972 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:40.622957    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k95f6\" (UniqueName: \"kubernetes.io/projected/c19eefde-3ee9-4c3f-b551-cdc4d30a96c2-kube-api-access-k95f6\") pod \"maas-api-e2e-api-b-f61e4b-57cccbd8f-j8kw2\" (UID: \"c19eefde-3ee9-4c3f-b551-cdc4d30a96c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f61e4b-57cccbd8f-j8kw2"
Oct 30 19:50:40.724388 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:40.724354    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c19eefde-3ee9-4c3f-b551-cdc4d30a96c2-metrics-certs\") pod \"maas-api-e2e-api-b-f61e4b-57cccbd8f-j8kw2\" (UID: \"c19eefde-3ee9-4c3f-b551-cdc4d30a96c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f61e4b-57cccbd8f-j8kw2"
Oct 30 19:50:40.724569 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:40.724498    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c19eefde-3ee9-4c3f-b551-cdc4d30a96c2-maas-api-tls\") pod \"maas-api-e2e-api-b-f61e4b-57cccbd8f-j8kw2\" (UID: \"c19eefde-3ee9-4c3f-b551-cdc4d30a96c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f61e4b-57cccbd8f-j8kw2"
Oct 30 19:50:40.724729 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:40.724604    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k95f6\" (UniqueName: \"kubernetes.io/projected/c19eefde-3ee9-4c3f-b551-cdc4d30a96c2-kube-api-access-k95f6\") pod \"maas-api-e2e-api-b-f61e4b-57cccbd8f-j8kw2\" (UID: \"c19eefde-3ee9-4c3f-b551-cdc4d30a96c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f61e4b-57cccbd8f-j8kw2"
Oct 30 19:50:40.724729 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:40.724662    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c19eefde-3ee9-4c3f-b551-cdc4d30a96c2-openshift-service-ca\") pod \"maas-api-e2e-api-b-f61e4b-57cccbd8f-j8kw2\" (UID: \"c19eefde-3ee9-4c3f-b551-cdc4d30a96c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f61e4b-57cccbd8f-j8kw2"
Oct 30 19:50:40.725472 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:40.725443    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c19eefde-3ee9-4c3f-b551-cdc4d30a96c2-openshift-service-ca\") pod \"maas-api-e2e-api-b-f61e4b-57cccbd8f-j8kw2\" (UID: \"c19eefde-3ee9-4c3f-b551-cdc4d30a96c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f61e4b-57cccbd8f-j8kw2"
Oct 30 19:50:40.726982 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:40.726959    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c19eefde-3ee9-4c3f-b551-cdc4d30a96c2-metrics-certs\") pod \"maas-api-e2e-api-b-f61e4b-57cccbd8f-j8kw2\" (UID: \"c19eefde-3ee9-4c3f-b551-cdc4d30a96c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f61e4b-57cccbd8f-j8kw2"
Oct 30 19:50:40.727163 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:40.727142    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c19eefde-3ee9-4c3f-b551-cdc4d30a96c2-maas-api-tls\") pod \"maas-api-e2e-api-b-f61e4b-57cccbd8f-j8kw2\" (UID: \"c19eefde-3ee9-4c3f-b551-cdc4d30a96c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f61e4b-57cccbd8f-j8kw2"
Oct 30 19:50:40.744317 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:40.744286    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k95f6\" (UniqueName: \"kubernetes.io/projected/c19eefde-3ee9-4c3f-b551-cdc4d30a96c2-kube-api-access-k95f6\") pod \"maas-api-e2e-api-b-f61e4b-57cccbd8f-j8kw2\" (UID: \"c19eefde-3ee9-4c3f-b551-cdc4d30a96c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f61e4b-57cccbd8f-j8kw2"
Oct 30 19:50:40.877336 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:40.877303    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f61e4b-57cccbd8f-j8kw2"
Oct 30 19:50:41.193472 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:41.192566    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f61e4b-57cccbd8f-j8kw2"]
Oct 30 19:50:41.198039 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:50:41.197996    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc19eefde_3ee9_4c3f_b551_cdc4d30a96c2.slice/crio-ca6c97a04a6d94722e1c881f39c1906e31d29201508b1e1c7c7302ba26719ab3 WatchSource:0}: Error finding container ca6c97a04a6d94722e1c881f39c1906e31d29201508b1e1c7c7302ba26719ab3: Status 404 returned error can't find the container with id ca6c97a04a6d94722e1c881f39c1906e31d29201508b1e1c7c7302ba26719ab3
Oct 30 19:50:41.513800 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:41.513717    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f61e4b-57cccbd8f-j8kw2" event={"ID":"c19eefde-3ee9-4c3f-b551-cdc4d30a96c2","Type":"ContainerStarted","Data":"ca6c97a04a6d94722e1c881f39c1906e31d29201508b1e1c7c7302ba26719ab3"}
Oct 30 19:50:42.525350 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:42.525308    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f61e4b-57cccbd8f-j8kw2" event={"ID":"c19eefde-3ee9-4c3f-b551-cdc4d30a96c2","Type":"ContainerStarted","Data":"3efe0093a179aef260833884a15e9ff7dfd8b790b45755fb86a3a47551ccc866"}
Oct 30 19:50:42.525794 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:42.525389    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f61e4b-57cccbd8f-j8kw2"
Oct 30 19:50:42.552573 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:42.552515    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f61e4b-57cccbd8f-j8kw2" podStartSLOduration=2.056393016 podStartE2EDuration="2.552502106s" totalImagesPullingTime="496.10909ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:50:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:50:42.548067485 +0000 UTC m=+1117.876185639" watchObservedRunningTime="2026-10-30 19:50:42.552502106 +0000 UTC m=+1117.880620259"
Oct 30 19:50:44.544782 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:44.544745    6996 generic.go:356] "Generic (PLEG): container finished" podID="d9ce7283-f1bf-4afe-97dd-ad5f5ce14786" containerID="c1399355ad0d85b6f237444d6ddce78a326cf9c5da2d7bb70520f69f038fe8ee" exitCode=0
Oct 30 19:50:44.545674 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:44.544818    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk" event={"ID":"d9ce7283-f1bf-4afe-97dd-ad5f5ce14786","Type":"ContainerDied","Data":"c1399355ad0d85b6f237444d6ddce78a326cf9c5da2d7bb70520f69f038fe8ee"}
Oct 30 19:50:44.545674 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:44.545301    6996 scope.go:122] "RemoveContainer" containerID="c1399355ad0d85b6f237444d6ddce78a326cf9c5da2d7bb70520f69f038fe8ee"
Oct 30 19:50:44.547015 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:44.546974    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:50:44.547133 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:44.547070    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:50:44.547133 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:44.547121    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:50:45.567567 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:45.567534    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk" event={"ID":"d9ce7283-f1bf-4afe-97dd-ad5f5ce14786","Type":"ContainerStarted","Data":"04907e862cfdc39fae50e894edabd276e2b328e1b3d6ec1ec52ac16d19f1f47a"}
Oct 30 19:50:46.000519 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:46.000398    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk"
Oct 30 19:50:46.000792 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:46.000768    6996 patch_prober.go:29] interesting pod/e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.118:15021/healthz/ready\": dial tcp 10.134.0.118:15021: connect: connection refused" start-of-body=
Oct 30 19:50:46.000849 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:46.000818    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk" podUID="d9ce7283-f1bf-4afe-97dd-ad5f5ce14786" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.118:15021/healthz/ready\": dial tcp 10.134.0.118:15021: connect: connection refused"
Oct 30 19:50:47.001187 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:47.001147    6996 patch_prober.go:29] interesting pod/e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.118:15021/healthz/ready\": dial tcp 10.134.0.118:15021: connect: connection refused" start-of-body=
Oct 30 19:50:47.001622 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:47.001205    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk" podUID="d9ce7283-f1bf-4afe-97dd-ad5f5ce14786" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.118:15021/healthz/ready\": dial tcp 10.134.0.118:15021: connect: connection refused"
Oct 30 19:50:48.000556 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:48.000521    6996 patch_prober.go:29] interesting pod/e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.118:15021/healthz/ready\": dial tcp 10.134.0.118:15021: connect: connection refused" start-of-body=
Oct 30 19:50:48.000740 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:48.000587    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk" podUID="d9ce7283-f1bf-4afe-97dd-ad5f5ce14786" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.118:15021/healthz/ready\": dial tcp 10.134.0.118:15021: connect: connection refused"
Oct 30 19:50:48.562741 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:48.562704    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f61e4b-57cccbd8f-j8kw2"
Oct 30 19:50:49.000770 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:49.000733    6996 patch_prober.go:29] interesting pod/e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.118:15021/healthz/ready\": dial tcp 10.134.0.118:15021: connect: connection refused" start-of-body=
Oct 30 19:50:49.000951 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:49.000788    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk" podUID="d9ce7283-f1bf-4afe-97dd-ad5f5ce14786" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.118:15021/healthz/ready\": dial tcp 10.134.0.118:15021: connect: connection refused"
Oct 30 19:50:50.001130 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:50.001088    6996 patch_prober.go:29] interesting pod/e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.118:15021/healthz/ready\": dial tcp 10.134.0.118:15021: connect: connection refused" start-of-body=
Oct 30 19:50:50.001654 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:50.001149    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk" podUID="d9ce7283-f1bf-4afe-97dd-ad5f5ce14786" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.118:15021/healthz/ready\": dial tcp 10.134.0.118:15021: connect: connection refused"
Oct 30 19:50:51.005641 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:51.005609    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk"
Oct 30 19:50:51.006121 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:51.006106    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk"
Oct 30 19:50:51.006911 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:50:51.006890    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk"
Oct 30 19:51:01.365429 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:51:01.365312    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 19:51:01.560858 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:01.560807    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 19:51:06.182810 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:06.182022    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-c9d69c-6d96fc497c-p25r9"]
Oct 30 19:51:06.182810 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:06.182282    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c9d69c-6d96fc497c-p25r9" podUID="5e65ecbf-2aba-4a2c-abfa-c5d6c789152a" containerName="maas-api" containerID="cri-o://e75b1fb062e28ba30d7d534d208e98bfa059a8ef3898788460813a0043a578fe" gracePeriod=30
Oct 30 19:51:06.487886 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:06.487547    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c9d69c-6d96fc497c-p25r9"
Oct 30 19:51:06.545621 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:06.545578    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5e65ecbf-2aba-4a2c-abfa-c5d6c789152a-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5e65ecbf-2aba-4a2c-abfa-c5d6c789152a-maas-api-tls\") pod \"5e65ecbf-2aba-4a2c-abfa-c5d6c789152a\" (UID: \"5e65ecbf-2aba-4a2c-abfa-c5d6c789152a\") "
Oct 30 19:51:06.546694 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:06.546640    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e65ecbf-2aba-4a2c-abfa-c5d6c789152a-openshift-service-ca" pod "5e65ecbf-2aba-4a2c-abfa-c5d6c789152a" (UID: "5e65ecbf-2aba-4a2c-abfa-c5d6c789152a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:51:06.546865 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:06.546840    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5e65ecbf-2aba-4a2c-abfa-c5d6c789152a-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e65ecbf-2aba-4a2c-abfa-c5d6c789152a-openshift-service-ca\") pod \"5e65ecbf-2aba-4a2c-abfa-c5d6c789152a\" (UID: \"5e65ecbf-2aba-4a2c-abfa-c5d6c789152a\") "
Oct 30 19:51:06.547011 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:06.546954    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5e65ecbf-2aba-4a2c-abfa-c5d6c789152a-metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e65ecbf-2aba-4a2c-abfa-c5d6c789152a-metrics-certs\") pod \"5e65ecbf-2aba-4a2c-abfa-c5d6c789152a\" (UID: \"5e65ecbf-2aba-4a2c-abfa-c5d6c789152a\") "
Oct 30 19:51:06.547380 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:06.547361    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5e65ecbf-2aba-4a2c-abfa-c5d6c789152a-kube-api-access-zcts9\" (UniqueName: \"kubernetes.io/projected/5e65ecbf-2aba-4a2c-abfa-c5d6c789152a-kube-api-access-zcts9\") pod \"5e65ecbf-2aba-4a2c-abfa-c5d6c789152a\" (UID: \"5e65ecbf-2aba-4a2c-abfa-c5d6c789152a\") "
Oct 30 19:51:06.548438 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:06.548416    6996 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e65ecbf-2aba-4a2c-abfa-c5d6c789152a-openshift-service-ca\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:51:06.573336 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:06.573268    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e65ecbf-2aba-4a2c-abfa-c5d6c789152a-metrics-certs" pod "5e65ecbf-2aba-4a2c-abfa-c5d6c789152a" (UID: "5e65ecbf-2aba-4a2c-abfa-c5d6c789152a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:51:06.613678 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:06.613631    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e65ecbf-2aba-4a2c-abfa-c5d6c789152a-maas-api-tls" pod "5e65ecbf-2aba-4a2c-abfa-c5d6c789152a" (UID: "5e65ecbf-2aba-4a2c-abfa-c5d6c789152a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:51:06.614237 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:06.614205    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e65ecbf-2aba-4a2c-abfa-c5d6c789152a-kube-api-access-zcts9" pod "5e65ecbf-2aba-4a2c-abfa-c5d6c789152a" (UID: "5e65ecbf-2aba-4a2c-abfa-c5d6c789152a"). InnerVolumeSpecName "kube-api-access-zcts9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:51:06.651871 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:06.650533    6996 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5e65ecbf-2aba-4a2c-abfa-c5d6c789152a-maas-api-tls\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:51:06.651871 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:06.650569    6996 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e65ecbf-2aba-4a2c-abfa-c5d6c789152a-metrics-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:51:06.651871 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:06.650583    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zcts9\" (UniqueName: \"kubernetes.io/projected/5e65ecbf-2aba-4a2c-abfa-c5d6c789152a-kube-api-access-zcts9\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:51:06.742295 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:06.742208    6996 generic.go:356] "Generic (PLEG): container finished" podID="5e65ecbf-2aba-4a2c-abfa-c5d6c789152a" containerID="e75b1fb062e28ba30d7d534d208e98bfa059a8ef3898788460813a0043a578fe" exitCode=0
Oct 30 19:51:06.742295 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:06.742258    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c9d69c-6d96fc497c-p25r9"
Oct 30 19:51:06.742494 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:06.742291    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c9d69c-6d96fc497c-p25r9" event={"ID":"5e65ecbf-2aba-4a2c-abfa-c5d6c789152a","Type":"ContainerDied","Data":"e75b1fb062e28ba30d7d534d208e98bfa059a8ef3898788460813a0043a578fe"}
Oct 30 19:51:06.742494 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:06.742349    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c9d69c-6d96fc497c-p25r9" event={"ID":"5e65ecbf-2aba-4a2c-abfa-c5d6c789152a","Type":"ContainerDied","Data":"b8656cca4871fd974ef128d31485c1223282f22a67427e6a2729dbda9cce04e8"}
Oct 30 19:51:06.742494 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:06.742372    6996 scope.go:122] "RemoveContainer" containerID="e75b1fb062e28ba30d7d534d208e98bfa059a8ef3898788460813a0043a578fe"
Oct 30 19:51:06.754525 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:06.754468    6996 scope.go:122] "RemoveContainer" containerID="e75b1fb062e28ba30d7d534d208e98bfa059a8ef3898788460813a0043a578fe"
Oct 30 19:51:06.754851 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:51:06.754822    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e75b1fb062e28ba30d7d534d208e98bfa059a8ef3898788460813a0043a578fe\": container with ID starting with e75b1fb062e28ba30d7d534d208e98bfa059a8ef3898788460813a0043a578fe not found: ID does not exist" containerID="e75b1fb062e28ba30d7d534d208e98bfa059a8ef3898788460813a0043a578fe"
Oct 30 19:51:06.754972 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:06.754859    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e75b1fb062e28ba30d7d534d208e98bfa059a8ef3898788460813a0043a578fe"} err="failed to get container status \"e75b1fb062e28ba30d7d534d208e98bfa059a8ef3898788460813a0043a578fe\": rpc error: code = NotFound desc = could not find container \"e75b1fb062e28ba30d7d534d208e98bfa059a8ef3898788460813a0043a578fe\": container with ID starting with e75b1fb062e28ba30d7d534d208e98bfa059a8ef3898788460813a0043a578fe not found: ID does not exist"
Oct 30 19:51:06.769357 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:06.769332    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-c9d69c-6d96fc497c-p25r9"]
Oct 30 19:51:06.776294 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:06.776269    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-c9d69c-6d96fc497c-p25r9"]
Oct 30 19:51:06.821534 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:06.821474    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5e65ecbf-2aba-4a2c-abfa-c5d6c789152a" path="/var/lib/kubelet/pods/5e65ecbf-2aba-4a2c-abfa-c5d6c789152a/volumes"
Oct 30 19:51:09.651722 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:09.651686    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8"]
Oct 30 19:51:09.652270 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:09.651886    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8" podUID="003d559b-ba77-46e7-b4bc-4b9dd37653c5" containerName="istio-proxy" containerID="cri-o://5ffa3bc2e7a928b969ae3ce2229f6388ac686dd1f31f98e37ec023c19fd927dd" gracePeriod=30
Oct 30 19:51:14.817499 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:14.817443    6996 generic.go:356] "Generic (PLEG): container finished" podID="003d559b-ba77-46e7-b4bc-4b9dd37653c5" containerID="5ffa3bc2e7a928b969ae3ce2229f6388ac686dd1f31f98e37ec023c19fd927dd" exitCode=0
Oct 30 19:51:14.817871 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:14.817536    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8" event={"ID":"003d559b-ba77-46e7-b4bc-4b9dd37653c5","Type":"ContainerDied","Data":"5ffa3bc2e7a928b969ae3ce2229f6388ac686dd1f31f98e37ec023c19fd927dd"}
Oct 30 19:51:14.959638 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:14.959608    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8"
Oct 30 19:51:15.019729 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:15.019686    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/003d559b-ba77-46e7-b4bc-4b9dd37653c5-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/003d559b-ba77-46e7-b4bc-4b9dd37653c5-istiod-ca-cert\") pod \"003d559b-ba77-46e7-b4bc-4b9dd37653c5\" (UID: \"003d559b-ba77-46e7-b4bc-4b9dd37653c5\") "
Oct 30 19:51:15.019914 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:15.019787    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/003d559b-ba77-46e7-b4bc-4b9dd37653c5-kube-api-access-p22fk\" (UniqueName: \"kubernetes.io/projected/003d559b-ba77-46e7-b4bc-4b9dd37653c5-kube-api-access-p22fk\") pod \"003d559b-ba77-46e7-b4bc-4b9dd37653c5\" (UID: \"003d559b-ba77-46e7-b4bc-4b9dd37653c5\") "
Oct 30 19:51:15.019914 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:15.019842    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/003d559b-ba77-46e7-b4bc-4b9dd37653c5-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/003d559b-ba77-46e7-b4bc-4b9dd37653c5-istio-data\") pod \"003d559b-ba77-46e7-b4bc-4b9dd37653c5\" (UID: \"003d559b-ba77-46e7-b4bc-4b9dd37653c5\") "
Oct 30 19:51:15.019914 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:15.019897    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/003d559b-ba77-46e7-b4bc-4b9dd37653c5-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/003d559b-ba77-46e7-b4bc-4b9dd37653c5-workload-socket\") pod \"003d559b-ba77-46e7-b4bc-4b9dd37653c5\" (UID: \"003d559b-ba77-46e7-b4bc-4b9dd37653c5\") "
Oct 30 19:51:15.020097 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:15.019940    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/003d559b-ba77-46e7-b4bc-4b9dd37653c5-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/003d559b-ba77-46e7-b4bc-4b9dd37653c5-workload-certs\") pod \"003d559b-ba77-46e7-b4bc-4b9dd37653c5\" (UID: \"003d559b-ba77-46e7-b4bc-4b9dd37653c5\") "
Oct 30 19:51:15.020097 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:15.019972    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/003d559b-ba77-46e7-b4bc-4b9dd37653c5-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/003d559b-ba77-46e7-b4bc-4b9dd37653c5-credential-socket\") pod \"003d559b-ba77-46e7-b4bc-4b9dd37653c5\" (UID: \"003d559b-ba77-46e7-b4bc-4b9dd37653c5\") "
Oct 30 19:51:15.020097 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:15.020023    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/003d559b-ba77-46e7-b4bc-4b9dd37653c5-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/003d559b-ba77-46e7-b4bc-4b9dd37653c5-istio-podinfo\") pod \"003d559b-ba77-46e7-b4bc-4b9dd37653c5\" (UID: \"003d559b-ba77-46e7-b4bc-4b9dd37653c5\") "
Oct 30 19:51:15.020259 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:15.020108    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/003d559b-ba77-46e7-b4bc-4b9dd37653c5-istio-token\" (UniqueName: \"kubernetes.io/projected/003d559b-ba77-46e7-b4bc-4b9dd37653c5-istio-token\") pod \"003d559b-ba77-46e7-b4bc-4b9dd37653c5\" (UID: \"003d559b-ba77-46e7-b4bc-4b9dd37653c5\") "
Oct 30 19:51:15.020259 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:15.020105    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/003d559b-ba77-46e7-b4bc-4b9dd37653c5-istiod-ca-cert" pod "003d559b-ba77-46e7-b4bc-4b9dd37653c5" (UID: "003d559b-ba77-46e7-b4bc-4b9dd37653c5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:51:15.020259 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:15.020185    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/003d559b-ba77-46e7-b4bc-4b9dd37653c5-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/003d559b-ba77-46e7-b4bc-4b9dd37653c5-istio-envoy\") pod \"003d559b-ba77-46e7-b4bc-4b9dd37653c5\" (UID: \"003d559b-ba77-46e7-b4bc-4b9dd37653c5\") "
Oct 30 19:51:15.021143 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:15.020566    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/003d559b-ba77-46e7-b4bc-4b9dd37653c5-workload-certs" pod "003d559b-ba77-46e7-b4bc-4b9dd37653c5" (UID: "003d559b-ba77-46e7-b4bc-4b9dd37653c5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:51:15.021143 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:15.020878    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/003d559b-ba77-46e7-b4bc-4b9dd37653c5-credential-socket" pod "003d559b-ba77-46e7-b4bc-4b9dd37653c5" (UID: "003d559b-ba77-46e7-b4bc-4b9dd37653c5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:51:15.021143 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:15.021027    6996 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/003d559b-ba77-46e7-b4bc-4b9dd37653c5-istiod-ca-cert\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:51:15.021143 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:15.021045    6996 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/003d559b-ba77-46e7-b4bc-4b9dd37653c5-workload-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:51:15.021143 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:15.021061    6996 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/003d559b-ba77-46e7-b4bc-4b9dd37653c5-credential-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:51:15.021143 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:15.021116    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/003d559b-ba77-46e7-b4bc-4b9dd37653c5-workload-socket" pod "003d559b-ba77-46e7-b4bc-4b9dd37653c5" (UID: "003d559b-ba77-46e7-b4bc-4b9dd37653c5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:51:15.021831 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:15.021802    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/003d559b-ba77-46e7-b4bc-4b9dd37653c5-istio-data" pod "003d559b-ba77-46e7-b4bc-4b9dd37653c5" (UID: "003d559b-ba77-46e7-b4bc-4b9dd37653c5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:51:15.022934 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:15.022906    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/003d559b-ba77-46e7-b4bc-4b9dd37653c5-istio-podinfo" pod "003d559b-ba77-46e7-b4bc-4b9dd37653c5" (UID: "003d559b-ba77-46e7-b4bc-4b9dd37653c5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:51:15.023089 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:15.023069    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/003d559b-ba77-46e7-b4bc-4b9dd37653c5-kube-api-access-p22fk" pod "003d559b-ba77-46e7-b4bc-4b9dd37653c5" (UID: "003d559b-ba77-46e7-b4bc-4b9dd37653c5"). InnerVolumeSpecName "kube-api-access-p22fk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:51:15.023638 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:15.023617    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/003d559b-ba77-46e7-b4bc-4b9dd37653c5-istio-envoy" pod "003d559b-ba77-46e7-b4bc-4b9dd37653c5" (UID: "003d559b-ba77-46e7-b4bc-4b9dd37653c5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:51:15.024039 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:15.023992    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/003d559b-ba77-46e7-b4bc-4b9dd37653c5-istio-token" pod "003d559b-ba77-46e7-b4bc-4b9dd37653c5" (UID: "003d559b-ba77-46e7-b4bc-4b9dd37653c5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:51:15.122251 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:15.122212    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p22fk\" (UniqueName: \"kubernetes.io/projected/003d559b-ba77-46e7-b4bc-4b9dd37653c5-kube-api-access-p22fk\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:51:15.122251 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:15.122243    6996 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/003d559b-ba77-46e7-b4bc-4b9dd37653c5-istio-data\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:51:15.122251 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:15.122253    6996 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/003d559b-ba77-46e7-b4bc-4b9dd37653c5-workload-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:51:15.122251 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:15.122261    6996 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/003d559b-ba77-46e7-b4bc-4b9dd37653c5-istio-podinfo\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:51:15.122624 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:15.122272    6996 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/003d559b-ba77-46e7-b4bc-4b9dd37653c5-istio-token\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:51:15.122624 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:15.122281    6996 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/003d559b-ba77-46e7-b4bc-4b9dd37653c5-istio-envoy\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:51:15.238304 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:15.238221    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-ec5e0f-6b5968df56-7mddg"]
Oct 30 19:51:15.239016 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:15.238987    6996 cpu_manager.go:414] "containerMap: removing container" podUID="003d559b-ba77-46e7-b4bc-4b9dd37653c5" containerName="istio-proxy"
Oct 30 19:51:15.239016 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:15.239015    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="003d559b-ba77-46e7-b4bc-4b9dd37653c5" containerName="istio-proxy"
Oct 30 19:51:15.239196 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:15.239046    6996 cpu_manager.go:414] "containerMap: removing container" podUID="5e65ecbf-2aba-4a2c-abfa-c5d6c789152a" containerName="maas-api"
Oct 30 19:51:15.239196 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:15.239055    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5e65ecbf-2aba-4a2c-abfa-c5d6c789152a" containerName="maas-api"
Oct 30 19:51:15.239298 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:15.239263    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="5e65ecbf-2aba-4a2c-abfa-c5d6c789152a" containerName="maas-api"
Oct 30 19:51:15.239298 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:15.239293    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="003d559b-ba77-46e7-b4bc-4b9dd37653c5" containerName="istio-proxy"
Oct 30 19:51:15.243245 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:15.243224    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ec5e0f-6b5968df56-7mddg"
Oct 30 19:51:15.249267 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:15.249243    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-ec5e0f-serving-cert\""
Oct 30 19:51:15.257822 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:15.257798    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-ec5e0f-6b5968df56-7mddg"]
Oct 30 19:51:15.325203 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:15.325159    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b01476c8-1aed-4fd1-acb0-6aea531856ca-metrics-certs\") pod \"maas-api-e2e-ipp-b-ec5e0f-6b5968df56-7mddg\" (UID: \"b01476c8-1aed-4fd1-acb0-6aea531856ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ec5e0f-6b5968df56-7mddg"
Oct 30 19:51:15.325582 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:15.325535    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgw2q\" (UniqueName: \"kubernetes.io/projected/b01476c8-1aed-4fd1-acb0-6aea531856ca-kube-api-access-rgw2q\") pod \"maas-api-e2e-ipp-b-ec5e0f-6b5968df56-7mddg\" (UID: \"b01476c8-1aed-4fd1-acb0-6aea531856ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ec5e0f-6b5968df56-7mddg"
Oct 30 19:51:15.326059 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:15.325763    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b01476c8-1aed-4fd1-acb0-6aea531856ca-maas-api-tls\") pod \"maas-api-e2e-ipp-b-ec5e0f-6b5968df56-7mddg\" (UID: \"b01476c8-1aed-4fd1-acb0-6aea531856ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ec5e0f-6b5968df56-7mddg"
Oct 30 19:51:15.326059 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:15.325925    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b01476c8-1aed-4fd1-acb0-6aea531856ca-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-ec5e0f-6b5968df56-7mddg\" (UID: \"b01476c8-1aed-4fd1-acb0-6aea531856ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ec5e0f-6b5968df56-7mddg"
Oct 30 19:51:15.427946 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:15.427911    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b01476c8-1aed-4fd1-acb0-6aea531856ca-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-ec5e0f-6b5968df56-7mddg\" (UID: \"b01476c8-1aed-4fd1-acb0-6aea531856ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ec5e0f-6b5968df56-7mddg"
Oct 30 19:51:15.428119 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:15.428044    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b01476c8-1aed-4fd1-acb0-6aea531856ca-metrics-certs\") pod \"maas-api-e2e-ipp-b-ec5e0f-6b5968df56-7mddg\" (UID: \"b01476c8-1aed-4fd1-acb0-6aea531856ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ec5e0f-6b5968df56-7mddg"
Oct 30 19:51:15.428119 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:15.428064    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgw2q\" (UniqueName: \"kubernetes.io/projected/b01476c8-1aed-4fd1-acb0-6aea531856ca-kube-api-access-rgw2q\") pod \"maas-api-e2e-ipp-b-ec5e0f-6b5968df56-7mddg\" (UID: \"b01476c8-1aed-4fd1-acb0-6aea531856ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ec5e0f-6b5968df56-7mddg"
Oct 30 19:51:15.428226 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:15.428142    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b01476c8-1aed-4fd1-acb0-6aea531856ca-maas-api-tls\") pod \"maas-api-e2e-ipp-b-ec5e0f-6b5968df56-7mddg\" (UID: \"b01476c8-1aed-4fd1-acb0-6aea531856ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ec5e0f-6b5968df56-7mddg"
Oct 30 19:51:15.428302 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:51:15.428288    6996 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-ec5e0f-serving-cert: secret "maas-api-e2e-ipp-b-ec5e0f-serving-cert" not found
Oct 30 19:51:15.428367 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:51:15.428356    6996 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b01476c8-1aed-4fd1-acb0-6aea531856ca-maas-api-tls podName:b01476c8-1aed-4fd1-acb0-6aea531856ca nodeName:}" failed. No retries permitted until 2026-10-30 19:51:15.928337968 +0000 UTC m=+1151.256456102 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b01476c8-1aed-4fd1-acb0-6aea531856ca-maas-api-tls") pod "maas-api-e2e-ipp-b-ec5e0f-6b5968df56-7mddg" (UID: "b01476c8-1aed-4fd1-acb0-6aea531856ca") : secret "maas-api-e2e-ipp-b-ec5e0f-serving-cert" not found
Oct 30 19:51:15.428642 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:15.428618    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b01476c8-1aed-4fd1-acb0-6aea531856ca-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-ec5e0f-6b5968df56-7mddg\" (UID: \"b01476c8-1aed-4fd1-acb0-6aea531856ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ec5e0f-6b5968df56-7mddg"
Oct 30 19:51:15.430472 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:15.430452    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b01476c8-1aed-4fd1-acb0-6aea531856ca-metrics-certs\") pod \"maas-api-e2e-ipp-b-ec5e0f-6b5968df56-7mddg\" (UID: \"b01476c8-1aed-4fd1-acb0-6aea531856ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ec5e0f-6b5968df56-7mddg"
Oct 30 19:51:15.441180 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:15.441150    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgw2q\" (UniqueName: \"kubernetes.io/projected/b01476c8-1aed-4fd1-acb0-6aea531856ca-kube-api-access-rgw2q\") pod \"maas-api-e2e-ipp-b-ec5e0f-6b5968df56-7mddg\" (UID: \"b01476c8-1aed-4fd1-acb0-6aea531856ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ec5e0f-6b5968df56-7mddg"
Oct 30 19:51:15.836459 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:15.836430    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8"
Oct 30 19:51:15.836459 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:15.836443    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8" event={"ID":"003d559b-ba77-46e7-b4bc-4b9dd37653c5","Type":"ContainerDied","Data":"73fb6e8d9f7b04860c85f886467004fcf470e6257128c9a1878e624af86d272d"}
Oct 30 19:51:15.837034 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:15.836560    6996 scope.go:122] "RemoveContainer" containerID="5ffa3bc2e7a928b969ae3ce2229f6388ac686dd1f31f98e37ec023c19fd927dd"
Oct 30 19:51:15.873916 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:15.873891    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8"]
Oct 30 19:51:15.878664 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:15.878638    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-bw7h8"]
Oct 30 19:51:15.935760 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:15.935733    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b01476c8-1aed-4fd1-acb0-6aea531856ca-maas-api-tls\") pod \"maas-api-e2e-ipp-b-ec5e0f-6b5968df56-7mddg\" (UID: \"b01476c8-1aed-4fd1-acb0-6aea531856ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ec5e0f-6b5968df56-7mddg"
Oct 30 19:51:15.938065 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:15.938045    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b01476c8-1aed-4fd1-acb0-6aea531856ca-maas-api-tls\") pod \"maas-api-e2e-ipp-b-ec5e0f-6b5968df56-7mddg\" (UID: \"b01476c8-1aed-4fd1-acb0-6aea531856ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ec5e0f-6b5968df56-7mddg"
Oct 30 19:51:16.158821 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:16.158748    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ec5e0f-6b5968df56-7mddg"
Oct 30 19:51:16.542875 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:16.533491    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-ec5e0f-6b5968df56-7mddg"]
Oct 30 19:51:16.568675 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:51:16.568630    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb01476c8_1aed_4fd1_acb0_6aea531856ca.slice/crio-a1416d7d4df522056e54528dce60b1416db181db3448e6b50bfba8edffad3c32 WatchSource:0}: Error finding container a1416d7d4df522056e54528dce60b1416db181db3448e6b50bfba8edffad3c32: Status 404 returned error can't find the container with id a1416d7d4df522056e54528dce60b1416db181db3448e6b50bfba8edffad3c32
Oct 30 19:51:16.574262 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:16.574233    6996 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 19:51:16.794392 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:16.794293    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="003d559b-ba77-46e7-b4bc-4b9dd37653c5" path="/var/lib/kubelet/pods/003d559b-ba77-46e7-b4bc-4b9dd37653c5/volumes"
Oct 30 19:51:16.846121 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:16.846080    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ec5e0f-6b5968df56-7mddg" event={"ID":"b01476c8-1aed-4fd1-acb0-6aea531856ca","Type":"ContainerStarted","Data":"a1416d7d4df522056e54528dce60b1416db181db3448e6b50bfba8edffad3c32"}
Oct 30 19:51:17.855010 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:17.854967    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ec5e0f-6b5968df56-7mddg" event={"ID":"b01476c8-1aed-4fd1-acb0-6aea531856ca","Type":"ContainerStarted","Data":"6beab979d1bcd0f90e148c7e98e57ecfd0cf3ac9883777d5933a41a147f0e1c5"}
Oct 30 19:51:17.855457 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:17.855059    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ec5e0f-6b5968df56-7mddg"
Oct 30 19:51:17.890397 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:17.890347    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ec5e0f-6b5968df56-7mddg" podStartSLOduration=2.466373023 podStartE2EDuration="2.890333197s" totalImagesPullingTime="423.960174ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:51:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:51:17.887449574 +0000 UTC m=+1153.215567729" watchObservedRunningTime="2026-10-30 19:51:17.890333197 +0000 UTC m=+1153.218451423"
Oct 30 19:51:21.472233 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:21.472192    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc"]
Oct 30 19:51:21.472708 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:21.472398    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc" podUID="e6d36c63-b64c-49e6-8c9e-7e939e777275" containerName="istio-proxy" containerID="cri-o://0d6ae7a1560e697ae216988b305c6caf28670c098d7d263200b1925c2ffdd222" gracePeriod=30
Oct 30 19:51:22.316443 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:22.316406    6996 patch_prober.go:29] interesting pod/e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:51:22.316677 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:22.316465    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc" podUID="e6d36c63-b64c-49e6-8c9e-7e939e777275" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:51:23.867915 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:23.867883    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ec5e0f-6b5968df56-7mddg"
Oct 30 19:51:26.762200 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:26.762171    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc"
Oct 30 19:51:26.912093 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:26.912055    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e6d36c63-b64c-49e6-8c9e-7e939e777275-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6d36c63-b64c-49e6-8c9e-7e939e777275-istiod-ca-cert\") pod \"e6d36c63-b64c-49e6-8c9e-7e939e777275\" (UID: \"e6d36c63-b64c-49e6-8c9e-7e939e777275\") "
Oct 30 19:51:26.912289 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:26.912108    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e6d36c63-b64c-49e6-8c9e-7e939e777275-istio-token\" (UniqueName: \"kubernetes.io/projected/e6d36c63-b64c-49e6-8c9e-7e939e777275-istio-token\") pod \"e6d36c63-b64c-49e6-8c9e-7e939e777275\" (UID: \"e6d36c63-b64c-49e6-8c9e-7e939e777275\") "
Oct 30 19:51:26.912289 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:26.912251    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e6d36c63-b64c-49e6-8c9e-7e939e777275-kube-api-access-gg9vp\" (UniqueName: \"kubernetes.io/projected/e6d36c63-b64c-49e6-8c9e-7e939e777275-kube-api-access-gg9vp\") pod \"e6d36c63-b64c-49e6-8c9e-7e939e777275\" (UID: \"e6d36c63-b64c-49e6-8c9e-7e939e777275\") "
Oct 30 19:51:26.912419 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:26.912292    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e6d36c63-b64c-49e6-8c9e-7e939e777275-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6d36c63-b64c-49e6-8c9e-7e939e777275-credential-socket\") pod \"e6d36c63-b64c-49e6-8c9e-7e939e777275\" (UID: \"e6d36c63-b64c-49e6-8c9e-7e939e777275\") "
Oct 30 19:51:26.912419 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:26.912343    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e6d36c63-b64c-49e6-8c9e-7e939e777275-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6d36c63-b64c-49e6-8c9e-7e939e777275-istio-podinfo\") pod \"e6d36c63-b64c-49e6-8c9e-7e939e777275\" (UID: \"e6d36c63-b64c-49e6-8c9e-7e939e777275\") "
Oct 30 19:51:26.912419 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:26.912378    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e6d36c63-b64c-49e6-8c9e-7e939e777275-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6d36c63-b64c-49e6-8c9e-7e939e777275-workload-certs\") pod \"e6d36c63-b64c-49e6-8c9e-7e939e777275\" (UID: \"e6d36c63-b64c-49e6-8c9e-7e939e777275\") "
Oct 30 19:51:26.912632 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:26.912407    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6d36c63-b64c-49e6-8c9e-7e939e777275-istiod-ca-cert" pod "e6d36c63-b64c-49e6-8c9e-7e939e777275" (UID: "e6d36c63-b64c-49e6-8c9e-7e939e777275"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:51:26.912632 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:26.912436    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e6d36c63-b64c-49e6-8c9e-7e939e777275-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6d36c63-b64c-49e6-8c9e-7e939e777275-istio-envoy\") pod \"e6d36c63-b64c-49e6-8c9e-7e939e777275\" (UID: \"e6d36c63-b64c-49e6-8c9e-7e939e777275\") "
Oct 30 19:51:26.912632 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:26.912507    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e6d36c63-b64c-49e6-8c9e-7e939e777275-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6d36c63-b64c-49e6-8c9e-7e939e777275-workload-socket\") pod \"e6d36c63-b64c-49e6-8c9e-7e939e777275\" (UID: \"e6d36c63-b64c-49e6-8c9e-7e939e777275\") "
Oct 30 19:51:26.912632 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:26.912618    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6d36c63-b64c-49e6-8c9e-7e939e777275-credential-socket" pod "e6d36c63-b64c-49e6-8c9e-7e939e777275" (UID: "e6d36c63-b64c-49e6-8c9e-7e939e777275"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:51:26.912836 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:26.912652    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e6d36c63-b64c-49e6-8c9e-7e939e777275-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6d36c63-b64c-49e6-8c9e-7e939e777275-istio-data\") pod \"e6d36c63-b64c-49e6-8c9e-7e939e777275\" (UID: \"e6d36c63-b64c-49e6-8c9e-7e939e777275\") "
Oct 30 19:51:26.912836 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:26.912725    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6d36c63-b64c-49e6-8c9e-7e939e777275-workload-certs" pod "e6d36c63-b64c-49e6-8c9e-7e939e777275" (UID: "e6d36c63-b64c-49e6-8c9e-7e939e777275"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:51:26.912932 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:26.912867    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6d36c63-b64c-49e6-8c9e-7e939e777275-workload-socket" pod "e6d36c63-b64c-49e6-8c9e-7e939e777275" (UID: "e6d36c63-b64c-49e6-8c9e-7e939e777275"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:51:26.913012 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:26.912994    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6d36c63-b64c-49e6-8c9e-7e939e777275-istio-data" pod "e6d36c63-b64c-49e6-8c9e-7e939e777275" (UID: "e6d36c63-b64c-49e6-8c9e-7e939e777275"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:51:26.913382 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:26.913361    6996 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6d36c63-b64c-49e6-8c9e-7e939e777275-istio-data\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:51:26.913574 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:26.913539    6996 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6d36c63-b64c-49e6-8c9e-7e939e777275-istiod-ca-cert\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:51:26.913574 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:26.913566    6996 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6d36c63-b64c-49e6-8c9e-7e939e777275-credential-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:51:26.913771 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:26.913583    6996 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6d36c63-b64c-49e6-8c9e-7e939e777275-workload-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:51:26.913771 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:26.913597    6996 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6d36c63-b64c-49e6-8c9e-7e939e777275-workload-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:51:26.914760 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:26.914733    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e6d36c63-b64c-49e6-8c9e-7e939e777275-istio-podinfo" pod "e6d36c63-b64c-49e6-8c9e-7e939e777275" (UID: "e6d36c63-b64c-49e6-8c9e-7e939e777275"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:51:26.914948 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:26.914922    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6d36c63-b64c-49e6-8c9e-7e939e777275-kube-api-access-gg9vp" pod "e6d36c63-b64c-49e6-8c9e-7e939e777275" (UID: "e6d36c63-b64c-49e6-8c9e-7e939e777275"). InnerVolumeSpecName "kube-api-access-gg9vp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:51:26.915050 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:26.914955    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6d36c63-b64c-49e6-8c9e-7e939e777275-istio-envoy" pod "e6d36c63-b64c-49e6-8c9e-7e939e777275" (UID: "e6d36c63-b64c-49e6-8c9e-7e939e777275"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:51:26.915050 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:26.915028    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6d36c63-b64c-49e6-8c9e-7e939e777275-istio-token" pod "e6d36c63-b64c-49e6-8c9e-7e939e777275" (UID: "e6d36c63-b64c-49e6-8c9e-7e939e777275"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:51:26.923424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:26.923399    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f"]
Oct 30 19:51:26.924146 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:26.924127    6996 cpu_manager.go:414] "containerMap: removing container" podUID="e6d36c63-b64c-49e6-8c9e-7e939e777275" containerName="istio-proxy"
Oct 30 19:51:26.924238 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:26.924147    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e6d36c63-b64c-49e6-8c9e-7e939e777275" containerName="istio-proxy"
Oct 30 19:51:26.924238 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:26.924187    6996 cpu_manager.go:414] "containerMap: removing container" podUID="e6d36c63-b64c-49e6-8c9e-7e939e777275" containerName="istio-proxy"
Oct 30 19:51:26.924238 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:26.924195    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e6d36c63-b64c-49e6-8c9e-7e939e777275" containerName="istio-proxy"
Oct 30 19:51:26.924402 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:26.924387    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="e6d36c63-b64c-49e6-8c9e-7e939e777275" containerName="istio-proxy"
Oct 30 19:51:26.924457 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:26.924410    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="e6d36c63-b64c-49e6-8c9e-7e939e777275" containerName="istio-proxy"
Oct 30 19:51:26.926273 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:26.926251    6996 generic.go:356] "Generic (PLEG): container finished" podID="e6d36c63-b64c-49e6-8c9e-7e939e777275" containerID="0d6ae7a1560e697ae216988b305c6caf28670c098d7d263200b1925c2ffdd222" exitCode=0
Oct 30 19:51:26.926377 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:26.926362    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc"
Oct 30 19:51:26.928542 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:26.928518    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc" event={"ID":"e6d36c63-b64c-49e6-8c9e-7e939e777275","Type":"ContainerDied","Data":"0d6ae7a1560e697ae216988b305c6caf28670c098d7d263200b1925c2ffdd222"}
Oct 30 19:51:26.928637 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:26.928558    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc" event={"ID":"e6d36c63-b64c-49e6-8c9e-7e939e777275","Type":"ContainerDied","Data":"07879f2e5fa731757fd7f574ea34b3fb2c701c68327455bf6ab9bae528bbef5a"}
Oct 30 19:51:26.928702 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:26.928644    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f"
Oct 30 19:51:26.928758 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:26.928646    6996 scope.go:122] "RemoveContainer" containerID="0d6ae7a1560e697ae216988b305c6caf28670c098d7d263200b1925c2ffdd222"
Oct 30 19:51:26.934859 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:26.934839    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-913351-openshift-default-dockercfg-wwrg9\""
Oct 30 19:51:26.951848 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:26.951821    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f"]
Oct 30 19:51:26.953594 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:26.953520    6996 scope.go:122] "RemoveContainer" containerID="c673f266600df1d3edc7064e0f0b7a22751b4e27d117c7e3f16a5b1e64c2ab80"
Oct 30 19:51:27.008637 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:27.008607    6996 scope.go:122] "RemoveContainer" containerID="0d6ae7a1560e697ae216988b305c6caf28670c098d7d263200b1925c2ffdd222"
Oct 30 19:51:27.009070 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:51:27.009031    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d6ae7a1560e697ae216988b305c6caf28670c098d7d263200b1925c2ffdd222\": container with ID starting with 0d6ae7a1560e697ae216988b305c6caf28670c098d7d263200b1925c2ffdd222 not found: ID does not exist" containerID="0d6ae7a1560e697ae216988b305c6caf28670c098d7d263200b1925c2ffdd222"
Oct 30 19:51:27.009205 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:27.009081    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d6ae7a1560e697ae216988b305c6caf28670c098d7d263200b1925c2ffdd222"} err="failed to get container status \"0d6ae7a1560e697ae216988b305c6caf28670c098d7d263200b1925c2ffdd222\": rpc error: code = NotFound desc = could not find container \"0d6ae7a1560e697ae216988b305c6caf28670c098d7d263200b1925c2ffdd222\": container with ID starting with 0d6ae7a1560e697ae216988b305c6caf28670c098d7d263200b1925c2ffdd222 not found: ID does not exist"
Oct 30 19:51:27.009205 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:27.009106    6996 scope.go:122] "RemoveContainer" containerID="c673f266600df1d3edc7064e0f0b7a22751b4e27d117c7e3f16a5b1e64c2ab80"
Oct 30 19:51:27.009543 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:51:27.009466    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c673f266600df1d3edc7064e0f0b7a22751b4e27d117c7e3f16a5b1e64c2ab80\": container with ID starting with c673f266600df1d3edc7064e0f0b7a22751b4e27d117c7e3f16a5b1e64c2ab80 not found: ID does not exist" containerID="c673f266600df1d3edc7064e0f0b7a22751b4e27d117c7e3f16a5b1e64c2ab80"
Oct 30 19:51:27.009640 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:27.009556    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c673f266600df1d3edc7064e0f0b7a22751b4e27d117c7e3f16a5b1e64c2ab80"} err="failed to get container status \"c673f266600df1d3edc7064e0f0b7a22751b4e27d117c7e3f16a5b1e64c2ab80\": rpc error: code = NotFound desc = could not find container \"c673f266600df1d3edc7064e0f0b7a22751b4e27d117c7e3f16a5b1e64c2ab80\": container with ID starting with c673f266600df1d3edc7064e0f0b7a22751b4e27d117c7e3f16a5b1e64c2ab80 not found: ID does not exist"
Oct 30 19:51:27.015295 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:27.015243    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29afff26-d867-4aaf-9d15-ea2f45d4a091-workload-certs\") pod \"e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f\" (UID: \"29afff26-d867-4aaf-9d15-ea2f45d4a091\") " pod="openshift-ingress/e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f"
Oct 30 19:51:27.015407 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:27.015350    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29afff26-d867-4aaf-9d15-ea2f45d4a091-istio-data\") pod \"e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f\" (UID: \"29afff26-d867-4aaf-9d15-ea2f45d4a091\") " pod="openshift-ingress/e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f"
Oct 30 19:51:27.015407 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:27.015389    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29afff26-d867-4aaf-9d15-ea2f45d4a091-istio-podinfo\") pod \"e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f\" (UID: \"29afff26-d867-4aaf-9d15-ea2f45d4a091\") " pod="openshift-ingress/e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f"
Oct 30 19:51:27.015539 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:27.015458    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29afff26-d867-4aaf-9d15-ea2f45d4a091-credential-socket\") pod \"e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f\" (UID: \"29afff26-d867-4aaf-9d15-ea2f45d4a091\") " pod="openshift-ingress/e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f"
Oct 30 19:51:27.015539 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:27.015511    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29afff26-d867-4aaf-9d15-ea2f45d4a091-istio-token\") pod \"e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f\" (UID: \"29afff26-d867-4aaf-9d15-ea2f45d4a091\") " pod="openshift-ingress/e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f"
Oct 30 19:51:27.015642 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:27.015571    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29afff26-d867-4aaf-9d15-ea2f45d4a091-istiod-ca-cert\") pod \"e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f\" (UID: \"29afff26-d867-4aaf-9d15-ea2f45d4a091\") " pod="openshift-ingress/e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f"
Oct 30 19:51:27.015824 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:27.015807    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29afff26-d867-4aaf-9d15-ea2f45d4a091-istio-envoy\") pod \"e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f\" (UID: \"29afff26-d867-4aaf-9d15-ea2f45d4a091\") " pod="openshift-ingress/e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f"
Oct 30 19:51:27.015920 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:27.015884    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29afff26-d867-4aaf-9d15-ea2f45d4a091-workload-socket\") pod \"e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f\" (UID: \"29afff26-d867-4aaf-9d15-ea2f45d4a091\") " pod="openshift-ingress/e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f"
Oct 30 19:51:27.016048 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:27.016031    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96rmp\" (UniqueName: \"kubernetes.io/projected/29afff26-d867-4aaf-9d15-ea2f45d4a091-kube-api-access-96rmp\") pod \"e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f\" (UID: \"29afff26-d867-4aaf-9d15-ea2f45d4a091\") " pod="openshift-ingress/e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f"
Oct 30 19:51:27.016203 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:27.016190    6996 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6d36c63-b64c-49e6-8c9e-7e939e777275-istio-token\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:51:27.016240 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:27.016212    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gg9vp\" (UniqueName: \"kubernetes.io/projected/e6d36c63-b64c-49e6-8c9e-7e939e777275-kube-api-access-gg9vp\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:51:27.016240 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:27.016228    6996 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6d36c63-b64c-49e6-8c9e-7e939e777275-istio-podinfo\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:51:27.016302 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:27.016242    6996 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6d36c63-b64c-49e6-8c9e-7e939e777275-istio-envoy\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:51:27.020768 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:27.020744    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc"]
Oct 30 19:51:27.025858 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:27.025832    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-1e3723-openshift-default-6fd569f6bc-7j6rc"]
Oct 30 19:51:27.118076 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:27.118035    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29afff26-d867-4aaf-9d15-ea2f45d4a091-istio-data\") pod \"e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f\" (UID: \"29afff26-d867-4aaf-9d15-ea2f45d4a091\") " pod="openshift-ingress/e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f"
Oct 30 19:51:27.118076 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:27.118076    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29afff26-d867-4aaf-9d15-ea2f45d4a091-istio-podinfo\") pod \"e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f\" (UID: \"29afff26-d867-4aaf-9d15-ea2f45d4a091\") " pod="openshift-ingress/e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f"
Oct 30 19:51:27.118300 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:27.118128    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29afff26-d867-4aaf-9d15-ea2f45d4a091-credential-socket\") pod \"e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f\" (UID: \"29afff26-d867-4aaf-9d15-ea2f45d4a091\") " pod="openshift-ingress/e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f"
Oct 30 19:51:27.118300 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:27.118149    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29afff26-d867-4aaf-9d15-ea2f45d4a091-istio-token\") pod \"e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f\" (UID: \"29afff26-d867-4aaf-9d15-ea2f45d4a091\") " pod="openshift-ingress/e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f"
Oct 30 19:51:27.118300 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:27.118227    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29afff26-d867-4aaf-9d15-ea2f45d4a091-istiod-ca-cert\") pod \"e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f\" (UID: \"29afff26-d867-4aaf-9d15-ea2f45d4a091\") " pod="openshift-ingress/e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f"
Oct 30 19:51:27.118441 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:27.118395    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29afff26-d867-4aaf-9d15-ea2f45d4a091-istio-envoy\") pod \"e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f\" (UID: \"29afff26-d867-4aaf-9d15-ea2f45d4a091\") " pod="openshift-ingress/e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f"
Oct 30 19:51:27.118539 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:27.118471    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29afff26-d867-4aaf-9d15-ea2f45d4a091-workload-socket\") pod \"e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f\" (UID: \"29afff26-d867-4aaf-9d15-ea2f45d4a091\") " pod="openshift-ingress/e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f"
Oct 30 19:51:27.118597 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:27.118565    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29afff26-d867-4aaf-9d15-ea2f45d4a091-istio-data\") pod \"e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f\" (UID: \"29afff26-d867-4aaf-9d15-ea2f45d4a091\") " pod="openshift-ingress/e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f"
Oct 30 19:51:27.118935 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:27.118610    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96rmp\" (UniqueName: \"kubernetes.io/projected/29afff26-d867-4aaf-9d15-ea2f45d4a091-kube-api-access-96rmp\") pod \"e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f\" (UID: \"29afff26-d867-4aaf-9d15-ea2f45d4a091\") " pod="openshift-ingress/e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f"
Oct 30 19:51:27.118935 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:27.118613    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29afff26-d867-4aaf-9d15-ea2f45d4a091-credential-socket\") pod \"e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f\" (UID: \"29afff26-d867-4aaf-9d15-ea2f45d4a091\") " pod="openshift-ingress/e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f"
Oct 30 19:51:27.118935 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:27.118773    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29afff26-d867-4aaf-9d15-ea2f45d4a091-workload-certs\") pod \"e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f\" (UID: \"29afff26-d867-4aaf-9d15-ea2f45d4a091\") " pod="openshift-ingress/e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f"
Oct 30 19:51:27.118935 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:27.118845    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29afff26-d867-4aaf-9d15-ea2f45d4a091-workload-socket\") pod \"e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f\" (UID: \"29afff26-d867-4aaf-9d15-ea2f45d4a091\") " pod="openshift-ingress/e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f"
Oct 30 19:51:27.119196 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:27.119056    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29afff26-d867-4aaf-9d15-ea2f45d4a091-istiod-ca-cert\") pod \"e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f\" (UID: \"29afff26-d867-4aaf-9d15-ea2f45d4a091\") " pod="openshift-ingress/e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f"
Oct 30 19:51:27.119196 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:27.119126    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29afff26-d867-4aaf-9d15-ea2f45d4a091-workload-certs\") pod \"e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f\" (UID: \"29afff26-d867-4aaf-9d15-ea2f45d4a091\") " pod="openshift-ingress/e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f"
Oct 30 19:51:27.120657 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:27.120635    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29afff26-d867-4aaf-9d15-ea2f45d4a091-istio-envoy\") pod \"e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f\" (UID: \"29afff26-d867-4aaf-9d15-ea2f45d4a091\") " pod="openshift-ingress/e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f"
Oct 30 19:51:27.120938 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:27.120922    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29afff26-d867-4aaf-9d15-ea2f45d4a091-istio-podinfo\") pod \"e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f\" (UID: \"29afff26-d867-4aaf-9d15-ea2f45d4a091\") " pod="openshift-ingress/e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f"
Oct 30 19:51:27.133874 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:27.133842    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-96rmp\" (UniqueName: \"kubernetes.io/projected/29afff26-d867-4aaf-9d15-ea2f45d4a091-kube-api-access-96rmp\") pod \"e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f\" (UID: \"29afff26-d867-4aaf-9d15-ea2f45d4a091\") " pod="openshift-ingress/e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f"
Oct 30 19:51:27.134168 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:27.134143    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29afff26-d867-4aaf-9d15-ea2f45d4a091-istio-token\") pod \"e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f\" (UID: \"29afff26-d867-4aaf-9d15-ea2f45d4a091\") " pod="openshift-ingress/e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f"
Oct 30 19:51:27.298183 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:27.298101    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f"
Oct 30 19:51:27.597168 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:27.597142    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f"]
Oct 30 19:51:27.598858 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:51:27.598829    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29afff26_d867_4aaf_9d15_ea2f45d4a091.slice/crio-19d171d9f816b93e9085b2f8a5ccbc2db1b0a154dc2c6eb11ec5a56e11076a89 WatchSource:0}: Error finding container 19d171d9f816b93e9085b2f8a5ccbc2db1b0a154dc2c6eb11ec5a56e11076a89: Status 404 returned error can't find the container with id 19d171d9f816b93e9085b2f8a5ccbc2db1b0a154dc2c6eb11ec5a56e11076a89
Oct 30 19:51:27.601773 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:27.601733    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:51:27.601904 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:27.601804    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:51:27.601904 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:27.601834    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:51:27.952066 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:27.951934    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f" event={"ID":"29afff26-d867-4aaf-9d15-ea2f45d4a091","Type":"ContainerStarted","Data":"2a02fb5bcf9c579f93759152f2aba99bcca11e186939fd23f9bb8e5e24ddbb5a"}
Oct 30 19:51:27.952066 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:27.952006    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f" event={"ID":"29afff26-d867-4aaf-9d15-ea2f45d4a091","Type":"ContainerStarted","Data":"19d171d9f816b93e9085b2f8a5ccbc2db1b0a154dc2c6eb11ec5a56e11076a89"}
Oct 30 19:51:27.985969 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:27.985909    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f" podStartSLOduration=1.9858947009999999 podStartE2EDuration="1.985894701s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:51:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:51:27.981728788 +0000 UTC m=+1163.309846942" watchObservedRunningTime="2026-10-30 19:51:27.985894701 +0000 UTC m=+1163.314012855"
Oct 30 19:51:28.298546 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:28.298451    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f"
Oct 30 19:51:28.299782 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:28.299755    6996 patch_prober.go:29] interesting pod/e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.126:15021/healthz/ready\": dial tcp 10.134.0.126:15021: connect: connection refused" start-of-body=
Oct 30 19:51:28.299900 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:28.299813    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f" podUID="29afff26-d867-4aaf-9d15-ea2f45d4a091" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.126:15021/healthz/ready\": dial tcp 10.134.0.126:15021: connect: connection refused"
Oct 30 19:51:28.793867 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:28.793833    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e6d36c63-b64c-49e6-8c9e-7e939e777275" path="/var/lib/kubelet/pods/e6d36c63-b64c-49e6-8c9e-7e939e777275/volumes"
Oct 30 19:51:29.309505 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:29.307863    6996 patch_prober.go:29] interesting pod/e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.126:15021/healthz/ready\": dial tcp 10.134.0.126:15021: connect: connection refused" start-of-body=
Oct 30 19:51:29.309505 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:29.307926    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f" podUID="29afff26-d867-4aaf-9d15-ea2f45d4a091" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.126:15021/healthz/ready\": dial tcp 10.134.0.126:15021: connect: connection refused"
Oct 30 19:51:30.299250 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:30.299213    6996 patch_prober.go:29] interesting pod/e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.126:15021/healthz/ready\": dial tcp 10.134.0.126:15021: connect: connection refused" start-of-body=
Oct 30 19:51:30.299410 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:30.299273    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f" podUID="29afff26-d867-4aaf-9d15-ea2f45d4a091" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.126:15021/healthz/ready\": dial tcp 10.134.0.126:15021: connect: connection refused"
Oct 30 19:51:31.298441 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:31.298410    6996 patch_prober.go:29] interesting pod/e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.126:15021/healthz/ready\": dial tcp 10.134.0.126:15021: connect: connection refused" start-of-body=
Oct 30 19:51:31.298947 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:31.298469    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f" podUID="29afff26-d867-4aaf-9d15-ea2f45d4a091" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.126:15021/healthz/ready\": dial tcp 10.134.0.126:15021: connect: connection refused"
Oct 30 19:51:31.299160 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:51:31.299129    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 19:51:32.298756 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:32.298719    6996 patch_prober.go:29] interesting pod/e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.126:15021/healthz/ready\": dial tcp 10.134.0.126:15021: connect: connection refused" start-of-body=
Oct 30 19:51:32.299258 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:32.298784    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f" podUID="29afff26-d867-4aaf-9d15-ea2f45d4a091" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.126:15021/healthz/ready\": dial tcp 10.134.0.126:15021: connect: connection refused"
Oct 30 19:51:32.308642 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:32.308612    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-2beb7b-6f9c4fb479-5cpxb"]
Oct 30 19:51:33.299039 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:33.299006    6996 patch_prober.go:29] interesting pod/e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.126:15021/healthz/ready\": dial tcp 10.134.0.126:15021: connect: connection refused" start-of-body=
Oct 30 19:51:33.299404 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:33.299059    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f" podUID="29afff26-d867-4aaf-9d15-ea2f45d4a091" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.126:15021/healthz/ready\": dial tcp 10.134.0.126:15021: connect: connection refused"
Oct 30 19:51:34.303243 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:34.303212    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f"
Oct 30 19:51:34.303680 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:34.303662    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f"
Oct 30 19:51:34.304346 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:34.304327    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f"
Oct 30 19:51:34.932794 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:34.932762    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-e907e4-7547657bb4-s8x4l"]
Oct 30 19:51:34.932973 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:34.932939    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e907e4-7547657bb4-s8x4l" podUID="8ffc6067-3cf9-4a43-8a7c-3b210debc122" containerName="maas-api" containerID="cri-o://aa24a7c186e4b726b9f5ef6ae231ee1be6dd0c40bd4ec1d6305fab391f6cc480" gracePeriod=30
Oct 30 19:51:35.368761 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:35.368736    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e907e4-7547657bb4-s8x4l"
Oct 30 19:51:35.437309 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:35.437275    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8ffc6067-3cf9-4a43-8a7c-3b210debc122-kube-api-access-gf4cn\" (UniqueName: \"kubernetes.io/projected/8ffc6067-3cf9-4a43-8a7c-3b210debc122-kube-api-access-gf4cn\") pod \"8ffc6067-3cf9-4a43-8a7c-3b210debc122\" (UID: \"8ffc6067-3cf9-4a43-8a7c-3b210debc122\") "
Oct 30 19:51:35.437474 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:35.437370    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8ffc6067-3cf9-4a43-8a7c-3b210debc122-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ffc6067-3cf9-4a43-8a7c-3b210debc122-openshift-service-ca\") pod \"8ffc6067-3cf9-4a43-8a7c-3b210debc122\" (UID: \"8ffc6067-3cf9-4a43-8a7c-3b210debc122\") "
Oct 30 19:51:35.437577 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:35.437518    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8ffc6067-3cf9-4a43-8a7c-3b210debc122-metrics-certs\" (UniqueName: \"kubernetes.io/secret/8ffc6067-3cf9-4a43-8a7c-3b210debc122-metrics-certs\") pod \"8ffc6067-3cf9-4a43-8a7c-3b210debc122\" (UID: \"8ffc6067-3cf9-4a43-8a7c-3b210debc122\") "
Oct 30 19:51:35.437645 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:35.437628    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8ffc6067-3cf9-4a43-8a7c-3b210debc122-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ffc6067-3cf9-4a43-8a7c-3b210debc122-maas-api-tls\") pod \"8ffc6067-3cf9-4a43-8a7c-3b210debc122\" (UID: \"8ffc6067-3cf9-4a43-8a7c-3b210debc122\") "
Oct 30 19:51:35.437805 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:35.437777    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ffc6067-3cf9-4a43-8a7c-3b210debc122-openshift-service-ca" pod "8ffc6067-3cf9-4a43-8a7c-3b210debc122" (UID: "8ffc6067-3cf9-4a43-8a7c-3b210debc122"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:51:35.438401 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:35.438375    6996 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ffc6067-3cf9-4a43-8a7c-3b210debc122-openshift-service-ca\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:51:35.439704 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:35.439675    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ffc6067-3cf9-4a43-8a7c-3b210debc122-metrics-certs" pod "8ffc6067-3cf9-4a43-8a7c-3b210debc122" (UID: "8ffc6067-3cf9-4a43-8a7c-3b210debc122"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:51:35.440125 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:35.440098    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ffc6067-3cf9-4a43-8a7c-3b210debc122-kube-api-access-gf4cn" pod "8ffc6067-3cf9-4a43-8a7c-3b210debc122" (UID: "8ffc6067-3cf9-4a43-8a7c-3b210debc122"). InnerVolumeSpecName "kube-api-access-gf4cn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:51:35.440376 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:35.440357    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ffc6067-3cf9-4a43-8a7c-3b210debc122-maas-api-tls" pod "8ffc6067-3cf9-4a43-8a7c-3b210debc122" (UID: "8ffc6067-3cf9-4a43-8a7c-3b210debc122"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:51:35.540155 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:35.540083    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gf4cn\" (UniqueName: \"kubernetes.io/projected/8ffc6067-3cf9-4a43-8a7c-3b210debc122-kube-api-access-gf4cn\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:51:35.540155 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:35.540112    6996 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8ffc6067-3cf9-4a43-8a7c-3b210debc122-metrics-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:51:35.540155 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:35.540122    6996 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ffc6067-3cf9-4a43-8a7c-3b210debc122-maas-api-tls\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:51:36.041645 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:36.041611    6996 generic.go:356] "Generic (PLEG): container finished" podID="8ffc6067-3cf9-4a43-8a7c-3b210debc122" containerID="aa24a7c186e4b726b9f5ef6ae231ee1be6dd0c40bd4ec1d6305fab391f6cc480" exitCode=0
Oct 30 19:51:36.041839 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:36.041665    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e907e4-7547657bb4-s8x4l"
Oct 30 19:51:36.041839 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:36.041664    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e907e4-7547657bb4-s8x4l" event={"ID":"8ffc6067-3cf9-4a43-8a7c-3b210debc122","Type":"ContainerDied","Data":"aa24a7c186e4b726b9f5ef6ae231ee1be6dd0c40bd4ec1d6305fab391f6cc480"}
Oct 30 19:51:36.041839 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:36.041789    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e907e4-7547657bb4-s8x4l" event={"ID":"8ffc6067-3cf9-4a43-8a7c-3b210debc122","Type":"ContainerDied","Data":"6bc36468981adc59929f8e8ed5da11c6bcb865cac181d4fcebf2147d51494c08"}
Oct 30 19:51:36.041839 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:36.041811    6996 scope.go:122] "RemoveContainer" containerID="aa24a7c186e4b726b9f5ef6ae231ee1be6dd0c40bd4ec1d6305fab391f6cc480"
Oct 30 19:51:36.063131 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:36.063106    6996 scope.go:122] "RemoveContainer" containerID="aa24a7c186e4b726b9f5ef6ae231ee1be6dd0c40bd4ec1d6305fab391f6cc480"
Oct 30 19:51:36.063450 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:51:36.063422    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa24a7c186e4b726b9f5ef6ae231ee1be6dd0c40bd4ec1d6305fab391f6cc480\": container with ID starting with aa24a7c186e4b726b9f5ef6ae231ee1be6dd0c40bd4ec1d6305fab391f6cc480 not found: ID does not exist" containerID="aa24a7c186e4b726b9f5ef6ae231ee1be6dd0c40bd4ec1d6305fab391f6cc480"
Oct 30 19:51:36.063573 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:36.063455    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa24a7c186e4b726b9f5ef6ae231ee1be6dd0c40bd4ec1d6305fab391f6cc480"} err="failed to get container status \"aa24a7c186e4b726b9f5ef6ae231ee1be6dd0c40bd4ec1d6305fab391f6cc480\": rpc error: code = NotFound desc = could not find container \"aa24a7c186e4b726b9f5ef6ae231ee1be6dd0c40bd4ec1d6305fab391f6cc480\": container with ID starting with aa24a7c186e4b726b9f5ef6ae231ee1be6dd0c40bd4ec1d6305fab391f6cc480 not found: ID does not exist"
Oct 30 19:51:36.075237 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:36.075209    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-e907e4-7547657bb4-s8x4l"]
Oct 30 19:51:36.079452 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:36.079425    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-e907e4-7547657bb4-s8x4l"]
Oct 30 19:51:36.793071 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:36.793038    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8ffc6067-3cf9-4a43-8a7c-3b210debc122" path="/var/lib/kubelet/pods/8ffc6067-3cf9-4a43-8a7c-3b210debc122/volumes"
Oct 30 19:51:37.309668 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:37.309610    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-2beb7b-6f9c4fb479-5cpxb" podUID="8937dc08-19b1-4247-a174-e80fc16b1fe3" containerName="payload-pre-processing" containerID="cri-o://a450e880007c12730f558c92a8d335bec4d00045b0aee58679a426ec58517f83" gracePeriod=25
Oct 30 19:51:37.600899 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:37.600875    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-2beb7b-6f9c4fb479-5cpxb"
Oct 30 19:51:37.683930 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:37.683889    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8937dc08-19b1-4247-a174-e80fc16b1fe3-kube-api-access-97lz8\" (UniqueName: \"kubernetes.io/projected/8937dc08-19b1-4247-a174-e80fc16b1fe3-kube-api-access-97lz8\") pod \"8937dc08-19b1-4247-a174-e80fc16b1fe3\" (UID: \"8937dc08-19b1-4247-a174-e80fc16b1fe3\") "
Oct 30 19:51:37.684253 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:37.684102    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8937dc08-19b1-4247-a174-e80fc16b1fe3-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8937dc08-19b1-4247-a174-e80fc16b1fe3-plugins-config-volume\") pod \"8937dc08-19b1-4247-a174-e80fc16b1fe3\" (UID: \"8937dc08-19b1-4247-a174-e80fc16b1fe3\") "
Oct 30 19:51:37.685135 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:37.685105    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8937dc08-19b1-4247-a174-e80fc16b1fe3-plugins-config-volume" pod "8937dc08-19b1-4247-a174-e80fc16b1fe3" (UID: "8937dc08-19b1-4247-a174-e80fc16b1fe3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:51:37.687598 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:37.687570    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8937dc08-19b1-4247-a174-e80fc16b1fe3-kube-api-access-97lz8" pod "8937dc08-19b1-4247-a174-e80fc16b1fe3" (UID: "8937dc08-19b1-4247-a174-e80fc16b1fe3"). InnerVolumeSpecName "kube-api-access-97lz8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:51:37.786252 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:37.786216    6996 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8937dc08-19b1-4247-a174-e80fc16b1fe3-plugins-config-volume\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:51:37.786252 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:37.786247    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-97lz8\" (UniqueName: \"kubernetes.io/projected/8937dc08-19b1-4247-a174-e80fc16b1fe3-kube-api-access-97lz8\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:51:38.059779 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:38.059695    6996 generic.go:356] "Generic (PLEG): container finished" podID="8937dc08-19b1-4247-a174-e80fc16b1fe3" containerID="a450e880007c12730f558c92a8d335bec4d00045b0aee58679a426ec58517f83" exitCode=0
Oct 30 19:51:38.059779 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:38.059743    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-2beb7b-6f9c4fb479-5cpxb"
Oct 30 19:51:38.060226 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:38.059780    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-2beb7b-6f9c4fb479-5cpxb" event={"ID":"8937dc08-19b1-4247-a174-e80fc16b1fe3","Type":"ContainerDied","Data":"a450e880007c12730f558c92a8d335bec4d00045b0aee58679a426ec58517f83"}
Oct 30 19:51:38.060226 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:38.059838    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-2beb7b-6f9c4fb479-5cpxb" event={"ID":"8937dc08-19b1-4247-a174-e80fc16b1fe3","Type":"ContainerDied","Data":"7b7ce6e3ee86cd24c774a4aaac4374e6db9f3025d81e7dad31b18ed9cc935794"}
Oct 30 19:51:38.060226 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:38.059866    6996 scope.go:122] "RemoveContainer" containerID="a450e880007c12730f558c92a8d335bec4d00045b0aee58679a426ec58517f83"
Oct 30 19:51:38.070901 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:38.070881    6996 scope.go:122] "RemoveContainer" containerID="a450e880007c12730f558c92a8d335bec4d00045b0aee58679a426ec58517f83"
Oct 30 19:51:38.071177 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:51:38.071159    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a450e880007c12730f558c92a8d335bec4d00045b0aee58679a426ec58517f83\": container with ID starting with a450e880007c12730f558c92a8d335bec4d00045b0aee58679a426ec58517f83 not found: ID does not exist" containerID="a450e880007c12730f558c92a8d335bec4d00045b0aee58679a426ec58517f83"
Oct 30 19:51:38.071215 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:38.071185    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a450e880007c12730f558c92a8d335bec4d00045b0aee58679a426ec58517f83"} err="failed to get container status \"a450e880007c12730f558c92a8d335bec4d00045b0aee58679a426ec58517f83\": rpc error: code = NotFound desc = could not find container \"a450e880007c12730f558c92a8d335bec4d00045b0aee58679a426ec58517f83\": container with ID starting with a450e880007c12730f558c92a8d335bec4d00045b0aee58679a426ec58517f83 not found: ID does not exist"
Oct 30 19:51:38.099235 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:38.099201    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-2beb7b-6f9c4fb479-5cpxb"]
Oct 30 19:51:38.102817 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:38.102794    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-2beb7b-6f9c4fb479-5cpxb"]
Oct 30 19:51:38.802038 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:38.802007    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8937dc08-19b1-4247-a174-e80fc16b1fe3" path="/var/lib/kubelet/pods/8937dc08-19b1-4247-a174-e80fc16b1fe3/volumes"
Oct 30 19:51:40.813333 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:40.813298    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8"]
Oct 30 19:51:40.813333 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:40.814346    6996 cpu_manager.go:414] "containerMap: removing container" podUID="8ffc6067-3cf9-4a43-8a7c-3b210debc122" containerName="maas-api"
Oct 30 19:51:40.813333 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:40.814370    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8ffc6067-3cf9-4a43-8a7c-3b210debc122" containerName="maas-api"
Oct 30 19:51:40.813333 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:40.814414    6996 cpu_manager.go:414] "containerMap: removing container" podUID="8937dc08-19b1-4247-a174-e80fc16b1fe3" containerName="payload-pre-processing"
Oct 30 19:51:40.813333 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:40.814424    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8937dc08-19b1-4247-a174-e80fc16b1fe3" containerName="payload-pre-processing"
Oct 30 19:51:40.813333 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:40.814650    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="8ffc6067-3cf9-4a43-8a7c-3b210debc122" containerName="maas-api"
Oct 30 19:51:40.813333 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:40.814671    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="8937dc08-19b1-4247-a174-e80fc16b1fe3" containerName="payload-pre-processing"
Oct 30 19:51:40.820117 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:40.820098    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8"
Oct 30 19:51:40.867890 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:40.867852    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8"]
Oct 30 19:51:40.884526 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:40.884460    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-ec5e0f-openshift-default-dockercfg-2xxlz\""
Oct 30 19:51:40.968955 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:40.968912    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81198f3c-12bb-4356-bc2e-1134a145c971-istio-envoy\") pod \"e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8\" (UID: \"81198f3c-12bb-4356-bc2e-1134a145c971\") " pod="openshift-ingress/e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8"
Oct 30 19:51:40.969132 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:40.968985    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81198f3c-12bb-4356-bc2e-1134a145c971-istio-token\") pod \"e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8\" (UID: \"81198f3c-12bb-4356-bc2e-1134a145c971\") " pod="openshift-ingress/e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8"
Oct 30 19:51:40.969245 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:40.969219    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81198f3c-12bb-4356-bc2e-1134a145c971-credential-socket\") pod \"e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8\" (UID: \"81198f3c-12bb-4356-bc2e-1134a145c971\") " pod="openshift-ingress/e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8"
Oct 30 19:51:40.969295 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:40.969274    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81198f3c-12bb-4356-bc2e-1134a145c971-workload-certs\") pod \"e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8\" (UID: \"81198f3c-12bb-4356-bc2e-1134a145c971\") " pod="openshift-ingress/e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8"
Oct 30 19:51:40.969393 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:40.969375    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81198f3c-12bb-4356-bc2e-1134a145c971-istiod-ca-cert\") pod \"e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8\" (UID: \"81198f3c-12bb-4356-bc2e-1134a145c971\") " pod="openshift-ingress/e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8"
Oct 30 19:51:40.969599 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:40.969572    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81198f3c-12bb-4356-bc2e-1134a145c971-workload-socket\") pod \"e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8\" (UID: \"81198f3c-12bb-4356-bc2e-1134a145c971\") " pod="openshift-ingress/e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8"
Oct 30 19:51:40.969878 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:40.969849    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81198f3c-12bb-4356-bc2e-1134a145c971-istio-data\") pod \"e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8\" (UID: \"81198f3c-12bb-4356-bc2e-1134a145c971\") " pod="openshift-ingress/e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8"
Oct 30 19:51:40.969965 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:40.969900    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81198f3c-12bb-4356-bc2e-1134a145c971-istio-podinfo\") pod \"e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8\" (UID: \"81198f3c-12bb-4356-bc2e-1134a145c971\") " pod="openshift-ingress/e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8"
Oct 30 19:51:40.970053 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:40.970020    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kfmf\" (UniqueName: \"kubernetes.io/projected/81198f3c-12bb-4356-bc2e-1134a145c971-kube-api-access-9kfmf\") pod \"e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8\" (UID: \"81198f3c-12bb-4356-bc2e-1134a145c971\") " pod="openshift-ingress/e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8"
Oct 30 19:51:41.071798 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:41.071758    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81198f3c-12bb-4356-bc2e-1134a145c971-credential-socket\") pod \"e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8\" (UID: \"81198f3c-12bb-4356-bc2e-1134a145c971\") " pod="openshift-ingress/e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8"
Oct 30 19:51:41.071984 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:41.071809    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81198f3c-12bb-4356-bc2e-1134a145c971-workload-certs\") pod \"e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8\" (UID: \"81198f3c-12bb-4356-bc2e-1134a145c971\") " pod="openshift-ingress/e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8"
Oct 30 19:51:41.071984 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:41.071857    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81198f3c-12bb-4356-bc2e-1134a145c971-istiod-ca-cert\") pod \"e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8\" (UID: \"81198f3c-12bb-4356-bc2e-1134a145c971\") " pod="openshift-ingress/e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8"
Oct 30 19:51:41.071984 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:41.071942    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81198f3c-12bb-4356-bc2e-1134a145c971-workload-socket\") pod \"e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8\" (UID: \"81198f3c-12bb-4356-bc2e-1134a145c971\") " pod="openshift-ingress/e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8"
Oct 30 19:51:41.072157 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:41.072074    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81198f3c-12bb-4356-bc2e-1134a145c971-istio-data\") pod \"e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8\" (UID: \"81198f3c-12bb-4356-bc2e-1134a145c971\") " pod="openshift-ingress/e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8"
Oct 30 19:51:41.072157 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:41.072100    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81198f3c-12bb-4356-bc2e-1134a145c971-istio-podinfo\") pod \"e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8\" (UID: \"81198f3c-12bb-4356-bc2e-1134a145c971\") " pod="openshift-ingress/e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8"
Oct 30 19:51:41.072157 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:41.072130    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kfmf\" (UniqueName: \"kubernetes.io/projected/81198f3c-12bb-4356-bc2e-1134a145c971-kube-api-access-9kfmf\") pod \"e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8\" (UID: \"81198f3c-12bb-4356-bc2e-1134a145c971\") " pod="openshift-ingress/e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8"
Oct 30 19:51:41.072311 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:41.072186    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81198f3c-12bb-4356-bc2e-1134a145c971-istio-envoy\") pod \"e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8\" (UID: \"81198f3c-12bb-4356-bc2e-1134a145c971\") " pod="openshift-ingress/e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8"
Oct 30 19:51:41.072311 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:41.072232    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81198f3c-12bb-4356-bc2e-1134a145c971-istio-token\") pod \"e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8\" (UID: \"81198f3c-12bb-4356-bc2e-1134a145c971\") " pod="openshift-ingress/e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8"
Oct 30 19:51:41.072311 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:41.072248    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81198f3c-12bb-4356-bc2e-1134a145c971-workload-certs\") pod \"e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8\" (UID: \"81198f3c-12bb-4356-bc2e-1134a145c971\") " pod="openshift-ingress/e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8"
Oct 30 19:51:41.072440 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:41.072387    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81198f3c-12bb-4356-bc2e-1134a145c971-credential-socket\") pod \"e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8\" (UID: \"81198f3c-12bb-4356-bc2e-1134a145c971\") " pod="openshift-ingress/e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8"
Oct 30 19:51:41.072718 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:41.072690    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81198f3c-12bb-4356-bc2e-1134a145c971-workload-socket\") pod \"e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8\" (UID: \"81198f3c-12bb-4356-bc2e-1134a145c971\") " pod="openshift-ingress/e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8"
Oct 30 19:51:41.072863 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:41.072764    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81198f3c-12bb-4356-bc2e-1134a145c971-istio-data\") pod \"e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8\" (UID: \"81198f3c-12bb-4356-bc2e-1134a145c971\") " pod="openshift-ingress/e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8"
Oct 30 19:51:41.073115 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:41.073094    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81198f3c-12bb-4356-bc2e-1134a145c971-istiod-ca-cert\") pod \"e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8\" (UID: \"81198f3c-12bb-4356-bc2e-1134a145c971\") " pod="openshift-ingress/e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8"
Oct 30 19:51:41.075021 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:41.074993    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81198f3c-12bb-4356-bc2e-1134a145c971-istio-envoy\") pod \"e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8\" (UID: \"81198f3c-12bb-4356-bc2e-1134a145c971\") " pod="openshift-ingress/e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8"
Oct 30 19:51:41.075246 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:41.075229    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81198f3c-12bb-4356-bc2e-1134a145c971-istio-podinfo\") pod \"e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8\" (UID: \"81198f3c-12bb-4356-bc2e-1134a145c971\") " pod="openshift-ingress/e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8"
Oct 30 19:51:41.085147 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:41.085114    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81198f3c-12bb-4356-bc2e-1134a145c971-istio-token\") pod \"e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8\" (UID: \"81198f3c-12bb-4356-bc2e-1134a145c971\") " pod="openshift-ingress/e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8"
Oct 30 19:51:41.085782 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:41.085765    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kfmf\" (UniqueName: \"kubernetes.io/projected/81198f3c-12bb-4356-bc2e-1134a145c971-kube-api-access-9kfmf\") pod \"e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8\" (UID: \"81198f3c-12bb-4356-bc2e-1134a145c971\") " pod="openshift-ingress/e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8"
Oct 30 19:51:41.139825 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:41.139803    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8"
Oct 30 19:51:41.448380 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:41.448348    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8"]
Oct 30 19:51:41.449248 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:51:41.449222    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81198f3c_12bb_4356_bc2e_1134a145c971.slice/crio-fa4f669e53a27449f763574439d5173aa309ff54b5d87cf06747caa8d9a26db8 WatchSource:0}: Error finding container fa4f669e53a27449f763574439d5173aa309ff54b5d87cf06747caa8d9a26db8: Status 404 returned error can't find the container with id fa4f669e53a27449f763574439d5173aa309ff54b5d87cf06747caa8d9a26db8
Oct 30 19:51:41.451882 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:41.451841    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:51:41.451972 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:41.451909    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:51:41.451972 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:41.451947    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:51:42.092982 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:42.092951    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8" event={"ID":"81198f3c-12bb-4356-bc2e-1134a145c971","Type":"ContainerStarted","Data":"37b7955bdc83b6382bb1e845ca7152935c5a1d21f28ee2754d7ec560f497f3a9"}
Oct 30 19:51:42.093369 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:42.093001    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8" event={"ID":"81198f3c-12bb-4356-bc2e-1134a145c971","Type":"ContainerStarted","Data":"fa4f669e53a27449f763574439d5173aa309ff54b5d87cf06747caa8d9a26db8"}
Oct 30 19:51:42.124380 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:42.124334    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8" podStartSLOduration=2.124320259 podStartE2EDuration="2.124320259s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:51:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:51:42.120251458 +0000 UTC m=+1177.448369611" watchObservedRunningTime="2026-10-30 19:51:42.124320259 +0000 UTC m=+1177.452438414"
Oct 30 19:51:42.157921 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:42.157883    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8"
Oct 30 19:51:42.168624 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:42.168586    6996 patch_prober.go:29] interesting pod/e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused" start-of-body=
Oct 30 19:51:42.168799 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:42.168650    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8" podUID="81198f3c-12bb-4356-bc2e-1134a145c971" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused"
Oct 30 19:51:43.159839 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:43.159804    6996 patch_prober.go:29] interesting pod/e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused" start-of-body=
Oct 30 19:51:43.160449 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:43.160413    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8" podUID="81198f3c-12bb-4356-bc2e-1134a145c971" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused"
Oct 30 19:51:44.159870 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:44.159832    6996 patch_prober.go:29] interesting pod/e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused" start-of-body=
Oct 30 19:51:44.160463 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:44.160429    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8" podUID="81198f3c-12bb-4356-bc2e-1134a145c971" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused"
Oct 30 19:51:45.140829 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:45.140296    6996 patch_prober.go:29] interesting pod/e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused" start-of-body=
Oct 30 19:51:45.140829 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:45.140357    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8" podUID="81198f3c-12bb-4356-bc2e-1134a145c971" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused"
Oct 30 19:51:46.151557 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:46.148433    6996 patch_prober.go:29] interesting pod/e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused" start-of-body=
Oct 30 19:51:46.151557 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:46.148511    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8" podUID="81198f3c-12bb-4356-bc2e-1134a145c971" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused"
Oct 30 19:51:47.149607 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:47.149568    6996 patch_prober.go:29] interesting pod/e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused" start-of-body=
Oct 30 19:51:47.149895 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:47.149864    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8" podUID="81198f3c-12bb-4356-bc2e-1134a145c971" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused"
Oct 30 19:51:48.144319 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:48.144289    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8"
Oct 30 19:51:48.144801 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:48.144739    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8"
Oct 30 19:51:48.145416 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:48.145395    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8"
Oct 30 19:51:53.768301 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:53.768266    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 19:51:56.544594 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:56.544554    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-f61e4b-66959bc884-hmnz9"]
Oct 30 19:51:56.555878 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:51:56.555838    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-f61e4b-7c54c4f9f5-gv9x9"]
Oct 30 19:52:00.979148 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:00.979113    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5f78687d5b-dkcrs_c72164f7-cc46-4f11-b1f0-f50adf9c4c0b/payload-processing/0.log"
Oct 30 19:52:01.191944 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:52:01.191909    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 19:52:01.200836 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:01.200814    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5f78687d5b-dkcrs_c72164f7-cc46-4f11-b1f0-f50adf9c4c0b/payload-processing/0.log"
Oct 30 19:52:01.545736 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:01.545676    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-f61e4b-66959bc884-hmnz9" podUID="3eafcd7e-b8d7-4f3b-a483-c2c7dc9b764a" containerName="payload-processing" containerID="cri-o://040a37c9c0d37e4b3e663de4afce594e700f256d82adb5605a52744a552bd1a9" gracePeriod=25
Oct 30 19:52:01.556323 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:01.556269    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f61e4b-7c54c4f9f5-gv9x9" podUID="7e524e27-0d0d-42db-b376-0c488d6a0622" containerName="payload-pre-processing" containerID="cri-o://dff798a149b1dc5b0f44cdda21d56089b87d4fe291ec7eb1bb53808bb57b04d7" gracePeriod=25
Oct 30 19:52:01.890935 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:01.890903    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-f61e4b-66959bc884-hmnz9"
Oct 30 19:52:01.908190 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:01.908158    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3eafcd7e-b8d7-4f3b-a483-c2c7dc9b764a-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3eafcd7e-b8d7-4f3b-a483-c2c7dc9b764a-plugins-config-volume\") pod \"3eafcd7e-b8d7-4f3b-a483-c2c7dc9b764a\" (UID: \"3eafcd7e-b8d7-4f3b-a483-c2c7dc9b764a\") "
Oct 30 19:52:01.908362 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:01.908334    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3eafcd7e-b8d7-4f3b-a483-c2c7dc9b764a-kube-api-access-dgnw6\" (UniqueName: \"kubernetes.io/projected/3eafcd7e-b8d7-4f3b-a483-c2c7dc9b764a-kube-api-access-dgnw6\") pod \"3eafcd7e-b8d7-4f3b-a483-c2c7dc9b764a\" (UID: \"3eafcd7e-b8d7-4f3b-a483-c2c7dc9b764a\") "
Oct 30 19:52:01.908613 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:01.908584    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3eafcd7e-b8d7-4f3b-a483-c2c7dc9b764a-plugins-config-volume" pod "3eafcd7e-b8d7-4f3b-a483-c2c7dc9b764a" (UID: "3eafcd7e-b8d7-4f3b-a483-c2c7dc9b764a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:52:01.909112 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:01.909089    6996 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3eafcd7e-b8d7-4f3b-a483-c2c7dc9b764a-plugins-config-volume\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:52:01.910599 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:01.910571    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3eafcd7e-b8d7-4f3b-a483-c2c7dc9b764a-kube-api-access-dgnw6" pod "3eafcd7e-b8d7-4f3b-a483-c2c7dc9b764a" (UID: "3eafcd7e-b8d7-4f3b-a483-c2c7dc9b764a"). InnerVolumeSpecName "kube-api-access-dgnw6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:52:01.936370 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:01.936345    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f61e4b-7c54c4f9f5-gv9x9"
Oct 30 19:52:02.009902 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:02.009866    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7e524e27-0d0d-42db-b376-0c488d6a0622-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7e524e27-0d0d-42db-b376-0c488d6a0622-plugins-config-volume\") pod \"7e524e27-0d0d-42db-b376-0c488d6a0622\" (UID: \"7e524e27-0d0d-42db-b376-0c488d6a0622\") "
Oct 30 19:52:02.010309 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:02.009968    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7e524e27-0d0d-42db-b376-0c488d6a0622-kube-api-access-zcg99\" (UniqueName: \"kubernetes.io/projected/7e524e27-0d0d-42db-b376-0c488d6a0622-kube-api-access-zcg99\") pod \"7e524e27-0d0d-42db-b376-0c488d6a0622\" (UID: \"7e524e27-0d0d-42db-b376-0c488d6a0622\") "
Oct 30 19:52:02.010369 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:02.010341    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e524e27-0d0d-42db-b376-0c488d6a0622-plugins-config-volume" pod "7e524e27-0d0d-42db-b376-0c488d6a0622" (UID: "7e524e27-0d0d-42db-b376-0c488d6a0622"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:52:02.010975 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:02.010948    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dgnw6\" (UniqueName: \"kubernetes.io/projected/3eafcd7e-b8d7-4f3b-a483-c2c7dc9b764a-kube-api-access-dgnw6\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:52:02.011119 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:02.010978    6996 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7e524e27-0d0d-42db-b376-0c488d6a0622-plugins-config-volume\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:52:02.012243 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:02.012216    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e524e27-0d0d-42db-b376-0c488d6a0622-kube-api-access-zcg99" pod "7e524e27-0d0d-42db-b376-0c488d6a0622" (UID: "7e524e27-0d0d-42db-b376-0c488d6a0622"). InnerVolumeSpecName "kube-api-access-zcg99". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:52:02.112699 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:02.112616    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zcg99\" (UniqueName: \"kubernetes.io/projected/7e524e27-0d0d-42db-b376-0c488d6a0622-kube-api-access-zcg99\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:52:02.347861 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:02.347829    6996 generic.go:356] "Generic (PLEG): container finished" podID="7e524e27-0d0d-42db-b376-0c488d6a0622" containerID="dff798a149b1dc5b0f44cdda21d56089b87d4fe291ec7eb1bb53808bb57b04d7" exitCode=0
Oct 30 19:52:02.348044 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:02.347873    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f61e4b-7c54c4f9f5-gv9x9"
Oct 30 19:52:02.348044 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:02.347908    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f61e4b-7c54c4f9f5-gv9x9" event={"ID":"7e524e27-0d0d-42db-b376-0c488d6a0622","Type":"ContainerDied","Data":"dff798a149b1dc5b0f44cdda21d56089b87d4fe291ec7eb1bb53808bb57b04d7"}
Oct 30 19:52:02.348044 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:02.347965    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f61e4b-7c54c4f9f5-gv9x9" event={"ID":"7e524e27-0d0d-42db-b376-0c488d6a0622","Type":"ContainerDied","Data":"3de60ea18e8279992d601c2157ced8b3e154d297be2b97b2772164f8979e7565"}
Oct 30 19:52:02.348044 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:02.347994    6996 scope.go:122] "RemoveContainer" containerID="dff798a149b1dc5b0f44cdda21d56089b87d4fe291ec7eb1bb53808bb57b04d7"
Oct 30 19:52:02.349610 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:02.349586    6996 generic.go:356] "Generic (PLEG): container finished" podID="3eafcd7e-b8d7-4f3b-a483-c2c7dc9b764a" containerID="040a37c9c0d37e4b3e663de4afce594e700f256d82adb5605a52744a552bd1a9" exitCode=0
Oct 30 19:52:02.349741 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:02.349639    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-f61e4b-66959bc884-hmnz9"
Oct 30 19:52:02.349741 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:02.349670    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-f61e4b-66959bc884-hmnz9" event={"ID":"3eafcd7e-b8d7-4f3b-a483-c2c7dc9b764a","Type":"ContainerDied","Data":"040a37c9c0d37e4b3e663de4afce594e700f256d82adb5605a52744a552bd1a9"}
Oct 30 19:52:02.349741 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:02.349727    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-f61e4b-66959bc884-hmnz9" event={"ID":"3eafcd7e-b8d7-4f3b-a483-c2c7dc9b764a","Type":"ContainerDied","Data":"6ecfee7b4e27669ffd74a9f1b227ce693ee1cd6104434cd79ace08009198afae"}
Oct 30 19:52:02.359615 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:02.359594    6996 scope.go:122] "RemoveContainer" containerID="dff798a149b1dc5b0f44cdda21d56089b87d4fe291ec7eb1bb53808bb57b04d7"
Oct 30 19:52:02.359894 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:52:02.359864    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dff798a149b1dc5b0f44cdda21d56089b87d4fe291ec7eb1bb53808bb57b04d7\": container with ID starting with dff798a149b1dc5b0f44cdda21d56089b87d4fe291ec7eb1bb53808bb57b04d7 not found: ID does not exist" containerID="dff798a149b1dc5b0f44cdda21d56089b87d4fe291ec7eb1bb53808bb57b04d7"
Oct 30 19:52:02.359961 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:02.359904    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dff798a149b1dc5b0f44cdda21d56089b87d4fe291ec7eb1bb53808bb57b04d7"} err="failed to get container status \"dff798a149b1dc5b0f44cdda21d56089b87d4fe291ec7eb1bb53808bb57b04d7\": rpc error: code = NotFound desc = could not find container \"dff798a149b1dc5b0f44cdda21d56089b87d4fe291ec7eb1bb53808bb57b04d7\": container with ID starting with dff798a149b1dc5b0f44cdda21d56089b87d4fe291ec7eb1bb53808bb57b04d7 not found: ID does not exist"
Oct 30 19:52:02.359961 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:02.359928    6996 scope.go:122] "RemoveContainer" containerID="040a37c9c0d37e4b3e663de4afce594e700f256d82adb5605a52744a552bd1a9"
Oct 30 19:52:02.370647 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:02.370630    6996 scope.go:122] "RemoveContainer" containerID="040a37c9c0d37e4b3e663de4afce594e700f256d82adb5605a52744a552bd1a9"
Oct 30 19:52:02.370905 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:52:02.370883    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"040a37c9c0d37e4b3e663de4afce594e700f256d82adb5605a52744a552bd1a9\": container with ID starting with 040a37c9c0d37e4b3e663de4afce594e700f256d82adb5605a52744a552bd1a9 not found: ID does not exist" containerID="040a37c9c0d37e4b3e663de4afce594e700f256d82adb5605a52744a552bd1a9"
Oct 30 19:52:02.370968 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:02.370914    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"040a37c9c0d37e4b3e663de4afce594e700f256d82adb5605a52744a552bd1a9"} err="failed to get container status \"040a37c9c0d37e4b3e663de4afce594e700f256d82adb5605a52744a552bd1a9\": rpc error: code = NotFound desc = could not find container \"040a37c9c0d37e4b3e663de4afce594e700f256d82adb5605a52744a552bd1a9\": container with ID starting with 040a37c9c0d37e4b3e663de4afce594e700f256d82adb5605a52744a552bd1a9 not found: ID does not exist"
Oct 30 19:52:02.387129 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:02.387101    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-f61e4b-66959bc884-hmnz9"]
Oct 30 19:52:02.394974 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:02.394945    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-f61e4b-66959bc884-hmnz9"]
Oct 30 19:52:02.410645 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:02.410617    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-f61e4b-7c54c4f9f5-gv9x9"]
Oct 30 19:52:02.417978 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:02.417953    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-f61e4b-7c54c4f9f5-gv9x9"]
Oct 30 19:52:02.783557 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:02.783047    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-2beb7b-59c9b6c955-qjwcz"]
Oct 30 19:52:02.783557 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:02.783372    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2beb7b-59c9b6c955-qjwcz" podUID="e5988db4-cc38-4cc9-ac41-cc3abbf865bb" containerName="maas-api" containerID="cri-o://9aea1f57d8f5bed49197e1c037ed5a36b3316df2d200ed0e940e25c476410c8a" gracePeriod=30
Oct 30 19:52:02.801088 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:02.801057    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3eafcd7e-b8d7-4f3b-a483-c2c7dc9b764a" path="/var/lib/kubelet/pods/3eafcd7e-b8d7-4f3b-a483-c2c7dc9b764a/volumes"
Oct 30 19:52:02.801709 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:02.801686    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7e524e27-0d0d-42db-b376-0c488d6a0622" path="/var/lib/kubelet/pods/7e524e27-0d0d-42db-b376-0c488d6a0622/volumes"
Oct 30 19:52:03.191505 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:03.190336    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2beb7b-59c9b6c955-qjwcz"
Oct 30 19:52:03.234187 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:03.234148    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e5988db4-cc38-4cc9-ac41-cc3abbf865bb-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5988db4-cc38-4cc9-ac41-cc3abbf865bb-openshift-service-ca\") pod \"e5988db4-cc38-4cc9-ac41-cc3abbf865bb\" (UID: \"e5988db4-cc38-4cc9-ac41-cc3abbf865bb\") "
Oct 30 19:52:03.234396 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:03.234266    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5988db4-cc38-4cc9-ac41-cc3abbf865bb-openshift-service-ca" pod "e5988db4-cc38-4cc9-ac41-cc3abbf865bb" (UID: "e5988db4-cc38-4cc9-ac41-cc3abbf865bb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:52:03.234771 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:03.234735    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e5988db4-cc38-4cc9-ac41-cc3abbf865bb-kube-api-access-2bvg2\" (UniqueName: \"kubernetes.io/projected/e5988db4-cc38-4cc9-ac41-cc3abbf865bb-kube-api-access-2bvg2\") pod \"e5988db4-cc38-4cc9-ac41-cc3abbf865bb\" (UID: \"e5988db4-cc38-4cc9-ac41-cc3abbf865bb\") "
Oct 30 19:52:03.234925 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:03.234896    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e5988db4-cc38-4cc9-ac41-cc3abbf865bb-metrics-certs\" (UniqueName: \"kubernetes.io/secret/e5988db4-cc38-4cc9-ac41-cc3abbf865bb-metrics-certs\") pod \"e5988db4-cc38-4cc9-ac41-cc3abbf865bb\" (UID: \"e5988db4-cc38-4cc9-ac41-cc3abbf865bb\") "
Oct 30 19:52:03.235016 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:03.234963    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e5988db4-cc38-4cc9-ac41-cc3abbf865bb-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5988db4-cc38-4cc9-ac41-cc3abbf865bb-maas-api-tls\") pod \"e5988db4-cc38-4cc9-ac41-cc3abbf865bb\" (UID: \"e5988db4-cc38-4cc9-ac41-cc3abbf865bb\") "
Oct 30 19:52:03.235946 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:03.235921    6996 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5988db4-cc38-4cc9-ac41-cc3abbf865bb-openshift-service-ca\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:52:03.237427 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:03.237403    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5988db4-cc38-4cc9-ac41-cc3abbf865bb-maas-api-tls" pod "e5988db4-cc38-4cc9-ac41-cc3abbf865bb" (UID: "e5988db4-cc38-4cc9-ac41-cc3abbf865bb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:52:03.239222 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:03.239201    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5988db4-cc38-4cc9-ac41-cc3abbf865bb-metrics-certs" pod "e5988db4-cc38-4cc9-ac41-cc3abbf865bb" (UID: "e5988db4-cc38-4cc9-ac41-cc3abbf865bb"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:52:03.240209 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:03.240192    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5988db4-cc38-4cc9-ac41-cc3abbf865bb-kube-api-access-2bvg2" pod "e5988db4-cc38-4cc9-ac41-cc3abbf865bb" (UID: "e5988db4-cc38-4cc9-ac41-cc3abbf865bb"). InnerVolumeSpecName "kube-api-access-2bvg2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:52:03.337379 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:03.337344    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2bvg2\" (UniqueName: \"kubernetes.io/projected/e5988db4-cc38-4cc9-ac41-cc3abbf865bb-kube-api-access-2bvg2\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:52:03.337379 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:03.337375    6996 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e5988db4-cc38-4cc9-ac41-cc3abbf865bb-metrics-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:52:03.337379 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:03.337385    6996 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5988db4-cc38-4cc9-ac41-cc3abbf865bb-maas-api-tls\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:52:03.368198 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:03.368100    6996 generic.go:356] "Generic (PLEG): container finished" podID="e5988db4-cc38-4cc9-ac41-cc3abbf865bb" containerID="9aea1f57d8f5bed49197e1c037ed5a36b3316df2d200ed0e940e25c476410c8a" exitCode=0
Oct 30 19:52:03.368198 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:03.368158    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2beb7b-59c9b6c955-qjwcz"
Oct 30 19:52:03.368198 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:03.368179    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2beb7b-59c9b6c955-qjwcz" event={"ID":"e5988db4-cc38-4cc9-ac41-cc3abbf865bb","Type":"ContainerDied","Data":"9aea1f57d8f5bed49197e1c037ed5a36b3316df2d200ed0e940e25c476410c8a"}
Oct 30 19:52:03.368463 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:03.368274    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2beb7b-59c9b6c955-qjwcz" event={"ID":"e5988db4-cc38-4cc9-ac41-cc3abbf865bb","Type":"ContainerDied","Data":"a02ae05329f9f4bf2fd37bbd94c0bca20e0bd200677cef06112c85c417a7db14"}
Oct 30 19:52:03.368463 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:03.368304    6996 scope.go:122] "RemoveContainer" containerID="9aea1f57d8f5bed49197e1c037ed5a36b3316df2d200ed0e940e25c476410c8a"
Oct 30 19:52:03.378956 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:03.378939    6996 scope.go:122] "RemoveContainer" containerID="9aea1f57d8f5bed49197e1c037ed5a36b3316df2d200ed0e940e25c476410c8a"
Oct 30 19:52:03.379207 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:52:03.379187    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9aea1f57d8f5bed49197e1c037ed5a36b3316df2d200ed0e940e25c476410c8a\": container with ID starting with 9aea1f57d8f5bed49197e1c037ed5a36b3316df2d200ed0e940e25c476410c8a not found: ID does not exist" containerID="9aea1f57d8f5bed49197e1c037ed5a36b3316df2d200ed0e940e25c476410c8a"
Oct 30 19:52:03.379262 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:03.379215    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9aea1f57d8f5bed49197e1c037ed5a36b3316df2d200ed0e940e25c476410c8a"} err="failed to get container status \"9aea1f57d8f5bed49197e1c037ed5a36b3316df2d200ed0e940e25c476410c8a\": rpc error: code = NotFound desc = could not find container \"9aea1f57d8f5bed49197e1c037ed5a36b3316df2d200ed0e940e25c476410c8a\": container with ID starting with 9aea1f57d8f5bed49197e1c037ed5a36b3316df2d200ed0e940e25c476410c8a not found: ID does not exist"
Oct 30 19:52:03.407181 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:03.407152    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-2beb7b-59c9b6c955-qjwcz"]
Oct 30 19:52:03.416316 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:03.416289    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-2beb7b-59c9b6c955-qjwcz"]
Oct 30 19:52:04.793458 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:04.793427    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e5988db4-cc38-4cc9-ac41-cc3abbf865bb" path="/var/lib/kubelet/pods/e5988db4-cc38-4cc9-ac41-cc3abbf865bb/volumes"
Oct 30 19:52:10.201560 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:10.201527    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5f78687d5b-dkcrs_c72164f7-cc46-4f11-b1f0-f50adf9c4c0b/payload-processing/0.log"
Oct 30 19:52:10.409068 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:10.409039    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5f78687d5b-dkcrs_c72164f7-cc46-4f11-b1f0-f50adf9c4c0b/payload-processing/0.log"
Oct 30 19:52:12.473257 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:12.473217    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk"]
Oct 30 19:52:12.473874 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:12.473517    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk" podUID="d9ce7283-f1bf-4afe-97dd-ad5f5ce14786" containerName="istio-proxy" containerID="cri-o://04907e862cfdc39fae50e894edabd276e2b328e1b3d6ec1ec52ac16d19f1f47a" gracePeriod=30
Oct 30 19:52:17.770070 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:17.770038    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk"
Oct 30 19:52:17.833523 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:17.833165    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d9ce7283-f1bf-4afe-97dd-ad5f5ce14786-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9ce7283-f1bf-4afe-97dd-ad5f5ce14786-istio-podinfo\") pod \"d9ce7283-f1bf-4afe-97dd-ad5f5ce14786\" (UID: \"d9ce7283-f1bf-4afe-97dd-ad5f5ce14786\") "
Oct 30 19:52:17.833523 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:17.833229    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d9ce7283-f1bf-4afe-97dd-ad5f5ce14786-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9ce7283-f1bf-4afe-97dd-ad5f5ce14786-workload-certs\") pod \"d9ce7283-f1bf-4afe-97dd-ad5f5ce14786\" (UID: \"d9ce7283-f1bf-4afe-97dd-ad5f5ce14786\") "
Oct 30 19:52:17.833523 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:17.833384    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d9ce7283-f1bf-4afe-97dd-ad5f5ce14786-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9ce7283-f1bf-4afe-97dd-ad5f5ce14786-istio-envoy\") pod \"d9ce7283-f1bf-4afe-97dd-ad5f5ce14786\" (UID: \"d9ce7283-f1bf-4afe-97dd-ad5f5ce14786\") "
Oct 30 19:52:17.833523 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:17.833458    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d9ce7283-f1bf-4afe-97dd-ad5f5ce14786-istio-token\" (UniqueName: \"kubernetes.io/projected/d9ce7283-f1bf-4afe-97dd-ad5f5ce14786-istio-token\") pod \"d9ce7283-f1bf-4afe-97dd-ad5f5ce14786\" (UID: \"d9ce7283-f1bf-4afe-97dd-ad5f5ce14786\") "
Oct 30 19:52:17.833523 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:17.833518    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d9ce7283-f1bf-4afe-97dd-ad5f5ce14786-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9ce7283-f1bf-4afe-97dd-ad5f5ce14786-istio-data\") pod \"d9ce7283-f1bf-4afe-97dd-ad5f5ce14786\" (UID: \"d9ce7283-f1bf-4afe-97dd-ad5f5ce14786\") "
Oct 30 19:52:17.834008 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:17.833609    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d9ce7283-f1bf-4afe-97dd-ad5f5ce14786-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9ce7283-f1bf-4afe-97dd-ad5f5ce14786-workload-socket\") pod \"d9ce7283-f1bf-4afe-97dd-ad5f5ce14786\" (UID: \"d9ce7283-f1bf-4afe-97dd-ad5f5ce14786\") "
Oct 30 19:52:17.834008 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:17.833713    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d9ce7283-f1bf-4afe-97dd-ad5f5ce14786-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9ce7283-f1bf-4afe-97dd-ad5f5ce14786-istiod-ca-cert\") pod \"d9ce7283-f1bf-4afe-97dd-ad5f5ce14786\" (UID: \"d9ce7283-f1bf-4afe-97dd-ad5f5ce14786\") "
Oct 30 19:52:17.834008 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:17.833767    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d9ce7283-f1bf-4afe-97dd-ad5f5ce14786-kube-api-access-w76wx\" (UniqueName: \"kubernetes.io/projected/d9ce7283-f1bf-4afe-97dd-ad5f5ce14786-kube-api-access-w76wx\") pod \"d9ce7283-f1bf-4afe-97dd-ad5f5ce14786\" (UID: \"d9ce7283-f1bf-4afe-97dd-ad5f5ce14786\") "
Oct 30 19:52:17.834008 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:17.833860    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d9ce7283-f1bf-4afe-97dd-ad5f5ce14786-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9ce7283-f1bf-4afe-97dd-ad5f5ce14786-credential-socket\") pod \"d9ce7283-f1bf-4afe-97dd-ad5f5ce14786\" (UID: \"d9ce7283-f1bf-4afe-97dd-ad5f5ce14786\") "
Oct 30 19:52:17.834581 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:17.834553    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9ce7283-f1bf-4afe-97dd-ad5f5ce14786-workload-socket" pod "d9ce7283-f1bf-4afe-97dd-ad5f5ce14786" (UID: "d9ce7283-f1bf-4afe-97dd-ad5f5ce14786"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:52:17.835015 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:17.834764    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9ce7283-f1bf-4afe-97dd-ad5f5ce14786-credential-socket" pod "d9ce7283-f1bf-4afe-97dd-ad5f5ce14786" (UID: "d9ce7283-f1bf-4afe-97dd-ad5f5ce14786"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:52:17.835132 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:17.835053    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9ce7283-f1bf-4afe-97dd-ad5f5ce14786-workload-certs" pod "d9ce7283-f1bf-4afe-97dd-ad5f5ce14786" (UID: "d9ce7283-f1bf-4afe-97dd-ad5f5ce14786"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:52:17.835374 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:17.835253    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9ce7283-f1bf-4afe-97dd-ad5f5ce14786-istio-data" pod "d9ce7283-f1bf-4afe-97dd-ad5f5ce14786" (UID: "d9ce7283-f1bf-4afe-97dd-ad5f5ce14786"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:52:17.835374 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:17.835342    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9ce7283-f1bf-4afe-97dd-ad5f5ce14786-istiod-ca-cert" pod "d9ce7283-f1bf-4afe-97dd-ad5f5ce14786" (UID: "d9ce7283-f1bf-4afe-97dd-ad5f5ce14786"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:52:17.837384 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:17.837308    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9ce7283-f1bf-4afe-97dd-ad5f5ce14786-istio-envoy" pod "d9ce7283-f1bf-4afe-97dd-ad5f5ce14786" (UID: "d9ce7283-f1bf-4afe-97dd-ad5f5ce14786"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:52:17.838945 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:17.838800    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9ce7283-f1bf-4afe-97dd-ad5f5ce14786-istio-token" pod "d9ce7283-f1bf-4afe-97dd-ad5f5ce14786" (UID: "d9ce7283-f1bf-4afe-97dd-ad5f5ce14786"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:52:17.839055 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:17.839037    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9ce7283-f1bf-4afe-97dd-ad5f5ce14786-kube-api-access-w76wx" pod "d9ce7283-f1bf-4afe-97dd-ad5f5ce14786" (UID: "d9ce7283-f1bf-4afe-97dd-ad5f5ce14786"). InnerVolumeSpecName "kube-api-access-w76wx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:52:17.839444 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:17.839415    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d9ce7283-f1bf-4afe-97dd-ad5f5ce14786-istio-podinfo" pod "d9ce7283-f1bf-4afe-97dd-ad5f5ce14786" (UID: "d9ce7283-f1bf-4afe-97dd-ad5f5ce14786"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:52:17.935895 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:17.935863    6996 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9ce7283-f1bf-4afe-97dd-ad5f5ce14786-istio-envoy\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:52:17.935895 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:17.935889    6996 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9ce7283-f1bf-4afe-97dd-ad5f5ce14786-istio-token\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:52:17.935895 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:17.935898    6996 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9ce7283-f1bf-4afe-97dd-ad5f5ce14786-istio-data\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:52:17.935895 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:17.935907    6996 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9ce7283-f1bf-4afe-97dd-ad5f5ce14786-workload-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:52:17.936188 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:17.935917    6996 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9ce7283-f1bf-4afe-97dd-ad5f5ce14786-istiod-ca-cert\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:52:17.936188 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:17.935927    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w76wx\" (UniqueName: \"kubernetes.io/projected/d9ce7283-f1bf-4afe-97dd-ad5f5ce14786-kube-api-access-w76wx\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:52:17.936188 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:17.935935    6996 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9ce7283-f1bf-4afe-97dd-ad5f5ce14786-credential-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:52:17.936188 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:17.935945    6996 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9ce7283-f1bf-4afe-97dd-ad5f5ce14786-istio-podinfo\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:52:17.936188 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:17.935954    6996 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9ce7283-f1bf-4afe-97dd-ad5f5ce14786-workload-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:52:18.533940 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:18.533902    6996 generic.go:356] "Generic (PLEG): container finished" podID="d9ce7283-f1bf-4afe-97dd-ad5f5ce14786" containerID="04907e862cfdc39fae50e894edabd276e2b328e1b3d6ec1ec52ac16d19f1f47a" exitCode=0
Oct 30 19:52:18.534112 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:18.533953    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk" event={"ID":"d9ce7283-f1bf-4afe-97dd-ad5f5ce14786","Type":"ContainerDied","Data":"04907e862cfdc39fae50e894edabd276e2b328e1b3d6ec1ec52ac16d19f1f47a"}
Oct 30 19:52:18.534112 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:18.533956    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk"
Oct 30 19:52:18.534112 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:18.533996    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk" event={"ID":"d9ce7283-f1bf-4afe-97dd-ad5f5ce14786","Type":"ContainerDied","Data":"f57f34124057e4afc50f7226930088784ada4b5984521a3ed8947013f783d3e0"}
Oct 30 19:52:18.534112 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:18.534019    6996 scope.go:122] "RemoveContainer" containerID="04907e862cfdc39fae50e894edabd276e2b328e1b3d6ec1ec52ac16d19f1f47a"
Oct 30 19:52:18.545916 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:18.545898    6996 scope.go:122] "RemoveContainer" containerID="c1399355ad0d85b6f237444d6ddce78a326cf9c5da2d7bb70520f69f038fe8ee"
Oct 30 19:52:18.559504 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:18.559466    6996 scope.go:122] "RemoveContainer" containerID="04907e862cfdc39fae50e894edabd276e2b328e1b3d6ec1ec52ac16d19f1f47a"
Oct 30 19:52:18.559806 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:52:18.559789    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04907e862cfdc39fae50e894edabd276e2b328e1b3d6ec1ec52ac16d19f1f47a\": container with ID starting with 04907e862cfdc39fae50e894edabd276e2b328e1b3d6ec1ec52ac16d19f1f47a not found: ID does not exist" containerID="04907e862cfdc39fae50e894edabd276e2b328e1b3d6ec1ec52ac16d19f1f47a"
Oct 30 19:52:18.559860 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:18.559814    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04907e862cfdc39fae50e894edabd276e2b328e1b3d6ec1ec52ac16d19f1f47a"} err="failed to get container status \"04907e862cfdc39fae50e894edabd276e2b328e1b3d6ec1ec52ac16d19f1f47a\": rpc error: code = NotFound desc = could not find container \"04907e862cfdc39fae50e894edabd276e2b328e1b3d6ec1ec52ac16d19f1f47a\": container with ID starting with 04907e862cfdc39fae50e894edabd276e2b328e1b3d6ec1ec52ac16d19f1f47a not found: ID does not exist"
Oct 30 19:52:18.559860 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:18.559831    6996 scope.go:122] "RemoveContainer" containerID="c1399355ad0d85b6f237444d6ddce78a326cf9c5da2d7bb70520f69f038fe8ee"
Oct 30 19:52:18.560084 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:52:18.560062    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1399355ad0d85b6f237444d6ddce78a326cf9c5da2d7bb70520f69f038fe8ee\": container with ID starting with c1399355ad0d85b6f237444d6ddce78a326cf9c5da2d7bb70520f69f038fe8ee not found: ID does not exist" containerID="c1399355ad0d85b6f237444d6ddce78a326cf9c5da2d7bb70520f69f038fe8ee"
Oct 30 19:52:18.560169 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:18.560088    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1399355ad0d85b6f237444d6ddce78a326cf9c5da2d7bb70520f69f038fe8ee"} err="failed to get container status \"c1399355ad0d85b6f237444d6ddce78a326cf9c5da2d7bb70520f69f038fe8ee\": rpc error: code = NotFound desc = could not find container \"c1399355ad0d85b6f237444d6ddce78a326cf9c5da2d7bb70520f69f038fe8ee\": container with ID starting with c1399355ad0d85b6f237444d6ddce78a326cf9c5da2d7bb70520f69f038fe8ee not found: ID does not exist"
Oct 30 19:52:18.562503 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:18.562461    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk"]
Oct 30 19:52:18.566620 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:18.566597    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-2beb7b-openshift-default-9df6554f9-hknkk"]
Oct 30 19:52:18.793332 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:18.793253    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d9ce7283-f1bf-4afe-97dd-ad5f5ce14786" path="/var/lib/kubelet/pods/d9ce7283-f1bf-4afe-97dd-ad5f5ce14786/volumes"
Oct 30 19:52:23.126672 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:23.126627    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 19:52:24.640195 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:24.640157    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f61e4b-57cccbd8f-j8kw2"]
Oct 30 19:52:24.640699 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:24.640535    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f61e4b-57cccbd8f-j8kw2" podUID="c19eefde-3ee9-4c3f-b551-cdc4d30a96c2" containerName="maas-api" containerID="cri-o://3efe0093a179aef260833884a15e9ff7dfd8b790b45755fb86a3a47551ccc866" gracePeriod=30
Oct 30 19:52:25.024545 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:25.019601    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f61e4b-57cccbd8f-j8kw2"
Oct 30 19:52:25.061024 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:25.060986    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c19eefde-3ee9-4c3f-b551-cdc4d30a96c2-kube-api-access-k95f6\" (UniqueName: \"kubernetes.io/projected/c19eefde-3ee9-4c3f-b551-cdc4d30a96c2-kube-api-access-k95f6\") pod \"c19eefde-3ee9-4c3f-b551-cdc4d30a96c2\" (UID: \"c19eefde-3ee9-4c3f-b551-cdc4d30a96c2\") "
Oct 30 19:52:25.061219 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:25.061098    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c19eefde-3ee9-4c3f-b551-cdc4d30a96c2-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c19eefde-3ee9-4c3f-b551-cdc4d30a96c2-maas-api-tls\") pod \"c19eefde-3ee9-4c3f-b551-cdc4d30a96c2\" (UID: \"c19eefde-3ee9-4c3f-b551-cdc4d30a96c2\") "
Oct 30 19:52:25.061219 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:25.061210    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c19eefde-3ee9-4c3f-b551-cdc4d30a96c2-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c19eefde-3ee9-4c3f-b551-cdc4d30a96c2-openshift-service-ca\") pod \"c19eefde-3ee9-4c3f-b551-cdc4d30a96c2\" (UID: \"c19eefde-3ee9-4c3f-b551-cdc4d30a96c2\") "
Oct 30 19:52:25.061457 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:25.061426    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c19eefde-3ee9-4c3f-b551-cdc4d30a96c2-metrics-certs\" (UniqueName: \"kubernetes.io/secret/c19eefde-3ee9-4c3f-b551-cdc4d30a96c2-metrics-certs\") pod \"c19eefde-3ee9-4c3f-b551-cdc4d30a96c2\" (UID: \"c19eefde-3ee9-4c3f-b551-cdc4d30a96c2\") "
Oct 30 19:52:25.061676 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:25.061646    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c19eefde-3ee9-4c3f-b551-cdc4d30a96c2-openshift-service-ca" pod "c19eefde-3ee9-4c3f-b551-cdc4d30a96c2" (UID: "c19eefde-3ee9-4c3f-b551-cdc4d30a96c2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:52:25.062222 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:25.062198    6996 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c19eefde-3ee9-4c3f-b551-cdc4d30a96c2-openshift-service-ca\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:52:25.063618 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:25.063587    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c19eefde-3ee9-4c3f-b551-cdc4d30a96c2-kube-api-access-k95f6" pod "c19eefde-3ee9-4c3f-b551-cdc4d30a96c2" (UID: "c19eefde-3ee9-4c3f-b551-cdc4d30a96c2"). InnerVolumeSpecName "kube-api-access-k95f6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:52:25.063776 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:25.063708    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c19eefde-3ee9-4c3f-b551-cdc4d30a96c2-metrics-certs" pod "c19eefde-3ee9-4c3f-b551-cdc4d30a96c2" (UID: "c19eefde-3ee9-4c3f-b551-cdc4d30a96c2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:52:25.064935 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:25.064909    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c19eefde-3ee9-4c3f-b551-cdc4d30a96c2-maas-api-tls" pod "c19eefde-3ee9-4c3f-b551-cdc4d30a96c2" (UID: "c19eefde-3ee9-4c3f-b551-cdc4d30a96c2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:52:25.163996 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:25.163960    6996 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c19eefde-3ee9-4c3f-b551-cdc4d30a96c2-metrics-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:52:25.164184 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:25.163997    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k95f6\" (UniqueName: \"kubernetes.io/projected/c19eefde-3ee9-4c3f-b551-cdc4d30a96c2-kube-api-access-k95f6\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:52:25.164184 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:25.164030    6996 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c19eefde-3ee9-4c3f-b551-cdc4d30a96c2-maas-api-tls\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:52:25.584627 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:25.584590    6996 generic.go:356] "Generic (PLEG): container finished" podID="c19eefde-3ee9-4c3f-b551-cdc4d30a96c2" containerID="3efe0093a179aef260833884a15e9ff7dfd8b790b45755fb86a3a47551ccc866" exitCode=0
Oct 30 19:52:25.584800 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:25.584632    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f61e4b-57cccbd8f-j8kw2"
Oct 30 19:52:25.584800 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:25.584671    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f61e4b-57cccbd8f-j8kw2" event={"ID":"c19eefde-3ee9-4c3f-b551-cdc4d30a96c2","Type":"ContainerDied","Data":"3efe0093a179aef260833884a15e9ff7dfd8b790b45755fb86a3a47551ccc866"}
Oct 30 19:52:25.584800 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:25.584725    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f61e4b-57cccbd8f-j8kw2" event={"ID":"c19eefde-3ee9-4c3f-b551-cdc4d30a96c2","Type":"ContainerDied","Data":"ca6c97a04a6d94722e1c881f39c1906e31d29201508b1e1c7c7302ba26719ab3"}
Oct 30 19:52:25.584800 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:25.584752    6996 scope.go:122] "RemoveContainer" containerID="3efe0093a179aef260833884a15e9ff7dfd8b790b45755fb86a3a47551ccc866"
Oct 30 19:52:25.595758 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:25.595740    6996 scope.go:122] "RemoveContainer" containerID="3efe0093a179aef260833884a15e9ff7dfd8b790b45755fb86a3a47551ccc866"
Oct 30 19:52:25.596045 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:52:25.596029    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3efe0093a179aef260833884a15e9ff7dfd8b790b45755fb86a3a47551ccc866\": container with ID starting with 3efe0093a179aef260833884a15e9ff7dfd8b790b45755fb86a3a47551ccc866 not found: ID does not exist" containerID="3efe0093a179aef260833884a15e9ff7dfd8b790b45755fb86a3a47551ccc866"
Oct 30 19:52:25.596127 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:25.596054    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3efe0093a179aef260833884a15e9ff7dfd8b790b45755fb86a3a47551ccc866"} err="failed to get container status \"3efe0093a179aef260833884a15e9ff7dfd8b790b45755fb86a3a47551ccc866\": rpc error: code = NotFound desc = could not find container \"3efe0093a179aef260833884a15e9ff7dfd8b790b45755fb86a3a47551ccc866\": container with ID starting with 3efe0093a179aef260833884a15e9ff7dfd8b790b45755fb86a3a47551ccc866 not found: ID does not exist"
Oct 30 19:52:25.613892 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:25.613867    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f61e4b-57cccbd8f-j8kw2"]
Oct 30 19:52:25.629815 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:25.629792    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f61e4b-57cccbd8f-j8kw2"]
Oct 30 19:52:26.793184 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:26.793152    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c19eefde-3ee9-4c3f-b551-cdc4d30a96c2" path="/var/lib/kubelet/pods/c19eefde-3ee9-4c3f-b551-cdc4d30a96c2/volumes"
Oct 30 19:52:28.119075 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:28.119042    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-f03120-7b689f64f9-vj88z"]
Oct 30 19:52:28.119761 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:28.119731    6996 cpu_manager.go:414] "containerMap: removing container" podUID="7e524e27-0d0d-42db-b376-0c488d6a0622" containerName="payload-pre-processing"
Oct 30 19:52:28.119761 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:28.119762    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7e524e27-0d0d-42db-b376-0c488d6a0622" containerName="payload-pre-processing"
Oct 30 19:52:28.119883 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:28.119795    6996 cpu_manager.go:414] "containerMap: removing container" podUID="d9ce7283-f1bf-4afe-97dd-ad5f5ce14786" containerName="istio-proxy"
Oct 30 19:52:28.119883 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:28.119801    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d9ce7283-f1bf-4afe-97dd-ad5f5ce14786" containerName="istio-proxy"
Oct 30 19:52:28.119883 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:28.119816    6996 cpu_manager.go:414] "containerMap: removing container" podUID="c19eefde-3ee9-4c3f-b551-cdc4d30a96c2" containerName="maas-api"
Oct 30 19:52:28.119883 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:28.119822    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c19eefde-3ee9-4c3f-b551-cdc4d30a96c2" containerName="maas-api"
Oct 30 19:52:28.119883 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:28.119829    6996 cpu_manager.go:414] "containerMap: removing container" podUID="3eafcd7e-b8d7-4f3b-a483-c2c7dc9b764a" containerName="payload-processing"
Oct 30 19:52:28.119883 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:28.119835    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3eafcd7e-b8d7-4f3b-a483-c2c7dc9b764a" containerName="payload-processing"
Oct 30 19:52:28.119883 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:28.119863    6996 cpu_manager.go:414] "containerMap: removing container" podUID="e5988db4-cc38-4cc9-ac41-cc3abbf865bb" containerName="maas-api"
Oct 30 19:52:28.119883 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:28.119868    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e5988db4-cc38-4cc9-ac41-cc3abbf865bb" containerName="maas-api"
Oct 30 19:52:28.119883 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:28.119875    6996 cpu_manager.go:414] "containerMap: removing container" podUID="d9ce7283-f1bf-4afe-97dd-ad5f5ce14786" containerName="istio-proxy"
Oct 30 19:52:28.119883 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:28.119880    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d9ce7283-f1bf-4afe-97dd-ad5f5ce14786" containerName="istio-proxy"
Oct 30 19:52:28.120206 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:28.120001    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="e5988db4-cc38-4cc9-ac41-cc3abbf865bb" containerName="maas-api"
Oct 30 19:52:28.120206 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:28.120010    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="d9ce7283-f1bf-4afe-97dd-ad5f5ce14786" containerName="istio-proxy"
Oct 30 19:52:28.120206 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:28.120022    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="7e524e27-0d0d-42db-b376-0c488d6a0622" containerName="payload-pre-processing"
Oct 30 19:52:28.120206 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:28.120031    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="d9ce7283-f1bf-4afe-97dd-ad5f5ce14786" containerName="istio-proxy"
Oct 30 19:52:28.120206 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:28.120042    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="c19eefde-3ee9-4c3f-b551-cdc4d30a96c2" containerName="maas-api"
Oct 30 19:52:28.120206 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:28.120048    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="3eafcd7e-b8d7-4f3b-a483-c2c7dc9b764a" containerName="payload-processing"
Oct 30 19:52:28.124872 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:28.124854    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-f03120-7b689f64f9-vj88z"
Oct 30 19:52:28.134018 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:28.133993    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-praxis-b-f03120\""
Oct 30 19:52:28.141460 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:28.141431    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-f03120-7b689f64f9-vj88z"]
Oct 30 19:52:28.166850 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:28.166820    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-f03120-79bf49bdd6-z9zzs"]
Oct 30 19:52:28.181818 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:28.181784    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-b-f03120-79bf49bdd6-z9zzs"
Oct 30 19:52:28.191767 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:28.191737    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-praxis-b-f03120-dockercfg-kl4st\""
Oct 30 19:52:28.198466 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:28.198438    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-f03120-79bf49bdd6-z9zzs"]
Oct 30 19:52:28.202920 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:28.202889    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzb7t\" (UniqueName: \"kubernetes.io/projected/2691319f-ce3f-4930-b1b6-2e35f8f55a64-kube-api-access-jzb7t\") pod \"payload-pre-processing-e2e-praxis-b-f03120-7b689f64f9-vj88z\" (UID: \"2691319f-ce3f-4930-b1b6-2e35f8f55a64\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-f03120-7b689f64f9-vj88z"
Oct 30 19:52:28.203130 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:28.203107    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2691319f-ce3f-4930-b1b6-2e35f8f55a64-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-b-f03120-7b689f64f9-vj88z\" (UID: \"2691319f-ce3f-4930-b1b6-2e35f8f55a64\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-f03120-7b689f64f9-vj88z"
Oct 30 19:52:28.330561 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:28.329801    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzb7t\" (UniqueName: \"kubernetes.io/projected/2691319f-ce3f-4930-b1b6-2e35f8f55a64-kube-api-access-jzb7t\") pod \"payload-pre-processing-e2e-praxis-b-f03120-7b689f64f9-vj88z\" (UID: \"2691319f-ce3f-4930-b1b6-2e35f8f55a64\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-f03120-7b689f64f9-vj88z"
Oct 30 19:52:28.330561 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:28.329862    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eaa59c33-59f0-43c2-b8ec-68d753622ab9-plugins-config-volume\") pod \"payload-processing-e2e-praxis-b-f03120-79bf49bdd6-z9zzs\" (UID: \"eaa59c33-59f0-43c2-b8ec-68d753622ab9\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-f03120-79bf49bdd6-z9zzs"
Oct 30 19:52:28.330561 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:28.329888    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdlrw\" (UniqueName: \"kubernetes.io/projected/eaa59c33-59f0-43c2-b8ec-68d753622ab9-kube-api-access-xdlrw\") pod \"payload-processing-e2e-praxis-b-f03120-79bf49bdd6-z9zzs\" (UID: \"eaa59c33-59f0-43c2-b8ec-68d753622ab9\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-f03120-79bf49bdd6-z9zzs"
Oct 30 19:52:28.330561 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:28.330015    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2691319f-ce3f-4930-b1b6-2e35f8f55a64-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-b-f03120-7b689f64f9-vj88z\" (UID: \"2691319f-ce3f-4930-b1b6-2e35f8f55a64\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-f03120-7b689f64f9-vj88z"
Oct 30 19:52:28.351602 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:28.330987    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2691319f-ce3f-4930-b1b6-2e35f8f55a64-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-b-f03120-7b689f64f9-vj88z\" (UID: \"2691319f-ce3f-4930-b1b6-2e35f8f55a64\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-f03120-7b689f64f9-vj88z"
Oct 30 19:52:28.391228 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:28.388021    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzb7t\" (UniqueName: \"kubernetes.io/projected/2691319f-ce3f-4930-b1b6-2e35f8f55a64-kube-api-access-jzb7t\") pod \"payload-pre-processing-e2e-praxis-b-f03120-7b689f64f9-vj88z\" (UID: \"2691319f-ce3f-4930-b1b6-2e35f8f55a64\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-f03120-7b689f64f9-vj88z"
Oct 30 19:52:28.431267 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:28.431221    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eaa59c33-59f0-43c2-b8ec-68d753622ab9-plugins-config-volume\") pod \"payload-processing-e2e-praxis-b-f03120-79bf49bdd6-z9zzs\" (UID: \"eaa59c33-59f0-43c2-b8ec-68d753622ab9\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-f03120-79bf49bdd6-z9zzs"
Oct 30 19:52:28.431267 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:28.431270    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdlrw\" (UniqueName: \"kubernetes.io/projected/eaa59c33-59f0-43c2-b8ec-68d753622ab9-kube-api-access-xdlrw\") pod \"payload-processing-e2e-praxis-b-f03120-79bf49bdd6-z9zzs\" (UID: \"eaa59c33-59f0-43c2-b8ec-68d753622ab9\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-f03120-79bf49bdd6-z9zzs"
Oct 30 19:52:28.432048 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:28.432025    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eaa59c33-59f0-43c2-b8ec-68d753622ab9-plugins-config-volume\") pod \"payload-processing-e2e-praxis-b-f03120-79bf49bdd6-z9zzs\" (UID: \"eaa59c33-59f0-43c2-b8ec-68d753622ab9\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-f03120-79bf49bdd6-z9zzs"
Oct 30 19:52:28.442725 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:28.442703    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-f03120-7b689f64f9-vj88z"
Oct 30 19:52:28.447965 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:28.447942    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdlrw\" (UniqueName: \"kubernetes.io/projected/eaa59c33-59f0-43c2-b8ec-68d753622ab9-kube-api-access-xdlrw\") pod \"payload-processing-e2e-praxis-b-f03120-79bf49bdd6-z9zzs\" (UID: \"eaa59c33-59f0-43c2-b8ec-68d753622ab9\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-f03120-79bf49bdd6-z9zzs"
Oct 30 19:52:28.510405 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:28.510068    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-b-f03120-79bf49bdd6-z9zzs"
Oct 30 19:52:28.728197 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:28.728165    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-f03120-7b689f64f9-vj88z"]
Oct 30 19:52:28.728344 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:52:28.728166    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2691319f_ce3f_4930_b1b6_2e35f8f55a64.slice/crio-add9f27fadc9298e2a550922c0872e83f0e730d1a2f27d837d89a7efe4fa9cab WatchSource:0}: Error finding container add9f27fadc9298e2a550922c0872e83f0e730d1a2f27d837d89a7efe4fa9cab: Status 404 returned error can't find the container with id add9f27fadc9298e2a550922c0872e83f0e730d1a2f27d837d89a7efe4fa9cab
Oct 30 19:52:28.811333 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:28.811309    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-f03120-79bf49bdd6-z9zzs"]
Oct 30 19:52:28.813413 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:52:28.813391    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeaa59c33_59f0_43c2_b8ec_68d753622ab9.slice/crio-1b80b763fd8030c83d6171638ac7d5feb71320314a7b8b388e3bfcde74c6511a WatchSource:0}: Error finding container 1b80b763fd8030c83d6171638ac7d5feb71320314a7b8b388e3bfcde74c6511a: Status 404 returned error can't find the container with id 1b80b763fd8030c83d6171638ac7d5feb71320314a7b8b388e3bfcde74c6511a
Oct 30 19:52:29.620244 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:29.620207    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-b-f03120-79bf49bdd6-z9zzs" event={"ID":"eaa59c33-59f0-43c2-b8ec-68d753622ab9","Type":"ContainerStarted","Data":"1468ba88c1fe4669d47383027a568dc1ea18d524d9e305555c21af3ae137d61c"}
Oct 30 19:52:29.620771 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:29.620273    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-b-f03120-79bf49bdd6-z9zzs" event={"ID":"eaa59c33-59f0-43c2-b8ec-68d753622ab9","Type":"ContainerStarted","Data":"1b80b763fd8030c83d6171638ac7d5feb71320314a7b8b388e3bfcde74c6511a"}
Oct 30 19:52:29.620771 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:29.620306    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-praxis-b-f03120-79bf49bdd6-z9zzs"
Oct 30 19:52:29.622151 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:29.622122    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-f03120-7b689f64f9-vj88z" event={"ID":"2691319f-ce3f-4930-b1b6-2e35f8f55a64","Type":"ContainerStarted","Data":"c96b47b9ada5a870e82723e1c48afa0ad13a3607d4eba7cad3aff50a855e2293"}
Oct 30 19:52:29.622245 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:29.622175    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-f03120-7b689f64f9-vj88z" event={"ID":"2691319f-ce3f-4930-b1b6-2e35f8f55a64","Type":"ContainerStarted","Data":"add9f27fadc9298e2a550922c0872e83f0e730d1a2f27d837d89a7efe4fa9cab"}
Oct 30 19:52:29.622245 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:29.622196    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-f03120-7b689f64f9-vj88z"
Oct 30 19:52:29.639852 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:29.639802    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-praxis-b-f03120-79bf49bdd6-z9zzs" podStartSLOduration=1.6397860610000001 podStartE2EDuration="1.639786061s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:52:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:52:29.638807782 +0000 UTC m=+1224.966925954" watchObservedRunningTime="2026-10-30 19:52:29.639786061 +0000 UTC m=+1224.967904216"
Oct 30 19:52:29.660969 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:29.660920    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-f03120-7b689f64f9-vj88z" podStartSLOduration=1.660908952 podStartE2EDuration="1.660908952s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:52:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:52:29.656116352 +0000 UTC m=+1224.984234507" watchObservedRunningTime="2026-10-30 19:52:29.660908952 +0000 UTC m=+1224.989027106"
Oct 30 19:52:31.191838 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:52:31.191799    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 19:52:32.241759 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:32.241701    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-490cdf-6b4679dff-ktbjd"]
Oct 30 19:52:32.247595 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:32.247573    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-490cdf-6b4679dff-ktbjd"
Oct 30 19:52:32.261923 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:32.261897    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-praxis-a-490cdf-serving-cert\""
Oct 30 19:52:32.266152 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:32.266122    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-490cdf-6b4679dff-ktbjd"]
Oct 30 19:52:32.398284 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:32.398240    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5c7dn\" (UniqueName: \"kubernetes.io/projected/94ebc6c2-a915-429d-bb47-b4ce1c118084-kube-api-access-5c7dn\") pod \"maas-api-e2e-praxis-a-490cdf-6b4679dff-ktbjd\" (UID: \"94ebc6c2-a915-429d-bb47-b4ce1c118084\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-490cdf-6b4679dff-ktbjd"
Oct 30 19:52:32.398805 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:32.398777    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/94ebc6c2-a915-429d-bb47-b4ce1c118084-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-490cdf-6b4679dff-ktbjd\" (UID: \"94ebc6c2-a915-429d-bb47-b4ce1c118084\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-490cdf-6b4679dff-ktbjd"
Oct 30 19:52:32.399204 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:32.399150    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94ebc6c2-a915-429d-bb47-b4ce1c118084-metrics-certs\") pod \"maas-api-e2e-praxis-a-490cdf-6b4679dff-ktbjd\" (UID: \"94ebc6c2-a915-429d-bb47-b4ce1c118084\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-490cdf-6b4679dff-ktbjd"
Oct 30 19:52:32.399337 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:32.399266    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/94ebc6c2-a915-429d-bb47-b4ce1c118084-maas-api-tls\") pod \"maas-api-e2e-praxis-a-490cdf-6b4679dff-ktbjd\" (UID: \"94ebc6c2-a915-429d-bb47-b4ce1c118084\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-490cdf-6b4679dff-ktbjd"
Oct 30 19:52:32.500945 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:32.500853    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5c7dn\" (UniqueName: \"kubernetes.io/projected/94ebc6c2-a915-429d-bb47-b4ce1c118084-kube-api-access-5c7dn\") pod \"maas-api-e2e-praxis-a-490cdf-6b4679dff-ktbjd\" (UID: \"94ebc6c2-a915-429d-bb47-b4ce1c118084\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-490cdf-6b4679dff-ktbjd"
Oct 30 19:52:32.500945 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:32.500905    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/94ebc6c2-a915-429d-bb47-b4ce1c118084-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-490cdf-6b4679dff-ktbjd\" (UID: \"94ebc6c2-a915-429d-bb47-b4ce1c118084\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-490cdf-6b4679dff-ktbjd"
Oct 30 19:52:32.501197 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:32.500963    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94ebc6c2-a915-429d-bb47-b4ce1c118084-metrics-certs\") pod \"maas-api-e2e-praxis-a-490cdf-6b4679dff-ktbjd\" (UID: \"94ebc6c2-a915-429d-bb47-b4ce1c118084\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-490cdf-6b4679dff-ktbjd"
Oct 30 19:52:32.501197 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:32.501099    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/94ebc6c2-a915-429d-bb47-b4ce1c118084-maas-api-tls\") pod \"maas-api-e2e-praxis-a-490cdf-6b4679dff-ktbjd\" (UID: \"94ebc6c2-a915-429d-bb47-b4ce1c118084\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-490cdf-6b4679dff-ktbjd"
Oct 30 19:52:32.501325 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:52:32.501240    6996 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-praxis-a-490cdf-serving-cert: secret "maas-api-e2e-praxis-a-490cdf-serving-cert" not found
Oct 30 19:52:32.501325 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:52:32.501324    6996 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94ebc6c2-a915-429d-bb47-b4ce1c118084-maas-api-tls podName:94ebc6c2-a915-429d-bb47-b4ce1c118084 nodeName:}" failed. No retries permitted until 2026-10-30 19:52:33.001304294 +0000 UTC m=+1228.329422425 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/94ebc6c2-a915-429d-bb47-b4ce1c118084-maas-api-tls") pod "maas-api-e2e-praxis-a-490cdf-6b4679dff-ktbjd" (UID: "94ebc6c2-a915-429d-bb47-b4ce1c118084") : secret "maas-api-e2e-praxis-a-490cdf-serving-cert" not found
Oct 30 19:52:32.501591 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:32.501568    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/94ebc6c2-a915-429d-bb47-b4ce1c118084-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-490cdf-6b4679dff-ktbjd\" (UID: \"94ebc6c2-a915-429d-bb47-b4ce1c118084\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-490cdf-6b4679dff-ktbjd"
Oct 30 19:52:32.503541 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:32.503512    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94ebc6c2-a915-429d-bb47-b4ce1c118084-metrics-certs\") pod \"maas-api-e2e-praxis-a-490cdf-6b4679dff-ktbjd\" (UID: \"94ebc6c2-a915-429d-bb47-b4ce1c118084\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-490cdf-6b4679dff-ktbjd"
Oct 30 19:52:32.524513 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:32.524447    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5c7dn\" (UniqueName: \"kubernetes.io/projected/94ebc6c2-a915-429d-bb47-b4ce1c118084-kube-api-access-5c7dn\") pod \"maas-api-e2e-praxis-a-490cdf-6b4679dff-ktbjd\" (UID: \"94ebc6c2-a915-429d-bb47-b4ce1c118084\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-490cdf-6b4679dff-ktbjd"
Oct 30 19:52:33.009755 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:33.009711    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/94ebc6c2-a915-429d-bb47-b4ce1c118084-maas-api-tls\") pod \"maas-api-e2e-praxis-a-490cdf-6b4679dff-ktbjd\" (UID: \"94ebc6c2-a915-429d-bb47-b4ce1c118084\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-490cdf-6b4679dff-ktbjd"
Oct 30 19:52:33.016418 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:33.016385    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/94ebc6c2-a915-429d-bb47-b4ce1c118084-maas-api-tls\") pod \"maas-api-e2e-praxis-a-490cdf-6b4679dff-ktbjd\" (UID: \"94ebc6c2-a915-429d-bb47-b4ce1c118084\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-490cdf-6b4679dff-ktbjd"
Oct 30 19:52:33.166256 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:33.166225    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-490cdf-6b4679dff-ktbjd"
Oct 30 19:52:33.478733 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:33.478704    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-490cdf-6b4679dff-ktbjd"]
Oct 30 19:52:33.479933 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:52:33.479904    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94ebc6c2_a915_429d_bb47_b4ce1c118084.slice/crio-0118b07831d62c15e0e5e194f4b85efdf33bc9b97420d0a322b058937a10224a WatchSource:0}: Error finding container 0118b07831d62c15e0e5e194f4b85efdf33bc9b97420d0a322b058937a10224a: Status 404 returned error can't find the container with id 0118b07831d62c15e0e5e194f4b85efdf33bc9b97420d0a322b058937a10224a
Oct 30 19:52:33.656505 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:33.655530    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-490cdf-6b4679dff-ktbjd" event={"ID":"94ebc6c2-a915-429d-bb47-b4ce1c118084","Type":"ContainerStarted","Data":"0118b07831d62c15e0e5e194f4b85efdf33bc9b97420d0a322b058937a10224a"}
Oct 30 19:52:34.666554 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:34.666513    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-490cdf-6b4679dff-ktbjd" event={"ID":"94ebc6c2-a915-429d-bb47-b4ce1c118084","Type":"ContainerStarted","Data":"333d7c66eaf972dc12722c8ef39f85223af5076558009c33f2942265ea63856b"}
Oct 30 19:52:34.666943 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:34.666631    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-490cdf-6b4679dff-ktbjd"
Oct 30 19:52:34.692185 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:34.692136    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-490cdf-6b4679dff-ktbjd" podStartSLOduration=2.244691041 podStartE2EDuration="2.692117891s" totalImagesPullingTime="447.42685ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:52:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:52:34.690250468 +0000 UTC m=+1230.018368623" watchObservedRunningTime="2026-10-30 19:52:34.692117891 +0000 UTC m=+1230.020236043"
Oct 30 19:52:37.797509 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:37.795660    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-913351-564dc8cdf7-cpcnr"]
Oct 30 19:52:37.797509 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:37.795877    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-913351-564dc8cdf7-cpcnr" podUID="a440bbbf-403d-4f26-976a-b869c6df1635" containerName="maas-api" containerID="cri-o://ca017c2bdf5d993f0d00de6a51efb2721e304b101c0d6355646e4ef3e42d63f1" gracePeriod=30
Oct 30 19:52:38.382745 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:38.382713    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-913351-564dc8cdf7-cpcnr"
Oct 30 19:52:38.406928 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:38.406891    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a440bbbf-403d-4f26-976a-b869c6df1635-kube-api-access-hrpdk\" (UniqueName: \"kubernetes.io/projected/a440bbbf-403d-4f26-976a-b869c6df1635-kube-api-access-hrpdk\") pod \"a440bbbf-403d-4f26-976a-b869c6df1635\" (UID: \"a440bbbf-403d-4f26-976a-b869c6df1635\") "
Oct 30 19:52:38.407318 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:38.407298    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a440bbbf-403d-4f26-976a-b869c6df1635-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a440bbbf-403d-4f26-976a-b869c6df1635-openshift-service-ca\") pod \"a440bbbf-403d-4f26-976a-b869c6df1635\" (UID: \"a440bbbf-403d-4f26-976a-b869c6df1635\") "
Oct 30 19:52:38.407440 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:38.407347    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a440bbbf-403d-4f26-976a-b869c6df1635-metrics-certs\" (UniqueName: \"kubernetes.io/secret/a440bbbf-403d-4f26-976a-b869c6df1635-metrics-certs\") pod \"a440bbbf-403d-4f26-976a-b869c6df1635\" (UID: \"a440bbbf-403d-4f26-976a-b869c6df1635\") "
Oct 30 19:52:38.407440 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:38.407377    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a440bbbf-403d-4f26-976a-b869c6df1635-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a440bbbf-403d-4f26-976a-b869c6df1635-maas-api-tls\") pod \"a440bbbf-403d-4f26-976a-b869c6df1635\" (UID: \"a440bbbf-403d-4f26-976a-b869c6df1635\") "
Oct 30 19:52:38.407994 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:38.407844    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a440bbbf-403d-4f26-976a-b869c6df1635-openshift-service-ca" pod "a440bbbf-403d-4f26-976a-b869c6df1635" (UID: "a440bbbf-403d-4f26-976a-b869c6df1635"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:52:38.408143 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:38.408123    6996 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a440bbbf-403d-4f26-976a-b869c6df1635-openshift-service-ca\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:52:38.412044 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:38.412001    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a440bbbf-403d-4f26-976a-b869c6df1635-maas-api-tls" pod "a440bbbf-403d-4f26-976a-b869c6df1635" (UID: "a440bbbf-403d-4f26-976a-b869c6df1635"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:52:38.412528 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:38.412466    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a440bbbf-403d-4f26-976a-b869c6df1635-kube-api-access-hrpdk" pod "a440bbbf-403d-4f26-976a-b869c6df1635" (UID: "a440bbbf-403d-4f26-976a-b869c6df1635"). InnerVolumeSpecName "kube-api-access-hrpdk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:52:38.414083 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:38.414057    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a440bbbf-403d-4f26-976a-b869c6df1635-metrics-certs" pod "a440bbbf-403d-4f26-976a-b869c6df1635" (UID: "a440bbbf-403d-4f26-976a-b869c6df1635"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:52:38.509728 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:38.509692    6996 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a440bbbf-403d-4f26-976a-b869c6df1635-metrics-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:52:38.509728 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:38.509721    6996 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a440bbbf-403d-4f26-976a-b869c6df1635-maas-api-tls\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:52:38.509728 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:38.509733    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hrpdk\" (UniqueName: \"kubernetes.io/projected/a440bbbf-403d-4f26-976a-b869c6df1635-kube-api-access-hrpdk\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:52:38.698906 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:38.698814    6996 generic.go:356] "Generic (PLEG): container finished" podID="a440bbbf-403d-4f26-976a-b869c6df1635" containerID="ca017c2bdf5d993f0d00de6a51efb2721e304b101c0d6355646e4ef3e42d63f1" exitCode=0
Oct 30 19:52:38.698906 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:38.698881    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-913351-564dc8cdf7-cpcnr"
Oct 30 19:52:38.699116 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:38.698906    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-913351-564dc8cdf7-cpcnr" event={"ID":"a440bbbf-403d-4f26-976a-b869c6df1635","Type":"ContainerDied","Data":"ca017c2bdf5d993f0d00de6a51efb2721e304b101c0d6355646e4ef3e42d63f1"}
Oct 30 19:52:38.699116 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:38.698975    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-913351-564dc8cdf7-cpcnr" event={"ID":"a440bbbf-403d-4f26-976a-b869c6df1635","Type":"ContainerDied","Data":"9c8d3313287ca880fcfe15d1de0ed7409856678e63d246423afc61fe3e063bcc"}
Oct 30 19:52:38.699116 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:38.699009    6996 scope.go:122] "RemoveContainer" containerID="ca017c2bdf5d993f0d00de6a51efb2721e304b101c0d6355646e4ef3e42d63f1"
Oct 30 19:52:38.711083 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:38.711015    6996 scope.go:122] "RemoveContainer" containerID="ca017c2bdf5d993f0d00de6a51efb2721e304b101c0d6355646e4ef3e42d63f1"
Oct 30 19:52:38.711401 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:52:38.711375    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca017c2bdf5d993f0d00de6a51efb2721e304b101c0d6355646e4ef3e42d63f1\": container with ID starting with ca017c2bdf5d993f0d00de6a51efb2721e304b101c0d6355646e4ef3e42d63f1 not found: ID does not exist" containerID="ca017c2bdf5d993f0d00de6a51efb2721e304b101c0d6355646e4ef3e42d63f1"
Oct 30 19:52:38.711530 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:38.711405    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca017c2bdf5d993f0d00de6a51efb2721e304b101c0d6355646e4ef3e42d63f1"} err="failed to get container status \"ca017c2bdf5d993f0d00de6a51efb2721e304b101c0d6355646e4ef3e42d63f1\": rpc error: code = NotFound desc = could not find container \"ca017c2bdf5d993f0d00de6a51efb2721e304b101c0d6355646e4ef3e42d63f1\": container with ID starting with ca017c2bdf5d993f0d00de6a51efb2721e304b101c0d6355646e4ef3e42d63f1 not found: ID does not exist"
Oct 30 19:52:38.760569 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:38.741677    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-913351-564dc8cdf7-cpcnr"]
Oct 30 19:52:38.760569 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:38.753929    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-913351-564dc8cdf7-cpcnr"]
Oct 30 19:52:38.795205 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:38.795166    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a440bbbf-403d-4f26-976a-b869c6df1635" path="/var/lib/kubelet/pods/a440bbbf-403d-4f26-976a-b869c6df1635/volumes"
Oct 30 19:52:40.651507 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:40.650937    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-praxis-b-f03120-79bf49bdd6-z9zzs"
Oct 30 19:52:40.666501 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:40.665664    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-f03120-7b689f64f9-vj88z"
Oct 30 19:52:40.830133 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:40.830077    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-490cdf-6b4679dff-ktbjd"
Oct 30 19:52:51.627248 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:51.627210    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj"]
Oct 30 19:52:51.628473 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:51.628450    6996 cpu_manager.go:414] "containerMap: removing container" podUID="a440bbbf-403d-4f26-976a-b869c6df1635" containerName="maas-api"
Oct 30 19:52:51.628473 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:51.628494    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a440bbbf-403d-4f26-976a-b869c6df1635" containerName="maas-api"
Oct 30 19:52:51.628713 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:51.628699    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="a440bbbf-403d-4f26-976a-b869c6df1635" containerName="maas-api"
Oct 30 19:52:51.634395 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:51.634372    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj"
Oct 30 19:52:51.659579 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:51.659543    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj"]
Oct 30 19:52:51.666216 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:51.665396    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5d54632-af1e-458b-b431-61ead4fccb41-istio-podinfo\") pod \"e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj\" (UID: \"f5d54632-af1e-458b-b431-61ead4fccb41\") " pod="openshift-ingress/e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj"
Oct 30 19:52:51.666216 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:51.665463    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5d54632-af1e-458b-b431-61ead4fccb41-istio-envoy\") pod \"e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj\" (UID: \"f5d54632-af1e-458b-b431-61ead4fccb41\") " pod="openshift-ingress/e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj"
Oct 30 19:52:51.666216 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:51.665524    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5d54632-af1e-458b-b431-61ead4fccb41-istiod-ca-cert\") pod \"e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj\" (UID: \"f5d54632-af1e-458b-b431-61ead4fccb41\") " pod="openshift-ingress/e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj"
Oct 30 19:52:51.666216 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:51.665680    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxmsr\" (UniqueName: \"kubernetes.io/projected/f5d54632-af1e-458b-b431-61ead4fccb41-kube-api-access-kxmsr\") pod \"e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj\" (UID: \"f5d54632-af1e-458b-b431-61ead4fccb41\") " pod="openshift-ingress/e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj"
Oct 30 19:52:51.666216 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:51.665773    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5d54632-af1e-458b-b431-61ead4fccb41-credential-socket\") pod \"e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj\" (UID: \"f5d54632-af1e-458b-b431-61ead4fccb41\") " pod="openshift-ingress/e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj"
Oct 30 19:52:51.666216 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:51.665836    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5d54632-af1e-458b-b431-61ead4fccb41-workload-certs\") pod \"e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj\" (UID: \"f5d54632-af1e-458b-b431-61ead4fccb41\") " pod="openshift-ingress/e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj"
Oct 30 19:52:51.666216 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:51.665912    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5d54632-af1e-458b-b431-61ead4fccb41-istio-token\") pod \"e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj\" (UID: \"f5d54632-af1e-458b-b431-61ead4fccb41\") " pod="openshift-ingress/e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj"
Oct 30 19:52:51.666216 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:51.665944    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5d54632-af1e-458b-b431-61ead4fccb41-workload-socket\") pod \"e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj\" (UID: \"f5d54632-af1e-458b-b431-61ead4fccb41\") " pod="openshift-ingress/e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj"
Oct 30 19:52:51.666216 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:51.666046    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5d54632-af1e-458b-b431-61ead4fccb41-istio-data\") pod \"e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj\" (UID: \"f5d54632-af1e-458b-b431-61ead4fccb41\") " pod="openshift-ingress/e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj"
Oct 30 19:52:51.726505 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:51.723380    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-praxis-b-ee4af9-openshift-default-dockercfg-bqft2\""
Oct 30 19:52:51.835563 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:51.826873    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5d54632-af1e-458b-b431-61ead4fccb41-credential-socket\") pod \"e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj\" (UID: \"f5d54632-af1e-458b-b431-61ead4fccb41\") " pod="openshift-ingress/e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj"
Oct 30 19:52:51.835563 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:51.826968    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5d54632-af1e-458b-b431-61ead4fccb41-workload-certs\") pod \"e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj\" (UID: \"f5d54632-af1e-458b-b431-61ead4fccb41\") " pod="openshift-ingress/e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj"
Oct 30 19:52:51.835563 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:51.827064    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5d54632-af1e-458b-b431-61ead4fccb41-istio-token\") pod \"e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj\" (UID: \"f5d54632-af1e-458b-b431-61ead4fccb41\") " pod="openshift-ingress/e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj"
Oct 30 19:52:51.835563 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:51.827097    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5d54632-af1e-458b-b431-61ead4fccb41-workload-socket\") pod \"e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj\" (UID: \"f5d54632-af1e-458b-b431-61ead4fccb41\") " pod="openshift-ingress/e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj"
Oct 30 19:52:51.835563 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:51.827206    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5d54632-af1e-458b-b431-61ead4fccb41-istio-data\") pod \"e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj\" (UID: \"f5d54632-af1e-458b-b431-61ead4fccb41\") " pod="openshift-ingress/e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj"
Oct 30 19:52:51.835563 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:51.827283    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5d54632-af1e-458b-b431-61ead4fccb41-istio-podinfo\") pod \"e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj\" (UID: \"f5d54632-af1e-458b-b431-61ead4fccb41\") " pod="openshift-ingress/e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj"
Oct 30 19:52:51.835563 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:51.827331    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5d54632-af1e-458b-b431-61ead4fccb41-istio-envoy\") pod \"e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj\" (UID: \"f5d54632-af1e-458b-b431-61ead4fccb41\") " pod="openshift-ingress/e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj"
Oct 30 19:52:51.835563 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:51.827375    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5d54632-af1e-458b-b431-61ead4fccb41-istiod-ca-cert\") pod \"e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj\" (UID: \"f5d54632-af1e-458b-b431-61ead4fccb41\") " pod="openshift-ingress/e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj"
Oct 30 19:52:51.835563 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:51.827466    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxmsr\" (UniqueName: \"kubernetes.io/projected/f5d54632-af1e-458b-b431-61ead4fccb41-kube-api-access-kxmsr\") pod \"e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj\" (UID: \"f5d54632-af1e-458b-b431-61ead4fccb41\") " pod="openshift-ingress/e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj"
Oct 30 19:52:51.835563 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:51.828316    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5d54632-af1e-458b-b431-61ead4fccb41-credential-socket\") pod \"e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj\" (UID: \"f5d54632-af1e-458b-b431-61ead4fccb41\") " pod="openshift-ingress/e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj"
Oct 30 19:52:51.851503 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:51.840916    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5d54632-af1e-458b-b431-61ead4fccb41-istio-data\") pod \"e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj\" (UID: \"f5d54632-af1e-458b-b431-61ead4fccb41\") " pod="openshift-ingress/e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj"
Oct 30 19:52:51.851503 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:51.841575    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5d54632-af1e-458b-b431-61ead4fccb41-workload-socket\") pod \"e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj\" (UID: \"f5d54632-af1e-458b-b431-61ead4fccb41\") " pod="openshift-ingress/e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj"
Oct 30 19:52:51.851503 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:51.846131    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5d54632-af1e-458b-b431-61ead4fccb41-istiod-ca-cert\") pod \"e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj\" (UID: \"f5d54632-af1e-458b-b431-61ead4fccb41\") " pod="openshift-ingress/e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj"
Oct 30 19:52:51.851503 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:51.846406    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5d54632-af1e-458b-b431-61ead4fccb41-istio-envoy\") pod \"e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj\" (UID: \"f5d54632-af1e-458b-b431-61ead4fccb41\") " pod="openshift-ingress/e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj"
Oct 30 19:52:51.878551 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:51.870810    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5d54632-af1e-458b-b431-61ead4fccb41-workload-certs\") pod \"e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj\" (UID: \"f5d54632-af1e-458b-b431-61ead4fccb41\") " pod="openshift-ingress/e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj"
Oct 30 19:52:51.925507 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:51.924159    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxmsr\" (UniqueName: \"kubernetes.io/projected/f5d54632-af1e-458b-b431-61ead4fccb41-kube-api-access-kxmsr\") pod \"e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj\" (UID: \"f5d54632-af1e-458b-b431-61ead4fccb41\") " pod="openshift-ingress/e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj"
Oct 30 19:52:51.925507 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:51.925050    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5d54632-af1e-458b-b431-61ead4fccb41-istio-token\") pod \"e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj\" (UID: \"f5d54632-af1e-458b-b431-61ead4fccb41\") " pod="openshift-ingress/e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj"
Oct 30 19:52:51.926384 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:51.926332    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5d54632-af1e-458b-b431-61ead4fccb41-istio-podinfo\") pod \"e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj\" (UID: \"f5d54632-af1e-458b-b431-61ead4fccb41\") " pod="openshift-ingress/e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj"
Oct 30 19:52:51.950861 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:51.950820    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj"
Oct 30 19:52:52.260856 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:52.260817    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj"]
Oct 30 19:52:52.261951 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:52:52.261922    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5d54632_af1e_458b_b431_61ead4fccb41.slice/crio-81ec55f46530eff6fab813e59cf519e3d9368c0b95bdf233d0ace212d2480a77 WatchSource:0}: Error finding container 81ec55f46530eff6fab813e59cf519e3d9368c0b95bdf233d0ace212d2480a77: Status 404 returned error can't find the container with id 81ec55f46530eff6fab813e59cf519e3d9368c0b95bdf233d0ace212d2480a77
Oct 30 19:52:52.264749 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:52.264718    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:52:52.264824 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:52.264783    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:52:52.264824 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:52.264811    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:52:52.980247 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:52.980210    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj" event={"ID":"f5d54632-af1e-458b-b431-61ead4fccb41","Type":"ContainerStarted","Data":"ae4a4d02ebdfb0c0f0f1f0971c0758bec522e491f62c1dd1f5cbe8fc088f0075"}
Oct 30 19:52:52.980690 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:52.980273    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj" event={"ID":"f5d54632-af1e-458b-b431-61ead4fccb41","Type":"ContainerStarted","Data":"81ec55f46530eff6fab813e59cf519e3d9368c0b95bdf233d0ace212d2480a77"}
Oct 30 19:52:53.008085 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:53.008036    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj" podStartSLOduration=2.008022811 podStartE2EDuration="2.008022811s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:52:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:52:53.004546264 +0000 UTC m=+1248.332664419" watchObservedRunningTime="2026-10-30 19:52:53.008022811 +0000 UTC m=+1248.336141028"
Oct 30 19:52:53.951457 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:53.951425    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj"
Oct 30 19:52:53.957220 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:53.957197    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj"
Oct 30 19:52:53.989089 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:53.989060    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj"
Oct 30 19:52:53.989959 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:53.989943    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj"
Oct 30 19:52:59.724217 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:59.724177    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-ec5e0f-6b5968df56-7mddg"]
Oct 30 19:52:59.724769 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:52:59.724473    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ec5e0f-6b5968df56-7mddg" podUID="b01476c8-1aed-4fd1-acb0-6aea531856ca" containerName="maas-api" containerID="cri-o://6beab979d1bcd0f90e148c7e98e57ecfd0cf3ac9883777d5933a41a147f0e1c5" gracePeriod=30
Oct 30 19:53:00.037841 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:00.037803    6996 generic.go:356] "Generic (PLEG): container finished" podID="b01476c8-1aed-4fd1-acb0-6aea531856ca" containerID="6beab979d1bcd0f90e148c7e98e57ecfd0cf3ac9883777d5933a41a147f0e1c5" exitCode=0
Oct 30 19:53:00.037999 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:00.037859    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ec5e0f-6b5968df56-7mddg" event={"ID":"b01476c8-1aed-4fd1-acb0-6aea531856ca","Type":"ContainerDied","Data":"6beab979d1bcd0f90e148c7e98e57ecfd0cf3ac9883777d5933a41a147f0e1c5"}
Oct 30 19:53:00.037999 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:00.037922    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ec5e0f-6b5968df56-7mddg" event={"ID":"b01476c8-1aed-4fd1-acb0-6aea531856ca","Type":"ContainerDied","Data":"a1416d7d4df522056e54528dce60b1416db181db3448e6b50bfba8edffad3c32"}
Oct 30 19:53:00.037999 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:00.037943    6996 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a1416d7d4df522056e54528dce60b1416db181db3448e6b50bfba8edffad3c32"
Oct 30 19:53:00.046035 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:00.046010    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ec5e0f-6b5968df56-7mddg"
Oct 30 19:53:00.062743 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:00.062716    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b01476c8-1aed-4fd1-acb0-6aea531856ca-kube-api-access-rgw2q\" (UniqueName: \"kubernetes.io/projected/b01476c8-1aed-4fd1-acb0-6aea531856ca-kube-api-access-rgw2q\") pod \"b01476c8-1aed-4fd1-acb0-6aea531856ca\" (UID: \"b01476c8-1aed-4fd1-acb0-6aea531856ca\") "
Oct 30 19:53:00.062905 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:00.062764    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b01476c8-1aed-4fd1-acb0-6aea531856ca-metrics-certs\" (UniqueName: \"kubernetes.io/secret/b01476c8-1aed-4fd1-acb0-6aea531856ca-metrics-certs\") pod \"b01476c8-1aed-4fd1-acb0-6aea531856ca\" (UID: \"b01476c8-1aed-4fd1-acb0-6aea531856ca\") "
Oct 30 19:53:00.062905 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:00.062800    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b01476c8-1aed-4fd1-acb0-6aea531856ca-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b01476c8-1aed-4fd1-acb0-6aea531856ca-openshift-service-ca\") pod \"b01476c8-1aed-4fd1-acb0-6aea531856ca\" (UID: \"b01476c8-1aed-4fd1-acb0-6aea531856ca\") "
Oct 30 19:53:00.063039 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:00.062933    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b01476c8-1aed-4fd1-acb0-6aea531856ca-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b01476c8-1aed-4fd1-acb0-6aea531856ca-maas-api-tls\") pod \"b01476c8-1aed-4fd1-acb0-6aea531856ca\" (UID: \"b01476c8-1aed-4fd1-acb0-6aea531856ca\") "
Oct 30 19:53:00.063287 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:00.063255    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b01476c8-1aed-4fd1-acb0-6aea531856ca-openshift-service-ca" pod "b01476c8-1aed-4fd1-acb0-6aea531856ca" (UID: "b01476c8-1aed-4fd1-acb0-6aea531856ca"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:53:00.063455 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:00.063432    6996 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b01476c8-1aed-4fd1-acb0-6aea531856ca-openshift-service-ca\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:53:00.065384 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:00.065352    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b01476c8-1aed-4fd1-acb0-6aea531856ca-maas-api-tls" pod "b01476c8-1aed-4fd1-acb0-6aea531856ca" (UID: "b01476c8-1aed-4fd1-acb0-6aea531856ca"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:53:00.065537 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:00.065439    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b01476c8-1aed-4fd1-acb0-6aea531856ca-kube-api-access-rgw2q" pod "b01476c8-1aed-4fd1-acb0-6aea531856ca" (UID: "b01476c8-1aed-4fd1-acb0-6aea531856ca"). InnerVolumeSpecName "kube-api-access-rgw2q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:53:00.065762 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:00.065742    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b01476c8-1aed-4fd1-acb0-6aea531856ca-metrics-certs" pod "b01476c8-1aed-4fd1-acb0-6aea531856ca" (UID: "b01476c8-1aed-4fd1-acb0-6aea531856ca"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:53:00.198504 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:00.173869    6996 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b01476c8-1aed-4fd1-acb0-6aea531856ca-maas-api-tls\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:53:00.211814 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:00.211782    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rgw2q\" (UniqueName: \"kubernetes.io/projected/b01476c8-1aed-4fd1-acb0-6aea531856ca-kube-api-access-rgw2q\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:53:00.212054 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:00.212036    6996 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b01476c8-1aed-4fd1-acb0-6aea531856ca-metrics-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:53:01.045625 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:01.045587    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ec5e0f-6b5968df56-7mddg"
Oct 30 19:53:01.081664 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:01.081629    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-ec5e0f-6b5968df56-7mddg"]
Oct 30 19:53:01.091949 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:01.091913    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-ec5e0f-6b5968df56-7mddg"]
Oct 30 19:53:01.193614 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:53:01.193576    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 19:53:02.794119 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:02.794088    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b01476c8-1aed-4fd1-acb0-6aea531856ca" path="/var/lib/kubelet/pods/b01476c8-1aed-4fd1-acb0-6aea531856ca/volumes"
Oct 30 19:53:09.324356 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:09.324319    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8"]
Oct 30 19:53:09.324892 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:09.324575    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8" podUID="81198f3c-12bb-4356-bc2e-1134a145c971" containerName="istio-proxy" containerID="cri-o://37b7955bdc83b6382bb1e845ca7152935c5a1d21f28ee2754d7ec560f497f3a9" gracePeriod=30
Oct 30 19:53:14.611422 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:14.611395    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8"
Oct 30 19:53:14.762399 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:14.762348    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81198f3c-12bb-4356-bc2e-1134a145c971-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81198f3c-12bb-4356-bc2e-1134a145c971-istio-data\") pod \"81198f3c-12bb-4356-bc2e-1134a145c971\" (UID: \"81198f3c-12bb-4356-bc2e-1134a145c971\") "
Oct 30 19:53:14.762718 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:14.762675    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81198f3c-12bb-4356-bc2e-1134a145c971-istio-data" pod "81198f3c-12bb-4356-bc2e-1134a145c971" (UID: "81198f3c-12bb-4356-bc2e-1134a145c971"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:53:14.762912 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:14.762883    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/81198f3c-12bb-4356-bc2e-1134a145c971-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81198f3c-12bb-4356-bc2e-1134a145c971-istiod-ca-cert\") pod \"81198f3c-12bb-4356-bc2e-1134a145c971\" (UID: \"81198f3c-12bb-4356-bc2e-1134a145c971\") "
Oct 30 19:53:14.763045 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:14.762995    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81198f3c-12bb-4356-bc2e-1134a145c971-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81198f3c-12bb-4356-bc2e-1134a145c971-workload-socket\") pod \"81198f3c-12bb-4356-bc2e-1134a145c971\" (UID: \"81198f3c-12bb-4356-bc2e-1134a145c971\") "
Oct 30 19:53:14.763119 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:14.763074    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81198f3c-12bb-4356-bc2e-1134a145c971-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81198f3c-12bb-4356-bc2e-1134a145c971-istio-envoy\") pod \"81198f3c-12bb-4356-bc2e-1134a145c971\" (UID: \"81198f3c-12bb-4356-bc2e-1134a145c971\") "
Oct 30 19:53:14.763119 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:14.763108    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81198f3c-12bb-4356-bc2e-1134a145c971-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81198f3c-12bb-4356-bc2e-1134a145c971-workload-certs\") pod \"81198f3c-12bb-4356-bc2e-1134a145c971\" (UID: \"81198f3c-12bb-4356-bc2e-1134a145c971\") "
Oct 30 19:53:14.763251 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:14.763181    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/81198f3c-12bb-4356-bc2e-1134a145c971-istio-token\" (UniqueName: \"kubernetes.io/projected/81198f3c-12bb-4356-bc2e-1134a145c971-istio-token\") pod \"81198f3c-12bb-4356-bc2e-1134a145c971\" (UID: \"81198f3c-12bb-4356-bc2e-1134a145c971\") "
Oct 30 19:53:14.763356 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:14.763267    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/81198f3c-12bb-4356-bc2e-1134a145c971-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81198f3c-12bb-4356-bc2e-1134a145c971-istio-podinfo\") pod \"81198f3c-12bb-4356-bc2e-1134a145c971\" (UID: \"81198f3c-12bb-4356-bc2e-1134a145c971\") "
Oct 30 19:53:14.763356 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:14.763304    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81198f3c-12bb-4356-bc2e-1134a145c971-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81198f3c-12bb-4356-bc2e-1134a145c971-credential-socket\") pod \"81198f3c-12bb-4356-bc2e-1134a145c971\" (UID: \"81198f3c-12bb-4356-bc2e-1134a145c971\") "
Oct 30 19:53:14.763545 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:14.763506    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/81198f3c-12bb-4356-bc2e-1134a145c971-kube-api-access-9kfmf\" (UniqueName: \"kubernetes.io/projected/81198f3c-12bb-4356-bc2e-1134a145c971-kube-api-access-9kfmf\") pod \"81198f3c-12bb-4356-bc2e-1134a145c971\" (UID: \"81198f3c-12bb-4356-bc2e-1134a145c971\") "
Oct 30 19:53:14.763917 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:14.763885    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81198f3c-12bb-4356-bc2e-1134a145c971-istiod-ca-cert" pod "81198f3c-12bb-4356-bc2e-1134a145c971" (UID: "81198f3c-12bb-4356-bc2e-1134a145c971"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:53:14.764677 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:14.764628    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81198f3c-12bb-4356-bc2e-1134a145c971-credential-socket" pod "81198f3c-12bb-4356-bc2e-1134a145c971" (UID: "81198f3c-12bb-4356-bc2e-1134a145c971"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:53:14.764778 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:14.764697    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81198f3c-12bb-4356-bc2e-1134a145c971-workload-socket" pod "81198f3c-12bb-4356-bc2e-1134a145c971" (UID: "81198f3c-12bb-4356-bc2e-1134a145c971"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:53:14.764864 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:14.764843    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81198f3c-12bb-4356-bc2e-1134a145c971-workload-certs" pod "81198f3c-12bb-4356-bc2e-1134a145c971" (UID: "81198f3c-12bb-4356-bc2e-1134a145c971"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:53:14.765400 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:14.765374    6996 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81198f3c-12bb-4356-bc2e-1134a145c971-credential-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:53:14.765507 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:14.765406    6996 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81198f3c-12bb-4356-bc2e-1134a145c971-istio-data\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:53:14.765507 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:14.765437    6996 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81198f3c-12bb-4356-bc2e-1134a145c971-istiod-ca-cert\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:53:14.765507 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:14.765452    6996 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81198f3c-12bb-4356-bc2e-1134a145c971-workload-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:53:14.765697 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:14.765530    6996 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81198f3c-12bb-4356-bc2e-1134a145c971-workload-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:53:14.767145 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:14.767114    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81198f3c-12bb-4356-bc2e-1134a145c971-istio-token" pod "81198f3c-12bb-4356-bc2e-1134a145c971" (UID: "81198f3c-12bb-4356-bc2e-1134a145c971"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:53:14.767366 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:14.767341    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/81198f3c-12bb-4356-bc2e-1134a145c971-istio-podinfo" pod "81198f3c-12bb-4356-bc2e-1134a145c971" (UID: "81198f3c-12bb-4356-bc2e-1134a145c971"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:53:14.767622 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:14.767602    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81198f3c-12bb-4356-bc2e-1134a145c971-istio-envoy" pod "81198f3c-12bb-4356-bc2e-1134a145c971" (UID: "81198f3c-12bb-4356-bc2e-1134a145c971"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:53:14.768025 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:14.768000    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81198f3c-12bb-4356-bc2e-1134a145c971-kube-api-access-9kfmf" pod "81198f3c-12bb-4356-bc2e-1134a145c971" (UID: "81198f3c-12bb-4356-bc2e-1134a145c971"). InnerVolumeSpecName "kube-api-access-9kfmf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:53:14.868311 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:14.867037    6996 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81198f3c-12bb-4356-bc2e-1134a145c971-istio-podinfo\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:53:14.868311 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:14.867064    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9kfmf\" (UniqueName: \"kubernetes.io/projected/81198f3c-12bb-4356-bc2e-1134a145c971-kube-api-access-9kfmf\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:53:14.868311 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:14.867077    6996 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81198f3c-12bb-4356-bc2e-1134a145c971-istio-envoy\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:53:14.868311 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:14.867085    6996 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81198f3c-12bb-4356-bc2e-1134a145c971-istio-token\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:53:15.184739 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:15.184647    6996 generic.go:356] "Generic (PLEG): container finished" podID="81198f3c-12bb-4356-bc2e-1134a145c971" containerID="37b7955bdc83b6382bb1e845ca7152935c5a1d21f28ee2754d7ec560f497f3a9" exitCode=0
Oct 30 19:53:15.184739 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:15.184704    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8"
Oct 30 19:53:15.184920 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:15.184730    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8" event={"ID":"81198f3c-12bb-4356-bc2e-1134a145c971","Type":"ContainerDied","Data":"37b7955bdc83b6382bb1e845ca7152935c5a1d21f28ee2754d7ec560f497f3a9"}
Oct 30 19:53:15.184920 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:15.184786    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8" event={"ID":"81198f3c-12bb-4356-bc2e-1134a145c971","Type":"ContainerDied","Data":"fa4f669e53a27449f763574439d5173aa309ff54b5d87cf06747caa8d9a26db8"}
Oct 30 19:53:15.184920 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:15.184809    6996 scope.go:122] "RemoveContainer" containerID="37b7955bdc83b6382bb1e845ca7152935c5a1d21f28ee2754d7ec560f497f3a9"
Oct 30 19:53:15.195917 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:15.195899    6996 scope.go:122] "RemoveContainer" containerID="37b7955bdc83b6382bb1e845ca7152935c5a1d21f28ee2754d7ec560f497f3a9"
Oct 30 19:53:15.196209 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:53:15.196177    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37b7955bdc83b6382bb1e845ca7152935c5a1d21f28ee2754d7ec560f497f3a9\": container with ID starting with 37b7955bdc83b6382bb1e845ca7152935c5a1d21f28ee2754d7ec560f497f3a9 not found: ID does not exist" containerID="37b7955bdc83b6382bb1e845ca7152935c5a1d21f28ee2754d7ec560f497f3a9"
Oct 30 19:53:15.196262 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:15.196216    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37b7955bdc83b6382bb1e845ca7152935c5a1d21f28ee2754d7ec560f497f3a9"} err="failed to get container status \"37b7955bdc83b6382bb1e845ca7152935c5a1d21f28ee2754d7ec560f497f3a9\": rpc error: code = NotFound desc = could not find container \"37b7955bdc83b6382bb1e845ca7152935c5a1d21f28ee2754d7ec560f497f3a9\": container with ID starting with 37b7955bdc83b6382bb1e845ca7152935c5a1d21f28ee2754d7ec560f497f3a9 not found: ID does not exist"
Oct 30 19:53:15.204203 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:15.204176    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8"]
Oct 30 19:53:15.209446 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:15.209423    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-ec5e0f-openshift-default-c6576c9bf-cn9k8"]
Oct 30 19:53:16.792812 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:16.792736    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="81198f3c-12bb-4356-bc2e-1134a145c971" path="/var/lib/kubelet/pods/81198f3c-12bb-4356-bc2e-1134a145c971/volumes"
Oct 30 19:53:19.507298 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:19.507255    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 19:53:31.190579 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:53:31.190466    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 19:53:31.348518 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:31.348449    6996 generic.go:356] "Generic (PLEG): container finished" podID="f5d54632-af1e-458b-b431-61ead4fccb41" containerID="ae4a4d02ebdfb0c0f0f1f0971c0758bec522e491f62c1dd1f5cbe8fc088f0075" exitCode=0
Oct 30 19:53:31.348712 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:31.348534    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj" event={"ID":"f5d54632-af1e-458b-b431-61ead4fccb41","Type":"ContainerDied","Data":"ae4a4d02ebdfb0c0f0f1f0971c0758bec522e491f62c1dd1f5cbe8fc088f0075"}
Oct 30 19:53:31.349029 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:31.349013    6996 scope.go:122] "RemoveContainer" containerID="ae4a4d02ebdfb0c0f0f1f0971c0758bec522e491f62c1dd1f5cbe8fc088f0075"
Oct 30 19:53:31.350729 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:31.350683    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:53:31.350814 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:31.350754    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:53:31.350814 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:31.350786    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:53:32.369688 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:32.369650    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj" event={"ID":"f5d54632-af1e-458b-b431-61ead4fccb41","Type":"ContainerStarted","Data":"35b5f8862e8075ad5c020ce3c69432a20c085ba9d3072314b0eee58db437d97d"}
Oct 30 19:53:32.774506 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:32.774404    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f"]
Oct 30 19:53:32.774670 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:32.774618    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f" podUID="29afff26-d867-4aaf-9d15-ea2f45d4a091" containerName="istio-proxy" containerID="cri-o://2a02fb5bcf9c579f93759152f2aba99bcca11e186939fd23f9bb8e5e24ddbb5a" gracePeriod=30
Oct 30 19:53:32.956374 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:32.955873    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj"
Oct 30 19:53:32.956374 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:32.956001    6996 patch_prober.go:29] interesting pod/e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.131:15021/healthz/ready\": dial tcp 10.134.0.131:15021: connect: connection refused" start-of-body=
Oct 30 19:53:32.956374 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:32.956071    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj" podUID="f5d54632-af1e-458b-b431-61ead4fccb41" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.131:15021/healthz/ready\": dial tcp 10.134.0.131:15021: connect: connection refused"
Oct 30 19:53:33.961874 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:33.961843    6996 patch_prober.go:29] interesting pod/e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.131:15021/healthz/ready\": dial tcp 10.134.0.131:15021: connect: connection refused" start-of-body=
Oct 30 19:53:33.962324 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:33.961898    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj" podUID="f5d54632-af1e-458b-b431-61ead4fccb41" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.131:15021/healthz/ready\": dial tcp 10.134.0.131:15021: connect: connection refused"
Oct 30 19:53:34.306456 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:34.306367    6996 patch_prober.go:29] interesting pod/e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:53:34.306456 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:34.306436    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f" podUID="29afff26-d867-4aaf-9d15-ea2f45d4a091" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:53:34.958505 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:34.957506    6996 patch_prober.go:29] interesting pod/e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.131:15021/healthz/ready\": dial tcp 10.134.0.131:15021: connect: connection refused" start-of-body=
Oct 30 19:53:34.958505 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:34.957601    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj" podUID="f5d54632-af1e-458b-b431-61ead4fccb41" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.131:15021/healthz/ready\": dial tcp 10.134.0.131:15021: connect: connection refused"
Oct 30 19:53:35.961504 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:35.956143    6996 patch_prober.go:29] interesting pod/e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.131:15021/healthz/ready\": dial tcp 10.134.0.131:15021: connect: connection refused" start-of-body=
Oct 30 19:53:35.961504 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:35.956201    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj" podUID="f5d54632-af1e-458b-b431-61ead4fccb41" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.131:15021/healthz/ready\": dial tcp 10.134.0.131:15021: connect: connection refused"
Oct 30 19:53:36.960507 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:36.958327    6996 patch_prober.go:29] interesting pod/e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.131:15021/healthz/ready\": dial tcp 10.134.0.131:15021: connect: connection refused" start-of-body=
Oct 30 19:53:36.960507 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:36.958403    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj" podUID="f5d54632-af1e-458b-b431-61ead4fccb41" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.131:15021/healthz/ready\": dial tcp 10.134.0.131:15021: connect: connection refused"
Oct 30 19:53:37.961640 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:37.958374    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj"
Oct 30 19:53:37.961640 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:37.959970    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj"
Oct 30 19:53:37.961640 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:37.960975    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj"
Oct 30 19:53:38.122880 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:38.122848    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f"
Oct 30 19:53:38.151633 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:38.151575    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/29afff26-d867-4aaf-9d15-ea2f45d4a091-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29afff26-d867-4aaf-9d15-ea2f45d4a091-istio-podinfo\") pod \"29afff26-d867-4aaf-9d15-ea2f45d4a091\" (UID: \"29afff26-d867-4aaf-9d15-ea2f45d4a091\") "
Oct 30 19:53:38.151818 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:38.151665    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29afff26-d867-4aaf-9d15-ea2f45d4a091-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29afff26-d867-4aaf-9d15-ea2f45d4a091-credential-socket\") pod \"29afff26-d867-4aaf-9d15-ea2f45d4a091\" (UID: \"29afff26-d867-4aaf-9d15-ea2f45d4a091\") "
Oct 30 19:53:38.151818 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:38.151696    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29afff26-d867-4aaf-9d15-ea2f45d4a091-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29afff26-d867-4aaf-9d15-ea2f45d4a091-workload-socket\") pod \"29afff26-d867-4aaf-9d15-ea2f45d4a091\" (UID: \"29afff26-d867-4aaf-9d15-ea2f45d4a091\") "
Oct 30 19:53:38.151943 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:38.151819    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29afff26-d867-4aaf-9d15-ea2f45d4a091-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29afff26-d867-4aaf-9d15-ea2f45d4a091-workload-certs\") pod \"29afff26-d867-4aaf-9d15-ea2f45d4a091\" (UID: \"29afff26-d867-4aaf-9d15-ea2f45d4a091\") "
Oct 30 19:53:38.152075 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:38.152052    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29afff26-d867-4aaf-9d15-ea2f45d4a091-credential-socket" pod "29afff26-d867-4aaf-9d15-ea2f45d4a091" (UID: "29afff26-d867-4aaf-9d15-ea2f45d4a091"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:53:38.152143 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:38.152053    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/29afff26-d867-4aaf-9d15-ea2f45d4a091-kube-api-access-96rmp\" (UniqueName: \"kubernetes.io/projected/29afff26-d867-4aaf-9d15-ea2f45d4a091-kube-api-access-96rmp\") pod \"29afff26-d867-4aaf-9d15-ea2f45d4a091\" (UID: \"29afff26-d867-4aaf-9d15-ea2f45d4a091\") "
Oct 30 19:53:38.152191 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:38.152153    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29afff26-d867-4aaf-9d15-ea2f45d4a091-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29afff26-d867-4aaf-9d15-ea2f45d4a091-istio-data\") pod \"29afff26-d867-4aaf-9d15-ea2f45d4a091\" (UID: \"29afff26-d867-4aaf-9d15-ea2f45d4a091\") "
Oct 30 19:53:38.152243 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:38.152210    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29afff26-d867-4aaf-9d15-ea2f45d4a091-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29afff26-d867-4aaf-9d15-ea2f45d4a091-istio-envoy\") pod \"29afff26-d867-4aaf-9d15-ea2f45d4a091\" (UID: \"29afff26-d867-4aaf-9d15-ea2f45d4a091\") "
Oct 30 19:53:38.152306 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:38.152287    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/29afff26-d867-4aaf-9d15-ea2f45d4a091-istio-token\" (UniqueName: \"kubernetes.io/projected/29afff26-d867-4aaf-9d15-ea2f45d4a091-istio-token\") pod \"29afff26-d867-4aaf-9d15-ea2f45d4a091\" (UID: \"29afff26-d867-4aaf-9d15-ea2f45d4a091\") "
Oct 30 19:53:38.152370 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:38.152330    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/29afff26-d867-4aaf-9d15-ea2f45d4a091-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29afff26-d867-4aaf-9d15-ea2f45d4a091-istiod-ca-cert\") pod \"29afff26-d867-4aaf-9d15-ea2f45d4a091\" (UID: \"29afff26-d867-4aaf-9d15-ea2f45d4a091\") "
Oct 30 19:53:38.152549 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:38.152524    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29afff26-d867-4aaf-9d15-ea2f45d4a091-workload-socket" pod "29afff26-d867-4aaf-9d15-ea2f45d4a091" (UID: "29afff26-d867-4aaf-9d15-ea2f45d4a091"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:53:38.153006 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:38.152983    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29afff26-d867-4aaf-9d15-ea2f45d4a091-istiod-ca-cert" pod "29afff26-d867-4aaf-9d15-ea2f45d4a091" (UID: "29afff26-d867-4aaf-9d15-ea2f45d4a091"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:53:38.159857 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:38.153529    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29afff26-d867-4aaf-9d15-ea2f45d4a091-istio-data" pod "29afff26-d867-4aaf-9d15-ea2f45d4a091" (UID: "29afff26-d867-4aaf-9d15-ea2f45d4a091"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:53:38.159857 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:38.152475    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29afff26-d867-4aaf-9d15-ea2f45d4a091-workload-certs" pod "29afff26-d867-4aaf-9d15-ea2f45d4a091" (UID: "29afff26-d867-4aaf-9d15-ea2f45d4a091"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:53:38.159857 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:38.156930    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29afff26-d867-4aaf-9d15-ea2f45d4a091-kube-api-access-96rmp" pod "29afff26-d867-4aaf-9d15-ea2f45d4a091" (UID: "29afff26-d867-4aaf-9d15-ea2f45d4a091"). InnerVolumeSpecName "kube-api-access-96rmp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:53:38.159857 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:38.157082    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29afff26-d867-4aaf-9d15-ea2f45d4a091-istio-token" pod "29afff26-d867-4aaf-9d15-ea2f45d4a091" (UID: "29afff26-d867-4aaf-9d15-ea2f45d4a091"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:53:38.159857 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:38.159756    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29afff26-d867-4aaf-9d15-ea2f45d4a091-istio-envoy" pod "29afff26-d867-4aaf-9d15-ea2f45d4a091" (UID: "29afff26-d867-4aaf-9d15-ea2f45d4a091"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:53:38.160517 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:38.160472    6996 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29afff26-d867-4aaf-9d15-ea2f45d4a091-credential-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:53:38.160670 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:38.160521    6996 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29afff26-d867-4aaf-9d15-ea2f45d4a091-workload-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:53:38.160670 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:38.160537    6996 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29afff26-d867-4aaf-9d15-ea2f45d4a091-workload-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:53:38.160670 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:38.160551    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-96rmp\" (UniqueName: \"kubernetes.io/projected/29afff26-d867-4aaf-9d15-ea2f45d4a091-kube-api-access-96rmp\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:53:38.160670 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:38.160567    6996 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29afff26-d867-4aaf-9d15-ea2f45d4a091-istio-data\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:53:38.160670 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:38.160582    6996 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29afff26-d867-4aaf-9d15-ea2f45d4a091-istio-envoy\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:53:38.160670 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:38.160596    6996 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29afff26-d867-4aaf-9d15-ea2f45d4a091-istio-token\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:53:38.160670 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:38.160609    6996 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29afff26-d867-4aaf-9d15-ea2f45d4a091-istiod-ca-cert\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:53:38.164078 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:38.164049    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/29afff26-d867-4aaf-9d15-ea2f45d4a091-istio-podinfo" pod "29afff26-d867-4aaf-9d15-ea2f45d4a091" (UID: "29afff26-d867-4aaf-9d15-ea2f45d4a091"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:53:38.262288 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:38.262255    6996 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29afff26-d867-4aaf-9d15-ea2f45d4a091-istio-podinfo\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:53:38.529553 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:38.521027    6996 generic.go:356] "Generic (PLEG): container finished" podID="29afff26-d867-4aaf-9d15-ea2f45d4a091" containerID="2a02fb5bcf9c579f93759152f2aba99bcca11e186939fd23f9bb8e5e24ddbb5a" exitCode=0
Oct 30 19:53:38.529553 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:38.522177    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f"
Oct 30 19:53:38.529553 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:38.527996    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f" event={"ID":"29afff26-d867-4aaf-9d15-ea2f45d4a091","Type":"ContainerDied","Data":"2a02fb5bcf9c579f93759152f2aba99bcca11e186939fd23f9bb8e5e24ddbb5a"}
Oct 30 19:53:38.529553 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:38.528061    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f" event={"ID":"29afff26-d867-4aaf-9d15-ea2f45d4a091","Type":"ContainerDied","Data":"19d171d9f816b93e9085b2f8a5ccbc2db1b0a154dc2c6eb11ec5a56e11076a89"}
Oct 30 19:53:38.529553 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:38.528090    6996 scope.go:122] "RemoveContainer" containerID="2a02fb5bcf9c579f93759152f2aba99bcca11e186939fd23f9bb8e5e24ddbb5a"
Oct 30 19:53:38.564992 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:38.564896    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f"]
Oct 30 19:53:38.570342 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:38.570315    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-913351-openshift-default-579dcdf795-lv42f"]
Oct 30 19:53:38.571948 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:38.571929    6996 scope.go:122] "RemoveContainer" containerID="2a02fb5bcf9c579f93759152f2aba99bcca11e186939fd23f9bb8e5e24ddbb5a"
Oct 30 19:53:38.572376 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:53:38.572350    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a02fb5bcf9c579f93759152f2aba99bcca11e186939fd23f9bb8e5e24ddbb5a\": container with ID starting with 2a02fb5bcf9c579f93759152f2aba99bcca11e186939fd23f9bb8e5e24ddbb5a not found: ID does not exist" containerID="2a02fb5bcf9c579f93759152f2aba99bcca11e186939fd23f9bb8e5e24ddbb5a"
Oct 30 19:53:38.572497 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:38.572386    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a02fb5bcf9c579f93759152f2aba99bcca11e186939fd23f9bb8e5e24ddbb5a"} err="failed to get container status \"2a02fb5bcf9c579f93759152f2aba99bcca11e186939fd23f9bb8e5e24ddbb5a\": rpc error: code = NotFound desc = could not find container \"2a02fb5bcf9c579f93759152f2aba99bcca11e186939fd23f9bb8e5e24ddbb5a\": container with ID starting with 2a02fb5bcf9c579f93759152f2aba99bcca11e186939fd23f9bb8e5e24ddbb5a not found: ID does not exist"
Oct 30 19:53:38.792816 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:38.792739    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="29afff26-d867-4aaf-9d15-ea2f45d4a091" path="/var/lib/kubelet/pods/29afff26-d867-4aaf-9d15-ea2f45d4a091/volumes"
Oct 30 19:53:44.420497 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:44.420442    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-490cdf-6b4679dff-ktbjd"]
Oct 30 19:53:44.421002 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:44.420849    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-490cdf-6b4679dff-ktbjd" podUID="94ebc6c2-a915-429d-bb47-b4ce1c118084" containerName="maas-api" containerID="cri-o://333d7c66eaf972dc12722c8ef39f85223af5076558009c33f2942265ea63856b" gracePeriod=30
Oct 30 19:53:44.574989 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:44.574955    6996 generic.go:356] "Generic (PLEG): container finished" podID="94ebc6c2-a915-429d-bb47-b4ce1c118084" containerID="333d7c66eaf972dc12722c8ef39f85223af5076558009c33f2942265ea63856b" exitCode=0
Oct 30 19:53:44.575195 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:44.575028    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-490cdf-6b4679dff-ktbjd" event={"ID":"94ebc6c2-a915-429d-bb47-b4ce1c118084","Type":"ContainerDied","Data":"333d7c66eaf972dc12722c8ef39f85223af5076558009c33f2942265ea63856b"}
Oct 30 19:53:44.829633 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:44.829600    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz"]
Oct 30 19:53:44.830639 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:44.830614    6996 cpu_manager.go:414] "containerMap: removing container" podUID="81198f3c-12bb-4356-bc2e-1134a145c971" containerName="istio-proxy"
Oct 30 19:53:44.830639 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:44.830637    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="81198f3c-12bb-4356-bc2e-1134a145c971" containerName="istio-proxy"
Oct 30 19:53:44.830801 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:44.830703    6996 cpu_manager.go:414] "containerMap: removing container" podUID="b01476c8-1aed-4fd1-acb0-6aea531856ca" containerName="maas-api"
Oct 30 19:53:44.830801 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:44.830711    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b01476c8-1aed-4fd1-acb0-6aea531856ca" containerName="maas-api"
Oct 30 19:53:44.830801 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:44.830737    6996 cpu_manager.go:414] "containerMap: removing container" podUID="29afff26-d867-4aaf-9d15-ea2f45d4a091" containerName="istio-proxy"
Oct 30 19:53:44.830801 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:44.830745    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="29afff26-d867-4aaf-9d15-ea2f45d4a091" containerName="istio-proxy"
Oct 30 19:53:44.831024 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:44.830917    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="b01476c8-1aed-4fd1-acb0-6aea531856ca" containerName="maas-api"
Oct 30 19:53:44.831024 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:44.830933    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="29afff26-d867-4aaf-9d15-ea2f45d4a091" containerName="istio-proxy"
Oct 30 19:53:44.831024 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:44.830945    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="81198f3c-12bb-4356-bc2e-1134a145c971" containerName="istio-proxy"
Oct 30 19:53:44.834150 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:44.834116    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz"
Oct 30 19:53:44.835635 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:44.835616    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-490cdf-6b4679dff-ktbjd"
Oct 30 19:53:44.838888 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:44.838805    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-praxis-a-b17e94-openshift-default-dockercfg-lz6js\""
Oct 30 19:53:44.871523 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:44.871470    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz"]
Oct 30 19:53:44.974540 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:44.974506    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/94ebc6c2-a915-429d-bb47-b4ce1c118084-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/94ebc6c2-a915-429d-bb47-b4ce1c118084-maas-api-tls\") pod \"94ebc6c2-a915-429d-bb47-b4ce1c118084\" (UID: \"94ebc6c2-a915-429d-bb47-b4ce1c118084\") "
Oct 30 19:53:44.974736 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:44.974671    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/94ebc6c2-a915-429d-bb47-b4ce1c118084-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/94ebc6c2-a915-429d-bb47-b4ce1c118084-openshift-service-ca\") pod \"94ebc6c2-a915-429d-bb47-b4ce1c118084\" (UID: \"94ebc6c2-a915-429d-bb47-b4ce1c118084\") "
Oct 30 19:53:44.974820 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:44.974741    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/94ebc6c2-a915-429d-bb47-b4ce1c118084-kube-api-access-5c7dn\" (UniqueName: \"kubernetes.io/projected/94ebc6c2-a915-429d-bb47-b4ce1c118084-kube-api-access-5c7dn\") pod \"94ebc6c2-a915-429d-bb47-b4ce1c118084\" (UID: \"94ebc6c2-a915-429d-bb47-b4ce1c118084\") "
Oct 30 19:53:44.974820 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:44.974776    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/94ebc6c2-a915-429d-bb47-b4ce1c118084-metrics-certs\" (UniqueName: \"kubernetes.io/secret/94ebc6c2-a915-429d-bb47-b4ce1c118084-metrics-certs\") pod \"94ebc6c2-a915-429d-bb47-b4ce1c118084\" (UID: \"94ebc6c2-a915-429d-bb47-b4ce1c118084\") "
Oct 30 19:53:44.975073 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:44.975038    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/94ebc6c2-a915-429d-bb47-b4ce1c118084-openshift-service-ca" pod "94ebc6c2-a915-429d-bb47-b4ce1c118084" (UID: "94ebc6c2-a915-429d-bb47-b4ce1c118084"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:53:44.975215 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:44.975190    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a5af397-4eab-444b-bb54-f5f0cc2ab73a-istio-podinfo\") pod \"e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz\" (UID: \"0a5af397-4eab-444b-bb54-f5f0cc2ab73a\") " pod="openshift-ingress/e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz"
Oct 30 19:53:44.975334 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:44.975308    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a5af397-4eab-444b-bb54-f5f0cc2ab73a-workload-certs\") pod \"e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz\" (UID: \"0a5af397-4eab-444b-bb54-f5f0cc2ab73a\") " pod="openshift-ingress/e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz"
Oct 30 19:53:44.975441 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:44.975380    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a5af397-4eab-444b-bb54-f5f0cc2ab73a-istio-data\") pod \"e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz\" (UID: \"0a5af397-4eab-444b-bb54-f5f0cc2ab73a\") " pod="openshift-ingress/e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz"
Oct 30 19:53:44.975570 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:44.975548    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a5af397-4eab-444b-bb54-f5f0cc2ab73a-workload-socket\") pod \"e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz\" (UID: \"0a5af397-4eab-444b-bb54-f5f0cc2ab73a\") " pod="openshift-ingress/e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz"
Oct 30 19:53:44.975820 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:44.975795    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a5af397-4eab-444b-bb54-f5f0cc2ab73a-istio-envoy\") pod \"e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz\" (UID: \"0a5af397-4eab-444b-bb54-f5f0cc2ab73a\") " pod="openshift-ingress/e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz"
Oct 30 19:53:44.976062 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:44.976043    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a5af397-4eab-444b-bb54-f5f0cc2ab73a-istiod-ca-cert\") pod \"e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz\" (UID: \"0a5af397-4eab-444b-bb54-f5f0cc2ab73a\") " pod="openshift-ingress/e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz"
Oct 30 19:53:44.976270 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:44.976251    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a5af397-4eab-444b-bb54-f5f0cc2ab73a-credential-socket\") pod \"e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz\" (UID: \"0a5af397-4eab-444b-bb54-f5f0cc2ab73a\") " pod="openshift-ingress/e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz"
Oct 30 19:53:44.976455 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:44.976439    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a5af397-4eab-444b-bb54-f5f0cc2ab73a-istio-token\") pod \"e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz\" (UID: \"0a5af397-4eab-444b-bb54-f5f0cc2ab73a\") " pod="openshift-ingress/e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz"
Oct 30 19:53:44.976771 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:44.976750    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfkvx\" (UniqueName: \"kubernetes.io/projected/0a5af397-4eab-444b-bb54-f5f0cc2ab73a-kube-api-access-jfkvx\") pod \"e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz\" (UID: \"0a5af397-4eab-444b-bb54-f5f0cc2ab73a\") " pod="openshift-ingress/e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz"
Oct 30 19:53:44.977114 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:44.977095    6996 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/94ebc6c2-a915-429d-bb47-b4ce1c118084-openshift-service-ca\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:53:44.977304 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:44.977277    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/94ebc6c2-a915-429d-bb47-b4ce1c118084-maas-api-tls" pod "94ebc6c2-a915-429d-bb47-b4ce1c118084" (UID: "94ebc6c2-a915-429d-bb47-b4ce1c118084"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:53:44.977555 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:44.977525    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94ebc6c2-a915-429d-bb47-b4ce1c118084-kube-api-access-5c7dn" pod "94ebc6c2-a915-429d-bb47-b4ce1c118084" (UID: "94ebc6c2-a915-429d-bb47-b4ce1c118084"). InnerVolumeSpecName "kube-api-access-5c7dn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:53:44.983801 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:44.983771    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/94ebc6c2-a915-429d-bb47-b4ce1c118084-metrics-certs" pod "94ebc6c2-a915-429d-bb47-b4ce1c118084" (UID: "94ebc6c2-a915-429d-bb47-b4ce1c118084"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:53:45.077987 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:45.077955    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a5af397-4eab-444b-bb54-f5f0cc2ab73a-workload-socket\") pod \"e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz\" (UID: \"0a5af397-4eab-444b-bb54-f5f0cc2ab73a\") " pod="openshift-ingress/e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz"
Oct 30 19:53:45.078149 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:45.078018    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a5af397-4eab-444b-bb54-f5f0cc2ab73a-istio-envoy\") pod \"e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz\" (UID: \"0a5af397-4eab-444b-bb54-f5f0cc2ab73a\") " pod="openshift-ingress/e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz"
Oct 30 19:53:45.078149 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:45.078038    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a5af397-4eab-444b-bb54-f5f0cc2ab73a-istiod-ca-cert\") pod \"e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz\" (UID: \"0a5af397-4eab-444b-bb54-f5f0cc2ab73a\") " pod="openshift-ingress/e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz"
Oct 30 19:53:45.078149 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:45.078070    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a5af397-4eab-444b-bb54-f5f0cc2ab73a-credential-socket\") pod \"e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz\" (UID: \"0a5af397-4eab-444b-bb54-f5f0cc2ab73a\") " pod="openshift-ingress/e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz"
Oct 30 19:53:45.078149 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:45.078100    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a5af397-4eab-444b-bb54-f5f0cc2ab73a-istio-token\") pod \"e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz\" (UID: \"0a5af397-4eab-444b-bb54-f5f0cc2ab73a\") " pod="openshift-ingress/e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz"
Oct 30 19:53:45.078149 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:45.078125    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfkvx\" (UniqueName: \"kubernetes.io/projected/0a5af397-4eab-444b-bb54-f5f0cc2ab73a-kube-api-access-jfkvx\") pod \"e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz\" (UID: \"0a5af397-4eab-444b-bb54-f5f0cc2ab73a\") " pod="openshift-ingress/e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz"
Oct 30 19:53:45.078406 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:45.078225    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a5af397-4eab-444b-bb54-f5f0cc2ab73a-istio-podinfo\") pod \"e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz\" (UID: \"0a5af397-4eab-444b-bb54-f5f0cc2ab73a\") " pod="openshift-ingress/e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz"
Oct 30 19:53:45.078406 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:45.078280    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a5af397-4eab-444b-bb54-f5f0cc2ab73a-workload-certs\") pod \"e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz\" (UID: \"0a5af397-4eab-444b-bb54-f5f0cc2ab73a\") " pod="openshift-ingress/e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz"
Oct 30 19:53:45.078406 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:45.078330    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a5af397-4eab-444b-bb54-f5f0cc2ab73a-istio-data\") pod \"e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz\" (UID: \"0a5af397-4eab-444b-bb54-f5f0cc2ab73a\") " pod="openshift-ingress/e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz"
Oct 30 19:53:45.078406 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:45.078395    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a5af397-4eab-444b-bb54-f5f0cc2ab73a-workload-socket\") pod \"e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz\" (UID: \"0a5af397-4eab-444b-bb54-f5f0cc2ab73a\") " pod="openshift-ingress/e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz"
Oct 30 19:53:45.078652 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:45.078497    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a5af397-4eab-444b-bb54-f5f0cc2ab73a-credential-socket\") pod \"e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz\" (UID: \"0a5af397-4eab-444b-bb54-f5f0cc2ab73a\") " pod="openshift-ingress/e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz"
Oct 30 19:53:45.078652 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:45.078528    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5c7dn\" (UniqueName: \"kubernetes.io/projected/94ebc6c2-a915-429d-bb47-b4ce1c118084-kube-api-access-5c7dn\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:53:45.078652 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:45.078555    6996 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94ebc6c2-a915-429d-bb47-b4ce1c118084-metrics-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:53:45.078652 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:45.078574    6996 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/94ebc6c2-a915-429d-bb47-b4ce1c118084-maas-api-tls\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:53:45.078792 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:45.078713    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a5af397-4eab-444b-bb54-f5f0cc2ab73a-workload-certs\") pod \"e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz\" (UID: \"0a5af397-4eab-444b-bb54-f5f0cc2ab73a\") " pod="openshift-ingress/e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz"
Oct 30 19:53:45.078792 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:45.078744    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a5af397-4eab-444b-bb54-f5f0cc2ab73a-istio-data\") pod \"e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz\" (UID: \"0a5af397-4eab-444b-bb54-f5f0cc2ab73a\") " pod="openshift-ingress/e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz"
Oct 30 19:53:45.078858 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:45.078833    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a5af397-4eab-444b-bb54-f5f0cc2ab73a-istiod-ca-cert\") pod \"e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz\" (UID: \"0a5af397-4eab-444b-bb54-f5f0cc2ab73a\") " pod="openshift-ingress/e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz"
Oct 30 19:53:45.080314 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:45.080288    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a5af397-4eab-444b-bb54-f5f0cc2ab73a-istio-envoy\") pod \"e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz\" (UID: \"0a5af397-4eab-444b-bb54-f5f0cc2ab73a\") " pod="openshift-ingress/e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz"
Oct 30 19:53:45.080691 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:45.080670    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a5af397-4eab-444b-bb54-f5f0cc2ab73a-istio-podinfo\") pod \"e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz\" (UID: \"0a5af397-4eab-444b-bb54-f5f0cc2ab73a\") " pod="openshift-ingress/e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz"
Oct 30 19:53:45.094925 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:45.094847    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfkvx\" (UniqueName: \"kubernetes.io/projected/0a5af397-4eab-444b-bb54-f5f0cc2ab73a-kube-api-access-jfkvx\") pod \"e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz\" (UID: \"0a5af397-4eab-444b-bb54-f5f0cc2ab73a\") " pod="openshift-ingress/e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz"
Oct 30 19:53:45.096845 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:45.096821    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a5af397-4eab-444b-bb54-f5f0cc2ab73a-istio-token\") pod \"e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz\" (UID: \"0a5af397-4eab-444b-bb54-f5f0cc2ab73a\") " pod="openshift-ingress/e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz"
Oct 30 19:53:45.159259 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:45.159226    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz"
Oct 30 19:53:45.450569 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:45.450528    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz"]
Oct 30 19:53:45.451062 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:53:45.451040    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a5af397_4eab_444b_bb54_f5f0cc2ab73a.slice/crio-5c8d8443e36dec043c0de0f51bfffc6c231617d513539031fe93e889158798b8 WatchSource:0}: Error finding container 5c8d8443e36dec043c0de0f51bfffc6c231617d513539031fe93e889158798b8: Status 404 returned error can't find the container with id 5c8d8443e36dec043c0de0f51bfffc6c231617d513539031fe93e889158798b8
Oct 30 19:53:45.453660 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:45.453627    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:53:45.453760 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:45.453690    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:53:45.453760 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:45.453719    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:53:45.586790 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:45.586543    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz" event={"ID":"0a5af397-4eab-444b-bb54-f5f0cc2ab73a","Type":"ContainerStarted","Data":"456b89a23267803472b4c6bd5cb8e187fd6a560a049e0080719f056e9d5fff33"}
Oct 30 19:53:45.586790 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:45.586610    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz" event={"ID":"0a5af397-4eab-444b-bb54-f5f0cc2ab73a","Type":"ContainerStarted","Data":"5c8d8443e36dec043c0de0f51bfffc6c231617d513539031fe93e889158798b8"}
Oct 30 19:53:45.588605 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:45.588573    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-490cdf-6b4679dff-ktbjd"
Oct 30 19:53:45.588605 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:45.588586    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-490cdf-6b4679dff-ktbjd" event={"ID":"94ebc6c2-a915-429d-bb47-b4ce1c118084","Type":"ContainerDied","Data":"0118b07831d62c15e0e5e194f4b85efdf33bc9b97420d0a322b058937a10224a"}
Oct 30 19:53:45.589958 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:45.588642    6996 scope.go:122] "RemoveContainer" containerID="333d7c66eaf972dc12722c8ef39f85223af5076558009c33f2942265ea63856b"
Oct 30 19:53:45.634598 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:45.634544    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz" podStartSLOduration=1.6345277710000001 podStartE2EDuration="1.634527771s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:53:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:53:45.615249913 +0000 UTC m=+1300.943368067" watchObservedRunningTime="2026-10-30 19:53:45.634527771 +0000 UTC m=+1300.962645926"
Oct 30 19:53:45.636195 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:45.636165    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-490cdf-6b4679dff-ktbjd"]
Oct 30 19:53:45.642856 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:45.642830    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-490cdf-6b4679dff-ktbjd"]
Oct 30 19:53:45.977506 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:45.977117    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 19:53:46.163506 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:46.161846    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz"
Oct 30 19:53:46.168087 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:46.167745    6996 patch_prober.go:29] interesting pod/e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.132:15021/healthz/ready\": dial tcp 10.134.0.132:15021: connect: connection refused" start-of-body=
Oct 30 19:53:46.168087 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:46.167805    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz" podUID="0a5af397-4eab-444b-bb54-f5f0cc2ab73a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.132:15021/healthz/ready\": dial tcp 10.134.0.132:15021: connect: connection refused"
Oct 30 19:53:46.826502 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:46.826089    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="94ebc6c2-a915-429d-bb47-b4ce1c118084" path="/var/lib/kubelet/pods/94ebc6c2-a915-429d-bb47-b4ce1c118084/volumes"
Oct 30 19:53:47.167222 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:47.167125    6996 patch_prober.go:29] interesting pod/e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.132:15021/healthz/ready\": dial tcp 10.134.0.132:15021: connect: connection refused" start-of-body=
Oct 30 19:53:47.167521 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:47.167489    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz" podUID="0a5af397-4eab-444b-bb54-f5f0cc2ab73a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.132:15021/healthz/ready\": dial tcp 10.134.0.132:15021: connect: connection refused"
Oct 30 19:53:48.159840 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:48.159802    6996 patch_prober.go:29] interesting pod/e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.132:15021/healthz/ready\": dial tcp 10.134.0.132:15021: connect: connection refused" start-of-body=
Oct 30 19:53:48.160436 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:48.160404    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz" podUID="0a5af397-4eab-444b-bb54-f5f0cc2ab73a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.132:15021/healthz/ready\": dial tcp 10.134.0.132:15021: connect: connection refused"
Oct 30 19:53:49.164896 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:49.164856    6996 patch_prober.go:29] interesting pod/e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.132:15021/healthz/ready\": dial tcp 10.134.0.132:15021: connect: connection refused" start-of-body=
Oct 30 19:53:49.165527 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:49.165465    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz" podUID="0a5af397-4eab-444b-bb54-f5f0cc2ab73a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.132:15021/healthz/ready\": dial tcp 10.134.0.132:15021: connect: connection refused"
Oct 30 19:53:50.160506 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:50.159901    6996 patch_prober.go:29] interesting pod/e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.132:15021/healthz/ready\": dial tcp 10.134.0.132:15021: connect: connection refused" start-of-body=
Oct 30 19:53:50.160506 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:50.159963    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz" podUID="0a5af397-4eab-444b-bb54-f5f0cc2ab73a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.132:15021/healthz/ready\": dial tcp 10.134.0.132:15021: connect: connection refused"
Oct 30 19:53:51.160509 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:51.159813    6996 patch_prober.go:29] interesting pod/e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.132:15021/healthz/ready\": dial tcp 10.134.0.132:15021: connect: connection refused" start-of-body=
Oct 30 19:53:51.160509 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:51.159880    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz" podUID="0a5af397-4eab-444b-bb54-f5f0cc2ab73a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.132:15021/healthz/ready\": dial tcp 10.134.0.132:15021: connect: connection refused"
Oct 30 19:53:52.164152 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:52.164120    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz"
Oct 30 19:53:52.164652 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:52.164558    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz"
Oct 30 19:53:52.165248 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:52.165230    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz"
Oct 30 19:53:55.964606 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:55.964568    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-55a8e6-86d8478fc6-85xvd"]
Oct 30 19:53:55.965652 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:55.965625    6996 cpu_manager.go:414] "containerMap: removing container" podUID="94ebc6c2-a915-429d-bb47-b4ce1c118084" containerName="maas-api"
Oct 30 19:53:55.965652 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:55.965653    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="94ebc6c2-a915-429d-bb47-b4ce1c118084" containerName="maas-api"
Oct 30 19:53:55.965885 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:55.965867    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="94ebc6c2-a915-429d-bb47-b4ce1c118084" containerName="maas-api"
Oct 30 19:53:55.969416 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:55.969393    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-55a8e6-86d8478fc6-85xvd"
Oct 30 19:53:55.981783 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:55.981447    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-55a8e6-serving-cert\""
Oct 30 19:53:55.986226 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:55.985354    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-55a8e6-86d8478fc6-85xvd"]
Oct 30 19:53:56.035975 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:56.035941    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d767ba6-5ac3-4732-b52a-b60c2705d1de-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-55a8e6-86d8478fc6-85xvd\" (UID: \"3d767ba6-5ac3-4732-b52a-b60c2705d1de\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-55a8e6-86d8478fc6-85xvd"
Oct 30 19:53:56.036165 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:56.036022    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d767ba6-5ac3-4732-b52a-b60c2705d1de-maas-api-tls\") pod \"maas-api-e2e-ipp-b-55a8e6-86d8478fc6-85xvd\" (UID: \"3d767ba6-5ac3-4732-b52a-b60c2705d1de\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-55a8e6-86d8478fc6-85xvd"
Oct 30 19:53:56.036303 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:56.036278    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d767ba6-5ac3-4732-b52a-b60c2705d1de-metrics-certs\") pod \"maas-api-e2e-ipp-b-55a8e6-86d8478fc6-85xvd\" (UID: \"3d767ba6-5ac3-4732-b52a-b60c2705d1de\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-55a8e6-86d8478fc6-85xvd"
Oct 30 19:53:56.036524 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:56.036466    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqzt6\" (UniqueName: \"kubernetes.io/projected/3d767ba6-5ac3-4732-b52a-b60c2705d1de-kube-api-access-fqzt6\") pod \"maas-api-e2e-ipp-b-55a8e6-86d8478fc6-85xvd\" (UID: \"3d767ba6-5ac3-4732-b52a-b60c2705d1de\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-55a8e6-86d8478fc6-85xvd"
Oct 30 19:53:56.137955 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:56.137919    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqzt6\" (UniqueName: \"kubernetes.io/projected/3d767ba6-5ac3-4732-b52a-b60c2705d1de-kube-api-access-fqzt6\") pod \"maas-api-e2e-ipp-b-55a8e6-86d8478fc6-85xvd\" (UID: \"3d767ba6-5ac3-4732-b52a-b60c2705d1de\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-55a8e6-86d8478fc6-85xvd"
Oct 30 19:53:56.137955 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:56.137971    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d767ba6-5ac3-4732-b52a-b60c2705d1de-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-55a8e6-86d8478fc6-85xvd\" (UID: \"3d767ba6-5ac3-4732-b52a-b60c2705d1de\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-55a8e6-86d8478fc6-85xvd"
Oct 30 19:53:56.138258 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:56.138006    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d767ba6-5ac3-4732-b52a-b60c2705d1de-maas-api-tls\") pod \"maas-api-e2e-ipp-b-55a8e6-86d8478fc6-85xvd\" (UID: \"3d767ba6-5ac3-4732-b52a-b60c2705d1de\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-55a8e6-86d8478fc6-85xvd"
Oct 30 19:53:56.138258 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:56.138081    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d767ba6-5ac3-4732-b52a-b60c2705d1de-metrics-certs\") pod \"maas-api-e2e-ipp-b-55a8e6-86d8478fc6-85xvd\" (UID: \"3d767ba6-5ac3-4732-b52a-b60c2705d1de\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-55a8e6-86d8478fc6-85xvd"
Oct 30 19:53:56.138706 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:56.138681    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d767ba6-5ac3-4732-b52a-b60c2705d1de-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-55a8e6-86d8478fc6-85xvd\" (UID: \"3d767ba6-5ac3-4732-b52a-b60c2705d1de\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-55a8e6-86d8478fc6-85xvd"
Oct 30 19:53:56.140657 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:56.140634    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d767ba6-5ac3-4732-b52a-b60c2705d1de-metrics-certs\") pod \"maas-api-e2e-ipp-b-55a8e6-86d8478fc6-85xvd\" (UID: \"3d767ba6-5ac3-4732-b52a-b60c2705d1de\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-55a8e6-86d8478fc6-85xvd"
Oct 30 19:53:56.140753 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:56.140658    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d767ba6-5ac3-4732-b52a-b60c2705d1de-maas-api-tls\") pod \"maas-api-e2e-ipp-b-55a8e6-86d8478fc6-85xvd\" (UID: \"3d767ba6-5ac3-4732-b52a-b60c2705d1de\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-55a8e6-86d8478fc6-85xvd"
Oct 30 19:53:56.150066 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:56.150043    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqzt6\" (UniqueName: \"kubernetes.io/projected/3d767ba6-5ac3-4732-b52a-b60c2705d1de-kube-api-access-fqzt6\") pod \"maas-api-e2e-ipp-b-55a8e6-86d8478fc6-85xvd\" (UID: \"3d767ba6-5ac3-4732-b52a-b60c2705d1de\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-55a8e6-86d8478fc6-85xvd"
Oct 30 19:53:56.291494 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:56.291400    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-55a8e6-86d8478fc6-85xvd"
Oct 30 19:53:56.579449 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:56.579422    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-55a8e6-86d8478fc6-85xvd"]
Oct 30 19:53:56.581139 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:53:56.581113    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d767ba6_5ac3_4732_b52a_b60c2705d1de.slice/crio-516d6c3100094cc3e5b55038f668889789250c3b651989c6118692d5dec5318c WatchSource:0}: Error finding container 516d6c3100094cc3e5b55038f668889789250c3b651989c6118692d5dec5318c: Status 404 returned error can't find the container with id 516d6c3100094cc3e5b55038f668889789250c3b651989c6118692d5dec5318c
Oct 30 19:53:56.774198 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:56.774161    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-55a8e6-86d8478fc6-85xvd" event={"ID":"3d767ba6-5ac3-4732-b52a-b60c2705d1de","Type":"ContainerStarted","Data":"516d6c3100094cc3e5b55038f668889789250c3b651989c6118692d5dec5318c"}
Oct 30 19:53:57.783562 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:57.783519    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-55a8e6-86d8478fc6-85xvd" event={"ID":"3d767ba6-5ac3-4732-b52a-b60c2705d1de","Type":"ContainerStarted","Data":"bde16c724b98aff03145e2c2ed0dc9f3f7fad0512fbe0e099831548cbc991344"}
Oct 30 19:53:57.784095 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:57.783591    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-55a8e6-86d8478fc6-85xvd"
Oct 30 19:53:57.810752 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:53:57.810698    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-55a8e6-86d8478fc6-85xvd" podStartSLOduration=2.377583704 podStartE2EDuration="2.810679624s" totalImagesPullingTime="433.09592ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:53:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:53:57.807190343 +0000 UTC m=+1313.135308496" watchObservedRunningTime="2026-10-30 19:53:57.810679624 +0000 UTC m=+1313.138797781"
Oct 30 19:54:01.198190 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:54:01.198134    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 19:54:03.801682 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:03.801649    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-55a8e6-86d8478fc6-85xvd"
Oct 30 19:54:26.235785 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:26.235750    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj"]
Oct 30 19:54:26.236247 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:26.236011    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj" podUID="f5d54632-af1e-458b-b431-61ead4fccb41" containerName="istio-proxy" containerID="cri-o://35b5f8862e8075ad5c020ce3c69432a20c085ba9d3072314b0eee58db437d97d" gracePeriod=30
Oct 30 19:54:27.938831 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:27.938787    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 19:54:31.202016 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:54:31.201969    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 19:54:31.567969 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:31.567942    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj"
Oct 30 19:54:31.639907 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:31.639874    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f5d54632-af1e-458b-b431-61ead4fccb41-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5d54632-af1e-458b-b431-61ead4fccb41-workload-socket\") pod \"f5d54632-af1e-458b-b431-61ead4fccb41\" (UID: \"f5d54632-af1e-458b-b431-61ead4fccb41\") "
Oct 30 19:54:31.640100 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:31.640011    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f5d54632-af1e-458b-b431-61ead4fccb41-istio-token\" (UniqueName: \"kubernetes.io/projected/f5d54632-af1e-458b-b431-61ead4fccb41-istio-token\") pod \"f5d54632-af1e-458b-b431-61ead4fccb41\" (UID: \"f5d54632-af1e-458b-b431-61ead4fccb41\") "
Oct 30 19:54:31.640100 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:31.640063    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f5d54632-af1e-458b-b431-61ead4fccb41-kube-api-access-kxmsr\" (UniqueName: \"kubernetes.io/projected/f5d54632-af1e-458b-b431-61ead4fccb41-kube-api-access-kxmsr\") pod \"f5d54632-af1e-458b-b431-61ead4fccb41\" (UID: \"f5d54632-af1e-458b-b431-61ead4fccb41\") "
Oct 30 19:54:31.640237 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:31.640175    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f5d54632-af1e-458b-b431-61ead4fccb41-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5d54632-af1e-458b-b431-61ead4fccb41-credential-socket\") pod \"f5d54632-af1e-458b-b431-61ead4fccb41\" (UID: \"f5d54632-af1e-458b-b431-61ead4fccb41\") "
Oct 30 19:54:31.640237 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:31.640216    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f5d54632-af1e-458b-b431-61ead4fccb41-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5d54632-af1e-458b-b431-61ead4fccb41-istio-podinfo\") pod \"f5d54632-af1e-458b-b431-61ead4fccb41\" (UID: \"f5d54632-af1e-458b-b431-61ead4fccb41\") "
Oct 30 19:54:31.640347 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:31.640288    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f5d54632-af1e-458b-b431-61ead4fccb41-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5d54632-af1e-458b-b431-61ead4fccb41-istio-envoy\") pod \"f5d54632-af1e-458b-b431-61ead4fccb41\" (UID: \"f5d54632-af1e-458b-b431-61ead4fccb41\") "
Oct 30 19:54:31.640347 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:31.640338    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f5d54632-af1e-458b-b431-61ead4fccb41-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5d54632-af1e-458b-b431-61ead4fccb41-istiod-ca-cert\") pod \"f5d54632-af1e-458b-b431-61ead4fccb41\" (UID: \"f5d54632-af1e-458b-b431-61ead4fccb41\") "
Oct 30 19:54:31.640458 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:31.640389    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f5d54632-af1e-458b-b431-61ead4fccb41-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5d54632-af1e-458b-b431-61ead4fccb41-istio-data\") pod \"f5d54632-af1e-458b-b431-61ead4fccb41\" (UID: \"f5d54632-af1e-458b-b431-61ead4fccb41\") "
Oct 30 19:54:31.640458 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:31.640427    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f5d54632-af1e-458b-b431-61ead4fccb41-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5d54632-af1e-458b-b431-61ead4fccb41-workload-certs\") pod \"f5d54632-af1e-458b-b431-61ead4fccb41\" (UID: \"f5d54632-af1e-458b-b431-61ead4fccb41\") "
Oct 30 19:54:31.640915 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:31.640890    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5d54632-af1e-458b-b431-61ead4fccb41-credential-socket" pod "f5d54632-af1e-458b-b431-61ead4fccb41" (UID: "f5d54632-af1e-458b-b431-61ead4fccb41"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:54:31.641224 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:31.641203    6996 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5d54632-af1e-458b-b431-61ead4fccb41-credential-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:54:31.641560 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:31.641539    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5d54632-af1e-458b-b431-61ead4fccb41-istio-data" pod "f5d54632-af1e-458b-b431-61ead4fccb41" (UID: "f5d54632-af1e-458b-b431-61ead4fccb41"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:54:31.641764 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:31.641741    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5d54632-af1e-458b-b431-61ead4fccb41-workload-socket" pod "f5d54632-af1e-458b-b431-61ead4fccb41" (UID: "f5d54632-af1e-458b-b431-61ead4fccb41"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:54:31.641904 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:31.641886    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5d54632-af1e-458b-b431-61ead4fccb41-istiod-ca-cert" pod "f5d54632-af1e-458b-b431-61ead4fccb41" (UID: "f5d54632-af1e-458b-b431-61ead4fccb41"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:54:31.642174 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:31.642126    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5d54632-af1e-458b-b431-61ead4fccb41-workload-certs" pod "f5d54632-af1e-458b-b431-61ead4fccb41" (UID: "f5d54632-af1e-458b-b431-61ead4fccb41"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:54:31.644229 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:31.644204    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f5d54632-af1e-458b-b431-61ead4fccb41-istio-podinfo" pod "f5d54632-af1e-458b-b431-61ead4fccb41" (UID: "f5d54632-af1e-458b-b431-61ead4fccb41"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:54:31.644563 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:31.644543    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5d54632-af1e-458b-b431-61ead4fccb41-istio-token" pod "f5d54632-af1e-458b-b431-61ead4fccb41" (UID: "f5d54632-af1e-458b-b431-61ead4fccb41"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:54:31.644726 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:31.644709    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5d54632-af1e-458b-b431-61ead4fccb41-kube-api-access-kxmsr" pod "f5d54632-af1e-458b-b431-61ead4fccb41" (UID: "f5d54632-af1e-458b-b431-61ead4fccb41"). InnerVolumeSpecName "kube-api-access-kxmsr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:54:31.649218 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:31.649178    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5d54632-af1e-458b-b431-61ead4fccb41-istio-envoy" pod "f5d54632-af1e-458b-b431-61ead4fccb41" (UID: "f5d54632-af1e-458b-b431-61ead4fccb41"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:54:31.742811 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:31.742780    6996 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5d54632-af1e-458b-b431-61ead4fccb41-istio-token\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:54:31.742811 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:31.742809    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kxmsr\" (UniqueName: \"kubernetes.io/projected/f5d54632-af1e-458b-b431-61ead4fccb41-kube-api-access-kxmsr\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:54:31.742994 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:31.742820    6996 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5d54632-af1e-458b-b431-61ead4fccb41-istio-podinfo\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:54:31.742994 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:31.742830    6996 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5d54632-af1e-458b-b431-61ead4fccb41-istio-envoy\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:54:31.742994 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:31.742838    6996 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5d54632-af1e-458b-b431-61ead4fccb41-istiod-ca-cert\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:54:31.742994 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:31.742846    6996 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5d54632-af1e-458b-b431-61ead4fccb41-istio-data\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:54:31.742994 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:31.742856    6996 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5d54632-af1e-458b-b431-61ead4fccb41-workload-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:54:31.742994 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:31.742865    6996 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5d54632-af1e-458b-b431-61ead4fccb41-workload-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:54:32.133264 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:32.133229    6996 generic.go:356] "Generic (PLEG): container finished" podID="f5d54632-af1e-458b-b431-61ead4fccb41" containerID="35b5f8862e8075ad5c020ce3c69432a20c085ba9d3072314b0eee58db437d97d" exitCode=0
Oct 30 19:54:32.133453 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:32.133283    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj"
Oct 30 19:54:32.133453 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:32.133304    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj" event={"ID":"f5d54632-af1e-458b-b431-61ead4fccb41","Type":"ContainerDied","Data":"35b5f8862e8075ad5c020ce3c69432a20c085ba9d3072314b0eee58db437d97d"}
Oct 30 19:54:32.133453 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:32.133354    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj" event={"ID":"f5d54632-af1e-458b-b431-61ead4fccb41","Type":"ContainerDied","Data":"81ec55f46530eff6fab813e59cf519e3d9368c0b95bdf233d0ace212d2480a77"}
Oct 30 19:54:32.133453 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:32.133376    6996 scope.go:122] "RemoveContainer" containerID="35b5f8862e8075ad5c020ce3c69432a20c085ba9d3072314b0eee58db437d97d"
Oct 30 19:54:32.146194 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:32.146175    6996 scope.go:122] "RemoveContainer" containerID="ae4a4d02ebdfb0c0f0f1f0971c0758bec522e491f62c1dd1f5cbe8fc088f0075"
Oct 30 19:54:32.156726 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:32.156689    6996 scope.go:122] "RemoveContainer" containerID="35b5f8862e8075ad5c020ce3c69432a20c085ba9d3072314b0eee58db437d97d"
Oct 30 19:54:32.157802 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:54:32.157260    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35b5f8862e8075ad5c020ce3c69432a20c085ba9d3072314b0eee58db437d97d\": container with ID starting with 35b5f8862e8075ad5c020ce3c69432a20c085ba9d3072314b0eee58db437d97d not found: ID does not exist" containerID="35b5f8862e8075ad5c020ce3c69432a20c085ba9d3072314b0eee58db437d97d"
Oct 30 19:54:32.157802 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:32.157309    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35b5f8862e8075ad5c020ce3c69432a20c085ba9d3072314b0eee58db437d97d"} err="failed to get container status \"35b5f8862e8075ad5c020ce3c69432a20c085ba9d3072314b0eee58db437d97d\": rpc error: code = NotFound desc = could not find container \"35b5f8862e8075ad5c020ce3c69432a20c085ba9d3072314b0eee58db437d97d\": container with ID starting with 35b5f8862e8075ad5c020ce3c69432a20c085ba9d3072314b0eee58db437d97d not found: ID does not exist"
Oct 30 19:54:32.157802 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:32.157334    6996 scope.go:122] "RemoveContainer" containerID="ae4a4d02ebdfb0c0f0f1f0971c0758bec522e491f62c1dd1f5cbe8fc088f0075"
Oct 30 19:54:32.157802 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:54:32.157724    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae4a4d02ebdfb0c0f0f1f0971c0758bec522e491f62c1dd1f5cbe8fc088f0075\": container with ID starting with ae4a4d02ebdfb0c0f0f1f0971c0758bec522e491f62c1dd1f5cbe8fc088f0075 not found: ID does not exist" containerID="ae4a4d02ebdfb0c0f0f1f0971c0758bec522e491f62c1dd1f5cbe8fc088f0075"
Oct 30 19:54:32.157802 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:32.157757    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae4a4d02ebdfb0c0f0f1f0971c0758bec522e491f62c1dd1f5cbe8fc088f0075"} err="failed to get container status \"ae4a4d02ebdfb0c0f0f1f0971c0758bec522e491f62c1dd1f5cbe8fc088f0075\": rpc error: code = NotFound desc = could not find container \"ae4a4d02ebdfb0c0f0f1f0971c0758bec522e491f62c1dd1f5cbe8fc088f0075\": container with ID starting with ae4a4d02ebdfb0c0f0f1f0971c0758bec522e491f62c1dd1f5cbe8fc088f0075 not found: ID does not exist"
Oct 30 19:54:32.160075 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:32.160052    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj"]
Oct 30 19:54:32.163398 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:32.163377    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-praxis-b-ee4af9-openshift-default-597677545c-6mjvj"]
Oct 30 19:54:32.793081 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:32.793049    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f5d54632-af1e-458b-b431-61ead4fccb41" path="/var/lib/kubelet/pods/f5d54632-af1e-458b-b431-61ead4fccb41/volumes"
Oct 30 19:54:34.075527 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:34.075475    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sbbjg"]
Oct 30 19:54:34.075918 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:34.075700    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sbbjg" podUID="81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be" containerName="istio-proxy" containerID="cri-o://cb4f913e0a9b69bddb3f83fc3d35534e8147f297257953c60d3b837e919b30e4" gracePeriod=30
Oct 30 19:54:39.201367 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:39.201330    6996 generic.go:356] "Generic (PLEG): container finished" podID="81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be" containerID="cb4f913e0a9b69bddb3f83fc3d35534e8147f297257953c60d3b837e919b30e4" exitCode=0
Oct 30 19:54:39.201736 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:39.201396    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sbbjg" event={"ID":"81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be","Type":"ContainerDied","Data":"cb4f913e0a9b69bddb3f83fc3d35534e8147f297257953c60d3b837e919b30e4"}
Oct 30 19:54:39.348933 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:39.348856    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sbbjg"
Oct 30 19:54:39.473660 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:39.473627    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be-kube-api-access-pvp96\" (UniqueName: \"kubernetes.io/projected/81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be-kube-api-access-pvp96\") pod \"81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be\" (UID: \"81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be\") "
Oct 30 19:54:39.473818 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:39.473738    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be-credential-socket\") pod \"81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be\" (UID: \"81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be\") "
Oct 30 19:54:39.473902 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:39.473843    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be-workload-socket\") pod \"81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be\" (UID: \"81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be\") "
Oct 30 19:54:39.473974 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:39.473906    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be-istio-data\") pod \"81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be\" (UID: \"81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be\") "
Oct 30 19:54:39.474092 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:39.474063    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be-istio-podinfo\") pod \"81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be\" (UID: \"81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be\") "
Oct 30 19:54:39.474092 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:39.474074    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be-credential-socket" pod "81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be" (UID: "81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:54:39.474287 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:39.474145    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be-istio-envoy\") pod \"81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be\" (UID: \"81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be\") "
Oct 30 19:54:39.474287 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:39.474171    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be-istiod-ca-cert\") pod \"81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be\" (UID: \"81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be\") "
Oct 30 19:54:39.474287 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:39.474238    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be-workload-socket" pod "81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be" (UID: "81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:54:39.474459 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:39.474305    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be-workload-certs\") pod \"81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be\" (UID: \"81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be\") "
Oct 30 19:54:39.474459 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:39.474338    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be-istio-data" pod "81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be" (UID: "81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:54:39.474459 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:39.474360    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be-istio-token\" (UniqueName: \"kubernetes.io/projected/81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be-istio-token\") pod \"81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be\" (UID: \"81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be\") "
Oct 30 19:54:39.474652 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:39.474570    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be-workload-certs" pod "81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be" (UID: "81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:54:39.474709 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:39.474657    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be-istiod-ca-cert" pod "81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be" (UID: "81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:54:39.475111 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:39.475089    6996 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be-istiod-ca-cert\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:54:39.475254 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:39.475114    6996 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be-workload-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:54:39.475254 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:39.475128    6996 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be-credential-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:54:39.475254 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:39.475140    6996 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be-workload-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:54:39.475254 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:39.475154    6996 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be-istio-data\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:54:39.476394 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:39.476370    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be-istio-podinfo" pod "81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be" (UID: "81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:54:39.476507 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:39.476457    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be-kube-api-access-pvp96" pod "81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be" (UID: "81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be"). InnerVolumeSpecName "kube-api-access-pvp96". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:54:39.476567 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:39.476538    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be-istio-token" pod "81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be" (UID: "81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:54:39.476865 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:39.476841    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be-istio-envoy" pod "81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be" (UID: "81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:54:39.576774 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:39.576732    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pvp96\" (UniqueName: \"kubernetes.io/projected/81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be-kube-api-access-pvp96\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:54:39.576774 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:39.576769    6996 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be-istio-podinfo\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:54:39.576979 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:39.576784    6996 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be-istio-envoy\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:54:39.576979 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:39.576796    6996 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be-istio-token\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:54:40.209877 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:40.209843    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sbbjg"
Oct 30 19:54:40.210346 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:40.209882    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sbbjg" event={"ID":"81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be","Type":"ContainerDied","Data":"c69c5d4cf2d2aeb52fd786928a7168a7622ddc43f9ebcb2c7591abda4f95bd8e"}
Oct 30 19:54:40.210346 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:40.209957    6996 scope.go:122] "RemoveContainer" containerID="cb4f913e0a9b69bddb3f83fc3d35534e8147f297257953c60d3b837e919b30e4"
Oct 30 19:54:40.238440 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:40.238411    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sbbjg"]
Oct 30 19:54:40.250058 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:40.250031    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-sbbjg"]
Oct 30 19:54:40.793008 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:40.792977    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be" path="/var/lib/kubelet/pods/81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be/volumes"
Oct 30 19:54:53.526052 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:53.525956    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2"]
Oct 30 19:54:53.526537 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:53.526198    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2" podUID="387db3d3-e192-44ca-b22b-e84f74c32a98" containerName="istio-proxy" containerID="cri-o://a6d46a3b14eae6336de683d86c01a3dd0b0362ecbb1f9ba41fa832f91a513f16" gracePeriod=30
Oct 30 19:54:55.809120 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:55.809076    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 19:54:59.546426 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:59.546389    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2"
Oct 30 19:54:59.550418 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:59.550384    6996 generic.go:356] "Generic (PLEG): container finished" podID="387db3d3-e192-44ca-b22b-e84f74c32a98" containerID="a6d46a3b14eae6336de683d86c01a3dd0b0362ecbb1f9ba41fa832f91a513f16" exitCode=0
Oct 30 19:54:59.550771 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:59.550619    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2" event={"ID":"387db3d3-e192-44ca-b22b-e84f74c32a98","Type":"ContainerDied","Data":"a6d46a3b14eae6336de683d86c01a3dd0b0362ecbb1f9ba41fa832f91a513f16"}
Oct 30 19:54:59.550957 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:59.550940    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2" event={"ID":"387db3d3-e192-44ca-b22b-e84f74c32a98","Type":"ContainerDied","Data":"7d87b1070409a16895469ff2b7cafb1456641100a457cd03db1b4a27a1f52580"}
Oct 30 19:54:59.551122 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:59.551063    6996 scope.go:122] "RemoveContainer" containerID="a6d46a3b14eae6336de683d86c01a3dd0b0362ecbb1f9ba41fa832f91a513f16"
Oct 30 19:54:59.575700 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:59.574187    6996 scope.go:122] "RemoveContainer" containerID="6c17d0200b74861a4493ca95d965286cc1240d19fcd752d0b07c24015c4442a3"
Oct 30 19:54:59.635825 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:59.635412    6996 scope.go:122] "RemoveContainer" containerID="a6d46a3b14eae6336de683d86c01a3dd0b0362ecbb1f9ba41fa832f91a513f16"
Oct 30 19:54:59.644886 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:59.636881    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/387db3d3-e192-44ca-b22b-e84f74c32a98-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/387db3d3-e192-44ca-b22b-e84f74c32a98-workload-socket\") pod \"387db3d3-e192-44ca-b22b-e84f74c32a98\" (UID: \"387db3d3-e192-44ca-b22b-e84f74c32a98\") "
Oct 30 19:54:59.644886 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:59.636957    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/387db3d3-e192-44ca-b22b-e84f74c32a98-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/387db3d3-e192-44ca-b22b-e84f74c32a98-istio-data\") pod \"387db3d3-e192-44ca-b22b-e84f74c32a98\" (UID: \"387db3d3-e192-44ca-b22b-e84f74c32a98\") "
Oct 30 19:54:59.644886 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:59.636991    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/387db3d3-e192-44ca-b22b-e84f74c32a98-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/387db3d3-e192-44ca-b22b-e84f74c32a98-workload-certs\") pod \"387db3d3-e192-44ca-b22b-e84f74c32a98\" (UID: \"387db3d3-e192-44ca-b22b-e84f74c32a98\") "
Oct 30 19:54:59.644886 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:59.637040    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/387db3d3-e192-44ca-b22b-e84f74c32a98-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/387db3d3-e192-44ca-b22b-e84f74c32a98-credential-socket\") pod \"387db3d3-e192-44ca-b22b-e84f74c32a98\" (UID: \"387db3d3-e192-44ca-b22b-e84f74c32a98\") "
Oct 30 19:54:59.644886 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:59.637078    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/387db3d3-e192-44ca-b22b-e84f74c32a98-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/387db3d3-e192-44ca-b22b-e84f74c32a98-istio-envoy\") pod \"387db3d3-e192-44ca-b22b-e84f74c32a98\" (UID: \"387db3d3-e192-44ca-b22b-e84f74c32a98\") "
Oct 30 19:54:59.644886 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:59.637183    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/387db3d3-e192-44ca-b22b-e84f74c32a98-kube-api-access-j5g7h\" (UniqueName: \"kubernetes.io/projected/387db3d3-e192-44ca-b22b-e84f74c32a98-kube-api-access-j5g7h\") pod \"387db3d3-e192-44ca-b22b-e84f74c32a98\" (UID: \"387db3d3-e192-44ca-b22b-e84f74c32a98\") "
Oct 30 19:54:59.644886 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:59.637255    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/387db3d3-e192-44ca-b22b-e84f74c32a98-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/387db3d3-e192-44ca-b22b-e84f74c32a98-istio-podinfo\") pod \"387db3d3-e192-44ca-b22b-e84f74c32a98\" (UID: \"387db3d3-e192-44ca-b22b-e84f74c32a98\") "
Oct 30 19:54:59.644886 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:59.637359    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/387db3d3-e192-44ca-b22b-e84f74c32a98-istio-token\" (UniqueName: \"kubernetes.io/projected/387db3d3-e192-44ca-b22b-e84f74c32a98-istio-token\") pod \"387db3d3-e192-44ca-b22b-e84f74c32a98\" (UID: \"387db3d3-e192-44ca-b22b-e84f74c32a98\") "
Oct 30 19:54:59.644886 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:59.637417    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/387db3d3-e192-44ca-b22b-e84f74c32a98-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/387db3d3-e192-44ca-b22b-e84f74c32a98-istiod-ca-cert\") pod \"387db3d3-e192-44ca-b22b-e84f74c32a98\" (UID: \"387db3d3-e192-44ca-b22b-e84f74c32a98\") "
Oct 30 19:54:59.644886 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:59.641224    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/387db3d3-e192-44ca-b22b-e84f74c32a98-istio-data" pod "387db3d3-e192-44ca-b22b-e84f74c32a98" (UID: "387db3d3-e192-44ca-b22b-e84f74c32a98"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:54:59.644886 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:59.641314    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/387db3d3-e192-44ca-b22b-e84f74c32a98-workload-socket" pod "387db3d3-e192-44ca-b22b-e84f74c32a98" (UID: "387db3d3-e192-44ca-b22b-e84f74c32a98"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:54:59.644886 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:59.641546    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/387db3d3-e192-44ca-b22b-e84f74c32a98-credential-socket" pod "387db3d3-e192-44ca-b22b-e84f74c32a98" (UID: "387db3d3-e192-44ca-b22b-e84f74c32a98"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:54:59.644886 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:59.641732    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/387db3d3-e192-44ca-b22b-e84f74c32a98-workload-certs" pod "387db3d3-e192-44ca-b22b-e84f74c32a98" (UID: "387db3d3-e192-44ca-b22b-e84f74c32a98"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:54:59.644886 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:54:59.644105    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6d46a3b14eae6336de683d86c01a3dd0b0362ecbb1f9ba41fa832f91a513f16\": container with ID starting with a6d46a3b14eae6336de683d86c01a3dd0b0362ecbb1f9ba41fa832f91a513f16 not found: ID does not exist" containerID="a6d46a3b14eae6336de683d86c01a3dd0b0362ecbb1f9ba41fa832f91a513f16"
Oct 30 19:54:59.644886 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:59.644145    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6d46a3b14eae6336de683d86c01a3dd0b0362ecbb1f9ba41fa832f91a513f16"} err="failed to get container status \"a6d46a3b14eae6336de683d86c01a3dd0b0362ecbb1f9ba41fa832f91a513f16\": rpc error: code = NotFound desc = could not find container \"a6d46a3b14eae6336de683d86c01a3dd0b0362ecbb1f9ba41fa832f91a513f16\": container with ID starting with a6d46a3b14eae6336de683d86c01a3dd0b0362ecbb1f9ba41fa832f91a513f16 not found: ID does not exist"
Oct 30 19:54:59.644886 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:59.644174    6996 scope.go:122] "RemoveContainer" containerID="6c17d0200b74861a4493ca95d965286cc1240d19fcd752d0b07c24015c4442a3"
Oct 30 19:54:59.644886 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:59.644566    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/387db3d3-e192-44ca-b22b-e84f74c32a98-istiod-ca-cert" pod "387db3d3-e192-44ca-b22b-e84f74c32a98" (UID: "387db3d3-e192-44ca-b22b-e84f74c32a98"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:54:59.646519 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:54:59.646382    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c17d0200b74861a4493ca95d965286cc1240d19fcd752d0b07c24015c4442a3\": container with ID starting with 6c17d0200b74861a4493ca95d965286cc1240d19fcd752d0b07c24015c4442a3 not found: ID does not exist" containerID="6c17d0200b74861a4493ca95d965286cc1240d19fcd752d0b07c24015c4442a3"
Oct 30 19:54:59.646519 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:59.646420    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c17d0200b74861a4493ca95d965286cc1240d19fcd752d0b07c24015c4442a3"} err="failed to get container status \"6c17d0200b74861a4493ca95d965286cc1240d19fcd752d0b07c24015c4442a3\": rpc error: code = NotFound desc = could not find container \"6c17d0200b74861a4493ca95d965286cc1240d19fcd752d0b07c24015c4442a3\": container with ID starting with 6c17d0200b74861a4493ca95d965286cc1240d19fcd752d0b07c24015c4442a3 not found: ID does not exist"
Oct 30 19:54:59.649623 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:59.649590    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/387db3d3-e192-44ca-b22b-e84f74c32a98-istio-podinfo" pod "387db3d3-e192-44ca-b22b-e84f74c32a98" (UID: "387db3d3-e192-44ca-b22b-e84f74c32a98"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:54:59.649745 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:59.649701    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/387db3d3-e192-44ca-b22b-e84f74c32a98-istio-envoy" pod "387db3d3-e192-44ca-b22b-e84f74c32a98" (UID: "387db3d3-e192-44ca-b22b-e84f74c32a98"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:54:59.650219 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:59.650184    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/387db3d3-e192-44ca-b22b-e84f74c32a98-istio-token" pod "387db3d3-e192-44ca-b22b-e84f74c32a98" (UID: "387db3d3-e192-44ca-b22b-e84f74c32a98"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:54:59.653766 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:59.653735    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/387db3d3-e192-44ca-b22b-e84f74c32a98-kube-api-access-j5g7h" pod "387db3d3-e192-44ca-b22b-e84f74c32a98" (UID: "387db3d3-e192-44ca-b22b-e84f74c32a98"). InnerVolumeSpecName "kube-api-access-j5g7h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:54:59.739257 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:59.739220    6996 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/387db3d3-e192-44ca-b22b-e84f74c32a98-istio-podinfo\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:54:59.739257 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:59.739255    6996 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/387db3d3-e192-44ca-b22b-e84f74c32a98-istio-token\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:54:59.739470 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:59.739266    6996 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/387db3d3-e192-44ca-b22b-e84f74c32a98-istiod-ca-cert\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:54:59.739470 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:59.739276    6996 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/387db3d3-e192-44ca-b22b-e84f74c32a98-workload-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:54:59.739470 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:59.739285    6996 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/387db3d3-e192-44ca-b22b-e84f74c32a98-istio-data\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:54:59.739470 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:59.739293    6996 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/387db3d3-e192-44ca-b22b-e84f74c32a98-workload-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:54:59.739470 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:59.739302    6996 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/387db3d3-e192-44ca-b22b-e84f74c32a98-credential-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:54:59.739470 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:59.739310    6996 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/387db3d3-e192-44ca-b22b-e84f74c32a98-istio-envoy\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:54:59.739470 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:54:59.739318    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j5g7h\" (UniqueName: \"kubernetes.io/projected/387db3d3-e192-44ca-b22b-e84f74c32a98-kube-api-access-j5g7h\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:55:00.560604 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:00.560568    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2"
Oct 30 19:55:00.593449 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:00.593415    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2"]
Oct 30 19:55:00.603117 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:00.603079    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6x4v2"]
Oct 30 19:55:00.794460 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:00.794429    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="387db3d3-e192-44ca-b22b-e84f74c32a98" path="/var/lib/kubelet/pods/387db3d3-e192-44ca-b22b-e84f74c32a98/volumes"
Oct 30 19:55:01.193888 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:55:01.193853    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 19:55:07.447172 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:07.447133    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-a29f33-5c99ccdc58-q7xcz"]
Oct 30 19:55:07.447997 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:07.447973    6996 cpu_manager.go:414] "containerMap: removing container" podUID="f5d54632-af1e-458b-b431-61ead4fccb41" containerName="istio-proxy"
Oct 30 19:55:07.448112 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:07.447999    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f5d54632-af1e-458b-b431-61ead4fccb41" containerName="istio-proxy"
Oct 30 19:55:07.448112 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:07.448016    6996 cpu_manager.go:414] "containerMap: removing container" podUID="387db3d3-e192-44ca-b22b-e84f74c32a98" containerName="istio-proxy"
Oct 30 19:55:07.448112 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:07.448024    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="387db3d3-e192-44ca-b22b-e84f74c32a98" containerName="istio-proxy"
Oct 30 19:55:07.448112 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:07.448072    6996 cpu_manager.go:414] "containerMap: removing container" podUID="81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be" containerName="istio-proxy"
Oct 30 19:55:07.448112 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:07.448080    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be" containerName="istio-proxy"
Oct 30 19:55:07.448112 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:07.448094    6996 cpu_manager.go:414] "containerMap: removing container" podUID="387db3d3-e192-44ca-b22b-e84f74c32a98" containerName="istio-proxy"
Oct 30 19:55:07.448112 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:07.448101    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="387db3d3-e192-44ca-b22b-e84f74c32a98" containerName="istio-proxy"
Oct 30 19:55:07.448448 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:07.448116    6996 cpu_manager.go:414] "containerMap: removing container" podUID="f5d54632-af1e-458b-b431-61ead4fccb41" containerName="istio-proxy"
Oct 30 19:55:07.448448 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:07.448123    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f5d54632-af1e-458b-b431-61ead4fccb41" containerName="istio-proxy"
Oct 30 19:55:07.448448 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:07.448308    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="81b16f1b-7e58-4ec5-a7d2-d2ed7e43e5be" containerName="istio-proxy"
Oct 30 19:55:07.448448 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:07.448324    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="387db3d3-e192-44ca-b22b-e84f74c32a98" containerName="istio-proxy"
Oct 30 19:55:07.448448 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:07.448334    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="f5d54632-af1e-458b-b431-61ead4fccb41" containerName="istio-proxy"
Oct 30 19:55:07.448448 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:07.448354    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="387db3d3-e192-44ca-b22b-e84f74c32a98" containerName="istio-proxy"
Oct 30 19:55:07.453783 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:07.453762    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-a29f33-5c99ccdc58-q7xcz"
Oct 30 19:55:07.463214 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:07.462863    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-praxis-b-a29f33-serving-cert\""
Oct 30 19:55:07.468953 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:07.468925    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-a29f33-5c99ccdc58-q7xcz"]
Oct 30 19:55:07.593553 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:07.593512    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/94bfd457-07b2-4c4e-85e1-1021b5201c82-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-a29f33-5c99ccdc58-q7xcz\" (UID: \"94bfd457-07b2-4c4e-85e1-1021b5201c82\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-a29f33-5c99ccdc58-q7xcz"
Oct 30 19:55:07.593748 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:07.593643    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/94bfd457-07b2-4c4e-85e1-1021b5201c82-maas-api-tls\") pod \"maas-api-e2e-praxis-b-a29f33-5c99ccdc58-q7xcz\" (UID: \"94bfd457-07b2-4c4e-85e1-1021b5201c82\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-a29f33-5c99ccdc58-q7xcz"
Oct 30 19:55:07.593800 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:07.593772    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4fb4\" (UniqueName: \"kubernetes.io/projected/94bfd457-07b2-4c4e-85e1-1021b5201c82-kube-api-access-w4fb4\") pod \"maas-api-e2e-praxis-b-a29f33-5c99ccdc58-q7xcz\" (UID: \"94bfd457-07b2-4c4e-85e1-1021b5201c82\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-a29f33-5c99ccdc58-q7xcz"
Oct 30 19:55:07.593843 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:07.593825    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94bfd457-07b2-4c4e-85e1-1021b5201c82-metrics-certs\") pod \"maas-api-e2e-praxis-b-a29f33-5c99ccdc58-q7xcz\" (UID: \"94bfd457-07b2-4c4e-85e1-1021b5201c82\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-a29f33-5c99ccdc58-q7xcz"
Oct 30 19:55:07.695310 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:07.695269    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4fb4\" (UniqueName: \"kubernetes.io/projected/94bfd457-07b2-4c4e-85e1-1021b5201c82-kube-api-access-w4fb4\") pod \"maas-api-e2e-praxis-b-a29f33-5c99ccdc58-q7xcz\" (UID: \"94bfd457-07b2-4c4e-85e1-1021b5201c82\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-a29f33-5c99ccdc58-q7xcz"
Oct 30 19:55:07.695545 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:07.695338    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94bfd457-07b2-4c4e-85e1-1021b5201c82-metrics-certs\") pod \"maas-api-e2e-praxis-b-a29f33-5c99ccdc58-q7xcz\" (UID: \"94bfd457-07b2-4c4e-85e1-1021b5201c82\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-a29f33-5c99ccdc58-q7xcz"
Oct 30 19:55:07.695545 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:07.695441    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/94bfd457-07b2-4c4e-85e1-1021b5201c82-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-a29f33-5c99ccdc58-q7xcz\" (UID: \"94bfd457-07b2-4c4e-85e1-1021b5201c82\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-a29f33-5c99ccdc58-q7xcz"
Oct 30 19:55:07.695545 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:07.695507    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/94bfd457-07b2-4c4e-85e1-1021b5201c82-maas-api-tls\") pod \"maas-api-e2e-praxis-b-a29f33-5c99ccdc58-q7xcz\" (UID: \"94bfd457-07b2-4c4e-85e1-1021b5201c82\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-a29f33-5c99ccdc58-q7xcz"
Oct 30 19:55:07.696236 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:07.696207    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/94bfd457-07b2-4c4e-85e1-1021b5201c82-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-a29f33-5c99ccdc58-q7xcz\" (UID: \"94bfd457-07b2-4c4e-85e1-1021b5201c82\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-a29f33-5c99ccdc58-q7xcz"
Oct 30 19:55:07.698121 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:07.698065    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/94bfd457-07b2-4c4e-85e1-1021b5201c82-maas-api-tls\") pod \"maas-api-e2e-praxis-b-a29f33-5c99ccdc58-q7xcz\" (UID: \"94bfd457-07b2-4c4e-85e1-1021b5201c82\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-a29f33-5c99ccdc58-q7xcz"
Oct 30 19:55:07.698218 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:07.698197    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94bfd457-07b2-4c4e-85e1-1021b5201c82-metrics-certs\") pod \"maas-api-e2e-praxis-b-a29f33-5c99ccdc58-q7xcz\" (UID: \"94bfd457-07b2-4c4e-85e1-1021b5201c82\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-a29f33-5c99ccdc58-q7xcz"
Oct 30 19:55:07.714248 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:07.714220    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4fb4\" (UniqueName: \"kubernetes.io/projected/94bfd457-07b2-4c4e-85e1-1021b5201c82-kube-api-access-w4fb4\") pod \"maas-api-e2e-praxis-b-a29f33-5c99ccdc58-q7xcz\" (UID: \"94bfd457-07b2-4c4e-85e1-1021b5201c82\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-a29f33-5c99ccdc58-q7xcz"
Oct 30 19:55:07.771767 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:07.771719    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-a29f33-5c99ccdc58-q7xcz"
Oct 30 19:55:08.070118 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:08.070081    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-a29f33-5c99ccdc58-q7xcz"]
Oct 30 19:55:08.074129 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:55:08.074101    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94bfd457_07b2_4c4e_85e1_1021b5201c82.slice/crio-415d2e510fee5412864a90e2effea869c92a49b6d13ff5c98b598f8141618cb9 WatchSource:0}: Error finding container 415d2e510fee5412864a90e2effea869c92a49b6d13ff5c98b598f8141618cb9: Status 404 returned error can't find the container with id 415d2e510fee5412864a90e2effea869c92a49b6d13ff5c98b598f8141618cb9
Oct 30 19:55:08.623938 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:08.623896    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg"]
Oct 30 19:55:08.625193 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:08.625167    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="f5d54632-af1e-458b-b431-61ead4fccb41" containerName="istio-proxy"
Oct 30 19:55:08.629691 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:08.629661    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg"
Oct 30 19:55:08.633943 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:08.633903    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-a29f33-5c99ccdc58-q7xcz" event={"ID":"94bfd457-07b2-4c4e-85e1-1021b5201c82","Type":"ContainerStarted","Data":"9a37a76f6ee35ddc1426a7cd78e42499f987592a63661b77aa264049870659c1"}
Oct 30 19:55:08.634102 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:08.633972    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-a29f33-5c99ccdc58-q7xcz" event={"ID":"94bfd457-07b2-4c4e-85e1-1021b5201c82","Type":"ContainerStarted","Data":"415d2e510fee5412864a90e2effea869c92a49b6d13ff5c98b598f8141618cb9"}
Oct 30 19:55:08.634546 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:08.634523    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-a29f33-5c99ccdc58-q7xcz"
Oct 30 19:55:08.682349 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:08.682313    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg"]
Oct 30 19:55:08.740250 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:08.740083    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d78490f-9250-4c76-9786-4d5499c34beb-credential-socket\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg\" (UID: \"7d78490f-9250-4c76-9786-4d5499c34beb\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg"
Oct 30 19:55:08.740748 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:08.740705    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d78490f-9250-4c76-9786-4d5499c34beb-workload-socket\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg\" (UID: \"7d78490f-9250-4c76-9786-4d5499c34beb\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg"
Oct 30 19:55:08.741227 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:08.741203    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d78490f-9250-4c76-9786-4d5499c34beb-istio-data\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg\" (UID: \"7d78490f-9250-4c76-9786-4d5499c34beb\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg"
Oct 30 19:55:08.742503 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:08.742453    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d78490f-9250-4c76-9786-4d5499c34beb-istio-envoy\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg\" (UID: \"7d78490f-9250-4c76-9786-4d5499c34beb\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg"
Oct 30 19:55:08.742761 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:08.742735    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d78490f-9250-4c76-9786-4d5499c34beb-workload-certs\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg\" (UID: \"7d78490f-9250-4c76-9786-4d5499c34beb\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg"
Oct 30 19:55:08.742871 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:08.742784    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d78490f-9250-4c76-9786-4d5499c34beb-istio-podinfo\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg\" (UID: \"7d78490f-9250-4c76-9786-4d5499c34beb\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg"
Oct 30 19:55:08.742871 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:08.742859    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9s7h5\" (UniqueName: \"kubernetes.io/projected/7d78490f-9250-4c76-9786-4d5499c34beb-kube-api-access-9s7h5\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg\" (UID: \"7d78490f-9250-4c76-9786-4d5499c34beb\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg"
Oct 30 19:55:08.742980 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:08.742900    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d78490f-9250-4c76-9786-4d5499c34beb-istiod-ca-cert\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg\" (UID: \"7d78490f-9250-4c76-9786-4d5499c34beb\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg"
Oct 30 19:55:08.743050 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:08.743038    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d78490f-9250-4c76-9786-4d5499c34beb-istio-token\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg\" (UID: \"7d78490f-9250-4c76-9786-4d5499c34beb\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg"
Oct 30 19:55:08.797569 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:08.797446    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-a29f33-5c99ccdc58-q7xcz" podStartSLOduration=1.335951093 podStartE2EDuration="1.797430527s" totalImagesPullingTime="461.479434ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:55:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:55:08.796315769 +0000 UTC m=+1384.124433924" watchObservedRunningTime="2026-10-30 19:55:08.797430527 +0000 UTC m=+1384.125548680"
Oct 30 19:55:08.844067 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:08.844016    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d78490f-9250-4c76-9786-4d5499c34beb-istio-envoy\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg\" (UID: \"7d78490f-9250-4c76-9786-4d5499c34beb\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg"
Oct 30 19:55:08.844256 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:08.844098    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d78490f-9250-4c76-9786-4d5499c34beb-workload-certs\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg\" (UID: \"7d78490f-9250-4c76-9786-4d5499c34beb\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg"
Oct 30 19:55:08.844256 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:08.844133    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d78490f-9250-4c76-9786-4d5499c34beb-istio-podinfo\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg\" (UID: \"7d78490f-9250-4c76-9786-4d5499c34beb\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg"
Oct 30 19:55:08.844256 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:08.844195    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9s7h5\" (UniqueName: \"kubernetes.io/projected/7d78490f-9250-4c76-9786-4d5499c34beb-kube-api-access-9s7h5\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg\" (UID: \"7d78490f-9250-4c76-9786-4d5499c34beb\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg"
Oct 30 19:55:08.844256 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:08.844231    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d78490f-9250-4c76-9786-4d5499c34beb-istiod-ca-cert\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg\" (UID: \"7d78490f-9250-4c76-9786-4d5499c34beb\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg"
Oct 30 19:55:08.844542 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:08.844371    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d78490f-9250-4c76-9786-4d5499c34beb-istio-token\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg\" (UID: \"7d78490f-9250-4c76-9786-4d5499c34beb\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg"
Oct 30 19:55:08.844542 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:08.844446    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d78490f-9250-4c76-9786-4d5499c34beb-credential-socket\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg\" (UID: \"7d78490f-9250-4c76-9786-4d5499c34beb\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg"
Oct 30 19:55:08.844542 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:08.844517    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d78490f-9250-4c76-9786-4d5499c34beb-workload-socket\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg\" (UID: \"7d78490f-9250-4c76-9786-4d5499c34beb\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg"
Oct 30 19:55:08.844702 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:08.844642    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d78490f-9250-4c76-9786-4d5499c34beb-istio-data\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg\" (UID: \"7d78490f-9250-4c76-9786-4d5499c34beb\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg"
Oct 30 19:55:08.845068 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:08.845045    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d78490f-9250-4c76-9786-4d5499c34beb-istio-data\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg\" (UID: \"7d78490f-9250-4c76-9786-4d5499c34beb\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg"
Oct 30 19:55:08.845200 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:08.845100    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d78490f-9250-4c76-9786-4d5499c34beb-workload-certs\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg\" (UID: \"7d78490f-9250-4c76-9786-4d5499c34beb\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg"
Oct 30 19:55:08.845380 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:08.845360    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d78490f-9250-4c76-9786-4d5499c34beb-credential-socket\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg\" (UID: \"7d78490f-9250-4c76-9786-4d5499c34beb\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg"
Oct 30 19:55:08.845445 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:08.845382    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d78490f-9250-4c76-9786-4d5499c34beb-workload-socket\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg\" (UID: \"7d78490f-9250-4c76-9786-4d5499c34beb\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg"
Oct 30 19:55:08.845445 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:08.845401    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d78490f-9250-4c76-9786-4d5499c34beb-istiod-ca-cert\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg\" (UID: \"7d78490f-9250-4c76-9786-4d5499c34beb\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg"
Oct 30 19:55:08.846834 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:08.846811    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d78490f-9250-4c76-9786-4d5499c34beb-istio-envoy\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg\" (UID: \"7d78490f-9250-4c76-9786-4d5499c34beb\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg"
Oct 30 19:55:08.848013 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:08.847996    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d78490f-9250-4c76-9786-4d5499c34beb-istio-podinfo\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg\" (UID: \"7d78490f-9250-4c76-9786-4d5499c34beb\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg"
Oct 30 19:55:08.862755 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:08.862732    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d78490f-9250-4c76-9786-4d5499c34beb-istio-token\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg\" (UID: \"7d78490f-9250-4c76-9786-4d5499c34beb\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg"
Oct 30 19:55:08.864229 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:08.864193    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9s7h5\" (UniqueName: \"kubernetes.io/projected/7d78490f-9250-4c76-9786-4d5499c34beb-kube-api-access-9s7h5\") pod \"e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg\" (UID: \"7d78490f-9250-4c76-9786-4d5499c34beb\") " pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg"
Oct 30 19:55:08.952203 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:08.952167    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg"
Oct 30 19:55:09.300043 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:09.300011    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg"]
Oct 30 19:55:09.302066 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:55:09.302036    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d78490f_9250_4c76_9786_4d5499c34beb.slice/crio-6c9603f51e8a633d0ef173c9155c1c8359fded0654c2c91bda1cb827698a0b98 WatchSource:0}: Error finding container 6c9603f51e8a633d0ef173c9155c1c8359fded0654c2c91bda1cb827698a0b98: Status 404 returned error can't find the container with id 6c9603f51e8a633d0ef173c9155c1c8359fded0654c2c91bda1cb827698a0b98
Oct 30 19:55:09.304797 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:09.304756    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:55:09.304896 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:09.304833    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:55:09.304896 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:09.304868    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:55:09.644450 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:09.644413    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg" event={"ID":"7d78490f-9250-4c76-9786-4d5499c34beb","Type":"ContainerStarted","Data":"a5e61c5578da65f9c92d5742d91400330c00050beda09e673ecd29f0bdd2eb4c"}
Oct 30 19:55:09.644984 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:09.644472    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg" event={"ID":"7d78490f-9250-4c76-9786-4d5499c34beb","Type":"ContainerStarted","Data":"6c9603f51e8a633d0ef173c9155c1c8359fded0654c2c91bda1cb827698a0b98"}
Oct 30 19:55:09.671664 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:09.671608    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg" podStartSLOduration=1.671593075 podStartE2EDuration="1.671593075s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:55:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:55:09.669010725 +0000 UTC m=+1384.997128875" watchObservedRunningTime="2026-10-30 19:55:09.671593075 +0000 UTC m=+1384.999711297"
Oct 30 19:55:09.958598 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:09.955664    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg"
Oct 30 19:55:10.968525 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:10.960137    6996 patch_prober.go:29] interesting pod/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.135:15021/healthz/ready\": dial tcp 10.134.0.135:15021: connect: connection refused" start-of-body=
Oct 30 19:55:10.968525 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:10.960199    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg" podUID="7d78490f-9250-4c76-9786-4d5499c34beb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.135:15021/healthz/ready\": dial tcp 10.134.0.135:15021: connect: connection refused"
Oct 30 19:55:11.956984 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:11.956934    6996 patch_prober.go:29] interesting pod/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.135:15021/healthz/ready\": dial tcp 10.134.0.135:15021: connect: connection refused" start-of-body=
Oct 30 19:55:11.957311 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:11.957276    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg" podUID="7d78490f-9250-4c76-9786-4d5499c34beb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.135:15021/healthz/ready\": dial tcp 10.134.0.135:15021: connect: connection refused"
Oct 30 19:55:12.964961 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:12.964013    6996 patch_prober.go:29] interesting pod/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.135:15021/healthz/ready\": dial tcp 10.134.0.135:15021: connect: connection refused" start-of-body=
Oct 30 19:55:12.964961 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:12.964103    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg" podUID="7d78490f-9250-4c76-9786-4d5499c34beb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.135:15021/healthz/ready\": dial tcp 10.134.0.135:15021: connect: connection refused"
Oct 30 19:55:13.094354 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:13.094313    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg"]
Oct 30 19:55:13.094566 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:13.094536    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg" podUID="0d1b1d3b-0290-4c57-9a3e-1afab986cb98" containerName="istio-proxy" containerID="cri-o://6fb733d4f7a8a52004c449204dace712960898006ac3adadf4084d24858885c5" gracePeriod=30
Oct 30 19:55:13.959503 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:13.958921    6996 patch_prober.go:29] interesting pod/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.135:15021/healthz/ready\": dial tcp 10.134.0.135:15021: connect: connection refused" start-of-body=
Oct 30 19:55:13.959503 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:13.958990    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg" podUID="7d78490f-9250-4c76-9786-4d5499c34beb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.135:15021/healthz/ready\": dial tcp 10.134.0.135:15021: connect: connection refused"
Oct 30 19:55:14.668440 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:14.668407    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-a29f33-5c99ccdc58-q7xcz"
Oct 30 19:55:14.963774 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:14.963680    6996 patch_prober.go:29] interesting pod/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.135:15021/healthz/ready\": dial tcp 10.134.0.135:15021: connect: connection refused" start-of-body=
Oct 30 19:55:14.964426 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:14.964024    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg" podUID="7d78490f-9250-4c76-9786-4d5499c34beb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.135:15021/healthz/ready\": dial tcp 10.134.0.135:15021: connect: connection refused"
Oct 30 19:55:15.952669 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:15.952630    6996 patch_prober.go:29] interesting pod/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.135:15021/healthz/ready\": dial tcp 10.134.0.135:15021: connect: connection refused" start-of-body=
Oct 30 19:55:15.953139 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:15.952696    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg" podUID="7d78490f-9250-4c76-9786-4d5499c34beb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.135:15021/healthz/ready\": dial tcp 10.134.0.135:15021: connect: connection refused"
Oct 30 19:55:16.351969 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:16.351929    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4"]
Oct 30 19:55:16.352204 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:16.352181    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4" podUID="e7778a88-c2f0-40f6-987a-8749b0cf3378" containerName="istio-proxy" containerID="cri-o://acf5ec0f48e7c4607b5377428016569da612b3cc40648339cf303c66a0e57fb2" gracePeriod=30
Oct 30 19:55:16.957773 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:16.957743    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg"
Oct 30 19:55:16.958210 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:16.958186    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg"
Oct 30 19:55:16.958816 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:16.958800    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg"
Oct 30 19:55:18.410969 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:18.410613    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg"
Oct 30 19:55:18.521271 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:18.521236    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0d1b1d3b-0290-4c57-9a3e-1afab986cb98-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d1b1d3b-0290-4c57-9a3e-1afab986cb98-istiod-ca-cert\") pod \"0d1b1d3b-0290-4c57-9a3e-1afab986cb98\" (UID: \"0d1b1d3b-0290-4c57-9a3e-1afab986cb98\") "
Oct 30 19:55:18.521455 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:18.521288    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0d1b1d3b-0290-4c57-9a3e-1afab986cb98-istio-token\" (UniqueName: \"kubernetes.io/projected/0d1b1d3b-0290-4c57-9a3e-1afab986cb98-istio-token\") pod \"0d1b1d3b-0290-4c57-9a3e-1afab986cb98\" (UID: \"0d1b1d3b-0290-4c57-9a3e-1afab986cb98\") "
Oct 30 19:55:18.521455 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:18.521335    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d1b1d3b-0290-4c57-9a3e-1afab986cb98-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d1b1d3b-0290-4c57-9a3e-1afab986cb98-credential-socket\") pod \"0d1b1d3b-0290-4c57-9a3e-1afab986cb98\" (UID: \"0d1b1d3b-0290-4c57-9a3e-1afab986cb98\") "
Oct 30 19:55:18.521455 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:18.521411    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d1b1d3b-0290-4c57-9a3e-1afab986cb98-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d1b1d3b-0290-4c57-9a3e-1afab986cb98-istio-data\") pod \"0d1b1d3b-0290-4c57-9a3e-1afab986cb98\" (UID: \"0d1b1d3b-0290-4c57-9a3e-1afab986cb98\") "
Oct 30 19:55:18.521586 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:18.521551    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0d1b1d3b-0290-4c57-9a3e-1afab986cb98-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d1b1d3b-0290-4c57-9a3e-1afab986cb98-istio-podinfo\") pod \"0d1b1d3b-0290-4c57-9a3e-1afab986cb98\" (UID: \"0d1b1d3b-0290-4c57-9a3e-1afab986cb98\") "
Oct 30 19:55:18.521693 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:18.521666    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d1b1d3b-0290-4c57-9a3e-1afab986cb98-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d1b1d3b-0290-4c57-9a3e-1afab986cb98-istio-envoy\") pod \"0d1b1d3b-0290-4c57-9a3e-1afab986cb98\" (UID: \"0d1b1d3b-0290-4c57-9a3e-1afab986cb98\") "
Oct 30 19:55:18.521843 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:18.521734    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d1b1d3b-0290-4c57-9a3e-1afab986cb98-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d1b1d3b-0290-4c57-9a3e-1afab986cb98-workload-socket\") pod \"0d1b1d3b-0290-4c57-9a3e-1afab986cb98\" (UID: \"0d1b1d3b-0290-4c57-9a3e-1afab986cb98\") "
Oct 30 19:55:18.521843 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:18.521741    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d1b1d3b-0290-4c57-9a3e-1afab986cb98-credential-socket" pod "0d1b1d3b-0290-4c57-9a3e-1afab986cb98" (UID: "0d1b1d3b-0290-4c57-9a3e-1afab986cb98"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:55:18.521843 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:18.521745    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d1b1d3b-0290-4c57-9a3e-1afab986cb98-istiod-ca-cert" pod "0d1b1d3b-0290-4c57-9a3e-1afab986cb98" (UID: "0d1b1d3b-0290-4c57-9a3e-1afab986cb98"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:55:18.521843 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:18.521775    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d1b1d3b-0290-4c57-9a3e-1afab986cb98-istio-data" pod "0d1b1d3b-0290-4c57-9a3e-1afab986cb98" (UID: "0d1b1d3b-0290-4c57-9a3e-1afab986cb98"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:55:18.521843 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:18.521794    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d1b1d3b-0290-4c57-9a3e-1afab986cb98-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d1b1d3b-0290-4c57-9a3e-1afab986cb98-workload-certs\") pod \"0d1b1d3b-0290-4c57-9a3e-1afab986cb98\" (UID: \"0d1b1d3b-0290-4c57-9a3e-1afab986cb98\") "
Oct 30 19:55:18.522122 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:18.521848    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0d1b1d3b-0290-4c57-9a3e-1afab986cb98-kube-api-access-8z2kh\" (UniqueName: \"kubernetes.io/projected/0d1b1d3b-0290-4c57-9a3e-1afab986cb98-kube-api-access-8z2kh\") pod \"0d1b1d3b-0290-4c57-9a3e-1afab986cb98\" (UID: \"0d1b1d3b-0290-4c57-9a3e-1afab986cb98\") "
Oct 30 19:55:18.522122 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:18.522002    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d1b1d3b-0290-4c57-9a3e-1afab986cb98-workload-socket" pod "0d1b1d3b-0290-4c57-9a3e-1afab986cb98" (UID: "0d1b1d3b-0290-4c57-9a3e-1afab986cb98"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:55:18.522122 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:18.522037    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d1b1d3b-0290-4c57-9a3e-1afab986cb98-workload-certs" pod "0d1b1d3b-0290-4c57-9a3e-1afab986cb98" (UID: "0d1b1d3b-0290-4c57-9a3e-1afab986cb98"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:55:18.522473 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:18.522452    6996 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d1b1d3b-0290-4c57-9a3e-1afab986cb98-workload-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:55:18.522601 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:18.522474    6996 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d1b1d3b-0290-4c57-9a3e-1afab986cb98-workload-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:55:18.522601 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:18.522524    6996 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d1b1d3b-0290-4c57-9a3e-1afab986cb98-istiod-ca-cert\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:55:18.522601 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:18.522534    6996 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d1b1d3b-0290-4c57-9a3e-1afab986cb98-credential-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:55:18.522601 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:18.522545    6996 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d1b1d3b-0290-4c57-9a3e-1afab986cb98-istio-data\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:55:18.523766 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:18.523741    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d1b1d3b-0290-4c57-9a3e-1afab986cb98-istio-token" pod "0d1b1d3b-0290-4c57-9a3e-1afab986cb98" (UID: "0d1b1d3b-0290-4c57-9a3e-1afab986cb98"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:55:18.524033 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:18.524013    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d1b1d3b-0290-4c57-9a3e-1afab986cb98-kube-api-access-8z2kh" pod "0d1b1d3b-0290-4c57-9a3e-1afab986cb98" (UID: "0d1b1d3b-0290-4c57-9a3e-1afab986cb98"). InnerVolumeSpecName "kube-api-access-8z2kh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:55:18.524211 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:18.524193    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0d1b1d3b-0290-4c57-9a3e-1afab986cb98-istio-podinfo" pod "0d1b1d3b-0290-4c57-9a3e-1afab986cb98" (UID: "0d1b1d3b-0290-4c57-9a3e-1afab986cb98"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:55:18.524278 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:18.524202    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d1b1d3b-0290-4c57-9a3e-1afab986cb98-istio-envoy" pod "0d1b1d3b-0290-4c57-9a3e-1afab986cb98" (UID: "0d1b1d3b-0290-4c57-9a3e-1afab986cb98"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:55:18.624387 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:18.624290    6996 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d1b1d3b-0290-4c57-9a3e-1afab986cb98-istio-podinfo\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:55:18.624387 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:18.624330    6996 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d1b1d3b-0290-4c57-9a3e-1afab986cb98-istio-envoy\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:55:18.624387 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:18.624339    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8z2kh\" (UniqueName: \"kubernetes.io/projected/0d1b1d3b-0290-4c57-9a3e-1afab986cb98-kube-api-access-8z2kh\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:55:18.624387 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:18.624348    6996 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d1b1d3b-0290-4c57-9a3e-1afab986cb98-istio-token\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:55:18.726727 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:18.726688    6996 generic.go:356] "Generic (PLEG): container finished" podID="0d1b1d3b-0290-4c57-9a3e-1afab986cb98" containerID="6fb733d4f7a8a52004c449204dace712960898006ac3adadf4084d24858885c5" exitCode=0
Oct 30 19:55:18.726902 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:18.726743    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg"
Oct 30 19:55:18.726902 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:18.726777    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg" event={"ID":"0d1b1d3b-0290-4c57-9a3e-1afab986cb98","Type":"ContainerDied","Data":"6fb733d4f7a8a52004c449204dace712960898006ac3adadf4084d24858885c5"}
Oct 30 19:55:18.726902 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:18.726835    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg" event={"ID":"0d1b1d3b-0290-4c57-9a3e-1afab986cb98","Type":"ContainerDied","Data":"22c7aba96449a04433a0c667aeed8e394af2591dedfe4f57db48447c25e91577"}
Oct 30 19:55:18.726902 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:18.726860    6996 scope.go:122] "RemoveContainer" containerID="6fb733d4f7a8a52004c449204dace712960898006ac3adadf4084d24858885c5"
Oct 30 19:55:18.738906 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:18.738883    6996 scope.go:122] "RemoveContainer" containerID="6fb733d4f7a8a52004c449204dace712960898006ac3adadf4084d24858885c5"
Oct 30 19:55:18.739245 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:55:18.739221    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6fb733d4f7a8a52004c449204dace712960898006ac3adadf4084d24858885c5\": container with ID starting with 6fb733d4f7a8a52004c449204dace712960898006ac3adadf4084d24858885c5 not found: ID does not exist" containerID="6fb733d4f7a8a52004c449204dace712960898006ac3adadf4084d24858885c5"
Oct 30 19:55:18.739357 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:18.739252    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fb733d4f7a8a52004c449204dace712960898006ac3adadf4084d24858885c5"} err="failed to get container status \"6fb733d4f7a8a52004c449204dace712960898006ac3adadf4084d24858885c5\": rpc error: code = NotFound desc = could not find container \"6fb733d4f7a8a52004c449204dace712960898006ac3adadf4084d24858885c5\": container with ID starting with 6fb733d4f7a8a52004c449204dace712960898006ac3adadf4084d24858885c5 not found: ID does not exist"
Oct 30 19:55:18.760867 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:18.760833    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg"]
Oct 30 19:55:18.765449 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:18.765421    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-2r2jg"]
Oct 30 19:55:18.793091 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:18.793055    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0d1b1d3b-0290-4c57-9a3e-1afab986cb98" path="/var/lib/kubelet/pods/0d1b1d3b-0290-4c57-9a3e-1afab986cb98/volumes"
Oct 30 19:55:21.628979 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:21.628911    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4"
Oct 30 19:55:21.750071 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:21.750033    6996 generic.go:356] "Generic (PLEG): container finished" podID="e7778a88-c2f0-40f6-987a-8749b0cf3378" containerID="acf5ec0f48e7c4607b5377428016569da612b3cc40648339cf303c66a0e57fb2" exitCode=0
Oct 30 19:55:21.750260 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:21.750089    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4"
Oct 30 19:55:21.750260 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:21.750099    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4" event={"ID":"e7778a88-c2f0-40f6-987a-8749b0cf3378","Type":"ContainerDied","Data":"acf5ec0f48e7c4607b5377428016569da612b3cc40648339cf303c66a0e57fb2"}
Oct 30 19:55:21.750260 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:21.750155    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4" event={"ID":"e7778a88-c2f0-40f6-987a-8749b0cf3378","Type":"ContainerDied","Data":"6dd3264cb7d446d6997a126c731af3d73877a1d43a76053790ed78f9d9b1b8ff"}
Oct 30 19:55:21.750260 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:21.750178    6996 scope.go:122] "RemoveContainer" containerID="acf5ec0f48e7c4607b5377428016569da612b3cc40648339cf303c66a0e57fb2"
Oct 30 19:55:21.760701 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:21.760682    6996 scope.go:122] "RemoveContainer" containerID="acf5ec0f48e7c4607b5377428016569da612b3cc40648339cf303c66a0e57fb2"
Oct 30 19:55:21.761023 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:55:21.760997    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"acf5ec0f48e7c4607b5377428016569da612b3cc40648339cf303c66a0e57fb2\": container with ID starting with acf5ec0f48e7c4607b5377428016569da612b3cc40648339cf303c66a0e57fb2 not found: ID does not exist" containerID="acf5ec0f48e7c4607b5377428016569da612b3cc40648339cf303c66a0e57fb2"
Oct 30 19:55:21.761070 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:21.761033    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"acf5ec0f48e7c4607b5377428016569da612b3cc40648339cf303c66a0e57fb2"} err="failed to get container status \"acf5ec0f48e7c4607b5377428016569da612b3cc40648339cf303c66a0e57fb2\": rpc error: code = NotFound desc = could not find container \"acf5ec0f48e7c4607b5377428016569da612b3cc40648339cf303c66a0e57fb2\": container with ID starting with acf5ec0f48e7c4607b5377428016569da612b3cc40648339cf303c66a0e57fb2 not found: ID does not exist"
Oct 30 19:55:21.776245 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:21.776204    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e7778a88-c2f0-40f6-987a-8749b0cf3378-kube-api-access-grz7n\" (UniqueName: \"kubernetes.io/projected/e7778a88-c2f0-40f6-987a-8749b0cf3378-kube-api-access-grz7n\") pod \"e7778a88-c2f0-40f6-987a-8749b0cf3378\" (UID: \"e7778a88-c2f0-40f6-987a-8749b0cf3378\") "
Oct 30 19:55:21.776449 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:21.776283    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e7778a88-c2f0-40f6-987a-8749b0cf3378-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e7778a88-c2f0-40f6-987a-8749b0cf3378-istio-data\") pod \"e7778a88-c2f0-40f6-987a-8749b0cf3378\" (UID: \"e7778a88-c2f0-40f6-987a-8749b0cf3378\") "
Oct 30 19:55:21.776449 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:21.776317    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e7778a88-c2f0-40f6-987a-8749b0cf3378-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e7778a88-c2f0-40f6-987a-8749b0cf3378-istio-envoy\") pod \"e7778a88-c2f0-40f6-987a-8749b0cf3378\" (UID: \"e7778a88-c2f0-40f6-987a-8749b0cf3378\") "
Oct 30 19:55:21.776449 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:21.776381    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e7778a88-c2f0-40f6-987a-8749b0cf3378-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7778a88-c2f0-40f6-987a-8749b0cf3378-workload-socket\") pod \"e7778a88-c2f0-40f6-987a-8749b0cf3378\" (UID: \"e7778a88-c2f0-40f6-987a-8749b0cf3378\") "
Oct 30 19:55:21.776449 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:21.776403    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e7778a88-c2f0-40f6-987a-8749b0cf3378-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e7778a88-c2f0-40f6-987a-8749b0cf3378-istio-podinfo\") pod \"e7778a88-c2f0-40f6-987a-8749b0cf3378\" (UID: \"e7778a88-c2f0-40f6-987a-8749b0cf3378\") "
Oct 30 19:55:21.776449 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:21.776451    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e7778a88-c2f0-40f6-987a-8749b0cf3378-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7778a88-c2f0-40f6-987a-8749b0cf3378-credential-socket\") pod \"e7778a88-c2f0-40f6-987a-8749b0cf3378\" (UID: \"e7778a88-c2f0-40f6-987a-8749b0cf3378\") "
Oct 30 19:55:21.776760 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:21.776579    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e7778a88-c2f0-40f6-987a-8749b0cf3378-istio-token\" (UniqueName: \"kubernetes.io/projected/e7778a88-c2f0-40f6-987a-8749b0cf3378-istio-token\") pod \"e7778a88-c2f0-40f6-987a-8749b0cf3378\" (UID: \"e7778a88-c2f0-40f6-987a-8749b0cf3378\") "
Oct 30 19:55:21.776760 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:21.776688    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e7778a88-c2f0-40f6-987a-8749b0cf3378-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e7778a88-c2f0-40f6-987a-8749b0cf3378-istiod-ca-cert\") pod \"e7778a88-c2f0-40f6-987a-8749b0cf3378\" (UID: \"e7778a88-c2f0-40f6-987a-8749b0cf3378\") "
Oct 30 19:55:21.776760 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:21.776731    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e7778a88-c2f0-40f6-987a-8749b0cf3378-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7778a88-c2f0-40f6-987a-8749b0cf3378-workload-certs\") pod \"e7778a88-c2f0-40f6-987a-8749b0cf3378\" (UID: \"e7778a88-c2f0-40f6-987a-8749b0cf3378\") "
Oct 30 19:55:21.777711 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:21.777678    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7778a88-c2f0-40f6-987a-8749b0cf3378-workload-certs" pod "e7778a88-c2f0-40f6-987a-8749b0cf3378" (UID: "e7778a88-c2f0-40f6-987a-8749b0cf3378"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:55:21.779944 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:21.779104    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7778a88-c2f0-40f6-987a-8749b0cf3378-credential-socket" pod "e7778a88-c2f0-40f6-987a-8749b0cf3378" (UID: "e7778a88-c2f0-40f6-987a-8749b0cf3378"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:55:21.779944 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:21.779433    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7778a88-c2f0-40f6-987a-8749b0cf3378-istiod-ca-cert" pod "e7778a88-c2f0-40f6-987a-8749b0cf3378" (UID: "e7778a88-c2f0-40f6-987a-8749b0cf3378"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:55:21.779944 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:21.779724    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7778a88-c2f0-40f6-987a-8749b0cf3378-workload-socket" pod "e7778a88-c2f0-40f6-987a-8749b0cf3378" (UID: "e7778a88-c2f0-40f6-987a-8749b0cf3378"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:55:21.779944 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:21.779915    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7778a88-c2f0-40f6-987a-8749b0cf3378-istio-data" pod "e7778a88-c2f0-40f6-987a-8749b0cf3378" (UID: "e7778a88-c2f0-40f6-987a-8749b0cf3378"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:55:21.800780 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:21.800285    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7778a88-c2f0-40f6-987a-8749b0cf3378-istio-token" pod "e7778a88-c2f0-40f6-987a-8749b0cf3378" (UID: "e7778a88-c2f0-40f6-987a-8749b0cf3378"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:55:21.800780 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:21.800458    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7778a88-c2f0-40f6-987a-8749b0cf3378-istio-envoy" pod "e7778a88-c2f0-40f6-987a-8749b0cf3378" (UID: "e7778a88-c2f0-40f6-987a-8749b0cf3378"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:55:21.806009 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:21.801599    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e7778a88-c2f0-40f6-987a-8749b0cf3378-istio-podinfo" pod "e7778a88-c2f0-40f6-987a-8749b0cf3378" (UID: "e7778a88-c2f0-40f6-987a-8749b0cf3378"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:55:21.806612 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:21.806582    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7778a88-c2f0-40f6-987a-8749b0cf3378-kube-api-access-grz7n" pod "e7778a88-c2f0-40f6-987a-8749b0cf3378" (UID: "e7778a88-c2f0-40f6-987a-8749b0cf3378"). InnerVolumeSpecName "kube-api-access-grz7n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:55:21.878714 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:21.878677    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-grz7n\" (UniqueName: \"kubernetes.io/projected/e7778a88-c2f0-40f6-987a-8749b0cf3378-kube-api-access-grz7n\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:55:21.878714 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:21.878710    6996 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e7778a88-c2f0-40f6-987a-8749b0cf3378-istio-data\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:55:21.878714 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:21.878720    6996 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e7778a88-c2f0-40f6-987a-8749b0cf3378-istio-envoy\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:55:21.878973 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:21.878729    6996 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7778a88-c2f0-40f6-987a-8749b0cf3378-workload-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:55:21.878973 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:21.878739    6996 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e7778a88-c2f0-40f6-987a-8749b0cf3378-istio-podinfo\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:55:21.878973 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:21.878748    6996 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7778a88-c2f0-40f6-987a-8749b0cf3378-credential-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:55:21.878973 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:21.878758    6996 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7778a88-c2f0-40f6-987a-8749b0cf3378-istio-token\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:55:21.878973 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:21.878767    6996 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e7778a88-c2f0-40f6-987a-8749b0cf3378-istiod-ca-cert\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:55:21.878973 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:21.878776    6996 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7778a88-c2f0-40f6-987a-8749b0cf3378-workload-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:55:22.100473 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:22.100433    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4"]
Oct 30 19:55:22.103711 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:22.103683    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-2prq4"]
Oct 30 19:55:22.792687 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:22.792649    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e7778a88-c2f0-40f6-987a-8749b0cf3378" path="/var/lib/kubelet/pods/e7778a88-c2f0-40f6-987a-8749b0cf3378/volumes"
Oct 30 19:55:24.515493 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:24.515433    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-55a8e6-86d8478fc6-85xvd"]
Oct 30 19:55:24.515917 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:24.515774    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-55a8e6-86d8478fc6-85xvd" podUID="3d767ba6-5ac3-4732-b52a-b60c2705d1de" containerName="maas-api" containerID="cri-o://bde16c724b98aff03145e2c2ed0dc9f3f7fad0512fbe0e099831548cbc991344" gracePeriod=30
Oct 30 19:55:24.791621 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:24.791446    6996 generic.go:356] "Generic (PLEG): container finished" podID="3d767ba6-5ac3-4732-b52a-b60c2705d1de" containerID="bde16c724b98aff03145e2c2ed0dc9f3f7fad0512fbe0e099831548cbc991344" exitCode=0
Oct 30 19:55:24.795934 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:24.795899    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-55a8e6-86d8478fc6-85xvd" event={"ID":"3d767ba6-5ac3-4732-b52a-b60c2705d1de","Type":"ContainerDied","Data":"bde16c724b98aff03145e2c2ed0dc9f3f7fad0512fbe0e099831548cbc991344"}
Oct 30 19:55:24.850733 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:24.850693    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-55a8e6-86d8478fc6-85xvd"
Oct 30 19:55:24.965252 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:24.965217    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3d767ba6-5ac3-4732-b52a-b60c2705d1de-metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d767ba6-5ac3-4732-b52a-b60c2705d1de-metrics-certs\") pod \"3d767ba6-5ac3-4732-b52a-b60c2705d1de\" (UID: \"3d767ba6-5ac3-4732-b52a-b60c2705d1de\") "
Oct 30 19:55:24.965437 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:24.965426    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3d767ba6-5ac3-4732-b52a-b60c2705d1de-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d767ba6-5ac3-4732-b52a-b60c2705d1de-maas-api-tls\") pod \"3d767ba6-5ac3-4732-b52a-b60c2705d1de\" (UID: \"3d767ba6-5ac3-4732-b52a-b60c2705d1de\") "
Oct 30 19:55:24.965562 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:24.965543    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3d767ba6-5ac3-4732-b52a-b60c2705d1de-kube-api-access-fqzt6\" (UniqueName: \"kubernetes.io/projected/3d767ba6-5ac3-4732-b52a-b60c2705d1de-kube-api-access-fqzt6\") pod \"3d767ba6-5ac3-4732-b52a-b60c2705d1de\" (UID: \"3d767ba6-5ac3-4732-b52a-b60c2705d1de\") "
Oct 30 19:55:24.965666 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:24.965648    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3d767ba6-5ac3-4732-b52a-b60c2705d1de-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d767ba6-5ac3-4732-b52a-b60c2705d1de-openshift-service-ca\") pod \"3d767ba6-5ac3-4732-b52a-b60c2705d1de\" (UID: \"3d767ba6-5ac3-4732-b52a-b60c2705d1de\") "
Oct 30 19:55:24.966017 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:24.965994    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d767ba6-5ac3-4732-b52a-b60c2705d1de-openshift-service-ca" pod "3d767ba6-5ac3-4732-b52a-b60c2705d1de" (UID: "3d767ba6-5ac3-4732-b52a-b60c2705d1de"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:55:24.966340 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:24.966321    6996 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d767ba6-5ac3-4732-b52a-b60c2705d1de-openshift-service-ca\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:55:24.967764 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:24.967739    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d767ba6-5ac3-4732-b52a-b60c2705d1de-maas-api-tls" pod "3d767ba6-5ac3-4732-b52a-b60c2705d1de" (UID: "3d767ba6-5ac3-4732-b52a-b60c2705d1de"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:55:24.967855 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:24.967760    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d767ba6-5ac3-4732-b52a-b60c2705d1de-metrics-certs" pod "3d767ba6-5ac3-4732-b52a-b60c2705d1de" (UID: "3d767ba6-5ac3-4732-b52a-b60c2705d1de"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:55:24.967855 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:24.967818    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d767ba6-5ac3-4732-b52a-b60c2705d1de-kube-api-access-fqzt6" pod "3d767ba6-5ac3-4732-b52a-b60c2705d1de" (UID: "3d767ba6-5ac3-4732-b52a-b60c2705d1de"). InnerVolumeSpecName "kube-api-access-fqzt6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:55:25.069814 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:25.069730    6996 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d767ba6-5ac3-4732-b52a-b60c2705d1de-maas-api-tls\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:55:25.069814 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:25.069770    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fqzt6\" (UniqueName: \"kubernetes.io/projected/3d767ba6-5ac3-4732-b52a-b60c2705d1de-kube-api-access-fqzt6\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:55:25.069814 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:25.069788    6996 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d767ba6-5ac3-4732-b52a-b60c2705d1de-metrics-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:55:25.884505 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:25.883820    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-55a8e6-86d8478fc6-85xvd" event={"ID":"3d767ba6-5ac3-4732-b52a-b60c2705d1de","Type":"ContainerDied","Data":"516d6c3100094cc3e5b55038f668889789250c3b651989c6118692d5dec5318c"}
Oct 30 19:55:25.884505 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:25.883897    6996 scope.go:122] "RemoveContainer" containerID="bde16c724b98aff03145e2c2ed0dc9f3f7fad0512fbe0e099831548cbc991344"
Oct 30 19:55:25.884505 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:25.884045    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-55a8e6-86d8478fc6-85xvd"
Oct 30 19:55:26.079424 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:26.079391    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-55a8e6-86d8478fc6-85xvd"]
Oct 30 19:55:26.129504 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:26.109266    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-55a8e6-86d8478fc6-85xvd"]
Oct 30 19:55:26.842504 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:26.841747    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3d767ba6-5ac3-4732-b52a-b60c2705d1de" path="/var/lib/kubelet/pods/3d767ba6-5ac3-4732-b52a-b60c2705d1de/volumes"
Oct 30 19:55:26.876506 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:26.872587    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s" podUID="ae905dfb-f30d-480a-899b-ea676abc1d8d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:55:26.876506 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:26.873045    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr" podUID="0e470010-625d-443f-81ec-9bf965988212" containerName="main" probeResult="failure" output="Get \"https://10.134.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:55:26.910527 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:26.908371    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq" podUID="89009638-003d-4a5d-ab13-1f8c8f64d3b7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.88:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:55:26.910527 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:26.908887    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk" podUID="a1857d2d-3b40-4e63-be33-be4ba476f926" containerName="main" probeResult="failure" output="Get \"https://10.134.0.90:8000/ready\": context deadline exceeded"
Oct 30 19:55:26.919511 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:26.918602    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b" podUID="19fe4fba-09ae-4620-b4d2-0ef8c2463d71" containerName="main" probeResult="failure" output="Get \"https://10.134.0.81:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:55:31.192239 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:55:31.192201    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 19:55:50.041993 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:50.041958    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 19:55:52.866507 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:52.863473    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b" podUID="19fe4fba-09ae-4620-b4d2-0ef8c2463d71" containerName="main" probeResult="failure" output="Get \"https://10.134.0.81:8000/ready\": context deadline exceeded"
Oct 30 19:55:52.879520 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:52.878760    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr" podUID="0e470010-625d-443f-81ec-9bf965988212" containerName="main" probeResult="failure" output="Get \"https://10.134.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:55:52.879520 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:52.879139    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s" podUID="ae905dfb-f30d-480a-899b-ea676abc1d8d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.73:8000/ready\": context deadline exceeded"
Oct 30 19:55:52.908503 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:52.907582    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk" podUID="a1857d2d-3b40-4e63-be33-be4ba476f926" containerName="main" probeResult="failure" output="Get \"https://10.134.0.90:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:55:52.908503 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:52.908017    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq" podUID="89009638-003d-4a5d-ab13-1f8c8f64d3b7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.88:8000/ready\": context deadline exceeded"
Oct 30 19:55:58.140826 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:58.140781    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz"]
Oct 30 19:55:58.141553 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:55:58.141021    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz" podUID="0a5af397-4eab-444b-bb54-f5f0cc2ab73a" containerName="istio-proxy" containerID="cri-o://456b89a23267803472b4c6bd5cb8e187fd6a560a049e0080719f056e9d5fff33" gracePeriod=30
Oct 30 19:56:01.220743 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:56:01.220696    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 19:56:03.422703 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:03.422618    6996 generic.go:356] "Generic (PLEG): container finished" podID="0a5af397-4eab-444b-bb54-f5f0cc2ab73a" containerID="456b89a23267803472b4c6bd5cb8e187fd6a560a049e0080719f056e9d5fff33" exitCode=0
Oct 30 19:56:03.422703 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:03.422667    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz" event={"ID":"0a5af397-4eab-444b-bb54-f5f0cc2ab73a","Type":"ContainerDied","Data":"456b89a23267803472b4c6bd5cb8e187fd6a560a049e0080719f056e9d5fff33"}
Oct 30 19:56:03.423067 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:03.422729    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz" event={"ID":"0a5af397-4eab-444b-bb54-f5f0cc2ab73a","Type":"ContainerDied","Data":"5c8d8443e36dec043c0de0f51bfffc6c231617d513539031fe93e889158798b8"}
Oct 30 19:56:03.423067 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:03.422752    6996 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5c8d8443e36dec043c0de0f51bfffc6c231617d513539031fe93e889158798b8"
Oct 30 19:56:03.425066 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:03.425050    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz"
Oct 30 19:56:03.528697 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:03.528664    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0a5af397-4eab-444b-bb54-f5f0cc2ab73a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a5af397-4eab-444b-bb54-f5f0cc2ab73a-workload-socket\") pod \"0a5af397-4eab-444b-bb54-f5f0cc2ab73a\" (UID: \"0a5af397-4eab-444b-bb54-f5f0cc2ab73a\") "
Oct 30 19:56:03.528882 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:03.528715    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0a5af397-4eab-444b-bb54-f5f0cc2ab73a-istio-token\" (UniqueName: \"kubernetes.io/projected/0a5af397-4eab-444b-bb54-f5f0cc2ab73a-istio-token\") pod \"0a5af397-4eab-444b-bb54-f5f0cc2ab73a\" (UID: \"0a5af397-4eab-444b-bb54-f5f0cc2ab73a\") "
Oct 30 19:56:03.528882 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:03.528819    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0a5af397-4eab-444b-bb54-f5f0cc2ab73a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a5af397-4eab-444b-bb54-f5f0cc2ab73a-istio-podinfo\") pod \"0a5af397-4eab-444b-bb54-f5f0cc2ab73a\" (UID: \"0a5af397-4eab-444b-bb54-f5f0cc2ab73a\") "
Oct 30 19:56:03.529014 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:03.528897    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0a5af397-4eab-444b-bb54-f5f0cc2ab73a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a5af397-4eab-444b-bb54-f5f0cc2ab73a-istio-data\") pod \"0a5af397-4eab-444b-bb54-f5f0cc2ab73a\" (UID: \"0a5af397-4eab-444b-bb54-f5f0cc2ab73a\") "
Oct 30 19:56:03.529014 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:03.528984    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0a5af397-4eab-444b-bb54-f5f0cc2ab73a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a5af397-4eab-444b-bb54-f5f0cc2ab73a-workload-certs\") pod \"0a5af397-4eab-444b-bb54-f5f0cc2ab73a\" (UID: \"0a5af397-4eab-444b-bb54-f5f0cc2ab73a\") "
Oct 30 19:56:03.529014 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:03.528997    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a5af397-4eab-444b-bb54-f5f0cc2ab73a-workload-socket" pod "0a5af397-4eab-444b-bb54-f5f0cc2ab73a" (UID: "0a5af397-4eab-444b-bb54-f5f0cc2ab73a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:56:03.529177 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:03.529013    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0a5af397-4eab-444b-bb54-f5f0cc2ab73a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a5af397-4eab-444b-bb54-f5f0cc2ab73a-credential-socket\") pod \"0a5af397-4eab-444b-bb54-f5f0cc2ab73a\" (UID: \"0a5af397-4eab-444b-bb54-f5f0cc2ab73a\") "
Oct 30 19:56:03.529235 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:03.529171    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0a5af397-4eab-444b-bb54-f5f0cc2ab73a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a5af397-4eab-444b-bb54-f5f0cc2ab73a-istio-envoy\") pod \"0a5af397-4eab-444b-bb54-f5f0cc2ab73a\" (UID: \"0a5af397-4eab-444b-bb54-f5f0cc2ab73a\") "
Oct 30 19:56:03.529235 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:03.529181    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a5af397-4eab-444b-bb54-f5f0cc2ab73a-istio-data" pod "0a5af397-4eab-444b-bb54-f5f0cc2ab73a" (UID: "0a5af397-4eab-444b-bb54-f5f0cc2ab73a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:56:03.529235 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:03.529221    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a5af397-4eab-444b-bb54-f5f0cc2ab73a-credential-socket" pod "0a5af397-4eab-444b-bb54-f5f0cc2ab73a" (UID: "0a5af397-4eab-444b-bb54-f5f0cc2ab73a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:56:03.529389 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:03.529240    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0a5af397-4eab-444b-bb54-f5f0cc2ab73a-kube-api-access-jfkvx\" (UniqueName: \"kubernetes.io/projected/0a5af397-4eab-444b-bb54-f5f0cc2ab73a-kube-api-access-jfkvx\") pod \"0a5af397-4eab-444b-bb54-f5f0cc2ab73a\" (UID: \"0a5af397-4eab-444b-bb54-f5f0cc2ab73a\") "
Oct 30 19:56:03.529389 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:03.529240    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a5af397-4eab-444b-bb54-f5f0cc2ab73a-workload-certs" pod "0a5af397-4eab-444b-bb54-f5f0cc2ab73a" (UID: "0a5af397-4eab-444b-bb54-f5f0cc2ab73a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:56:03.529389 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:03.529300    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0a5af397-4eab-444b-bb54-f5f0cc2ab73a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a5af397-4eab-444b-bb54-f5f0cc2ab73a-istiod-ca-cert\") pod \"0a5af397-4eab-444b-bb54-f5f0cc2ab73a\" (UID: \"0a5af397-4eab-444b-bb54-f5f0cc2ab73a\") "
Oct 30 19:56:03.529649 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:03.529621    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a5af397-4eab-444b-bb54-f5f0cc2ab73a-istiod-ca-cert" pod "0a5af397-4eab-444b-bb54-f5f0cc2ab73a" (UID: "0a5af397-4eab-444b-bb54-f5f0cc2ab73a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:56:03.530045 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:03.530028    6996 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a5af397-4eab-444b-bb54-f5f0cc2ab73a-workload-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:56:03.530126 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:03.530049    6996 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a5af397-4eab-444b-bb54-f5f0cc2ab73a-istio-data\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:56:03.530126 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:03.530063    6996 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a5af397-4eab-444b-bb54-f5f0cc2ab73a-workload-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:56:03.530126 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:03.530076    6996 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a5af397-4eab-444b-bb54-f5f0cc2ab73a-credential-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:56:03.530126 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:03.530091    6996 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a5af397-4eab-444b-bb54-f5f0cc2ab73a-istiod-ca-cert\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:56:03.531096 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:03.531071    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0a5af397-4eab-444b-bb54-f5f0cc2ab73a-istio-podinfo" pod "0a5af397-4eab-444b-bb54-f5f0cc2ab73a" (UID: "0a5af397-4eab-444b-bb54-f5f0cc2ab73a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:56:03.531334 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:03.531319    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a5af397-4eab-444b-bb54-f5f0cc2ab73a-kube-api-access-jfkvx" pod "0a5af397-4eab-444b-bb54-f5f0cc2ab73a" (UID: "0a5af397-4eab-444b-bb54-f5f0cc2ab73a"). InnerVolumeSpecName "kube-api-access-jfkvx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:56:03.531442 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:03.531425    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a5af397-4eab-444b-bb54-f5f0cc2ab73a-istio-envoy" pod "0a5af397-4eab-444b-bb54-f5f0cc2ab73a" (UID: "0a5af397-4eab-444b-bb54-f5f0cc2ab73a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:56:03.531547 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:03.531428    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a5af397-4eab-444b-bb54-f5f0cc2ab73a-istio-token" pod "0a5af397-4eab-444b-bb54-f5f0cc2ab73a" (UID: "0a5af397-4eab-444b-bb54-f5f0cc2ab73a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:56:03.631984 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:03.631957    6996 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a5af397-4eab-444b-bb54-f5f0cc2ab73a-istio-token\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:56:03.631984 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:03.631985    6996 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a5af397-4eab-444b-bb54-f5f0cc2ab73a-istio-podinfo\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:56:03.632158 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:03.631996    6996 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a5af397-4eab-444b-bb54-f5f0cc2ab73a-istio-envoy\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:56:03.632158 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:03.632006    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jfkvx\" (UniqueName: \"kubernetes.io/projected/0a5af397-4eab-444b-bb54-f5f0cc2ab73a-kube-api-access-jfkvx\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:56:04.430737 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:04.430703    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz"
Oct 30 19:56:04.462407 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:04.462199    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz"]
Oct 30 19:56:04.466007 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:04.465979    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-praxis-a-b17e94-openshift-default-666cf59769-vhjfz"]
Oct 30 19:56:04.473447 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:04.473409    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-a29f33-5c99ccdc58-q7xcz"]
Oct 30 19:56:04.474212 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:04.473901    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-a29f33-5c99ccdc58-q7xcz" podUID="94bfd457-07b2-4c4e-85e1-1021b5201c82" containerName="maas-api" containerID="cri-o://9a37a76f6ee35ddc1426a7cd78e42499f987592a63661b77aa264049870659c1" gracePeriod=30
Oct 30 19:56:04.647960 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:04.647913    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-a29f33-5c99ccdc58-q7xcz" podUID="94bfd457-07b2-4c4e-85e1-1021b5201c82" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.134:8443/health\": dial tcp 10.134.0.134:8443: connect: connection refused"
Oct 30 19:56:04.789966 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:04.789934    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-a29f33-5c99ccdc58-q7xcz"
Oct 30 19:56:04.793946 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:04.793919    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0a5af397-4eab-444b-bb54-f5f0cc2ab73a" path="/var/lib/kubelet/pods/0a5af397-4eab-444b-bb54-f5f0cc2ab73a/volumes"
Oct 30 19:56:04.958728 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:04.958631    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/94bfd457-07b2-4c4e-85e1-1021b5201c82-kube-api-access-w4fb4\" (UniqueName: \"kubernetes.io/projected/94bfd457-07b2-4c4e-85e1-1021b5201c82-kube-api-access-w4fb4\") pod \"94bfd457-07b2-4c4e-85e1-1021b5201c82\" (UID: \"94bfd457-07b2-4c4e-85e1-1021b5201c82\") "
Oct 30 19:56:04.960323 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:04.960289    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/94bfd457-07b2-4c4e-85e1-1021b5201c82-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/94bfd457-07b2-4c4e-85e1-1021b5201c82-maas-api-tls\") pod \"94bfd457-07b2-4c4e-85e1-1021b5201c82\" (UID: \"94bfd457-07b2-4c4e-85e1-1021b5201c82\") "
Oct 30 19:56:04.960673 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:04.960640    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/94bfd457-07b2-4c4e-85e1-1021b5201c82-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/94bfd457-07b2-4c4e-85e1-1021b5201c82-openshift-service-ca\") pod \"94bfd457-07b2-4c4e-85e1-1021b5201c82\" (UID: \"94bfd457-07b2-4c4e-85e1-1021b5201c82\") "
Oct 30 19:56:04.961052 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:04.961024    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/94bfd457-07b2-4c4e-85e1-1021b5201c82-metrics-certs\" (UniqueName: \"kubernetes.io/secret/94bfd457-07b2-4c4e-85e1-1021b5201c82-metrics-certs\") pod \"94bfd457-07b2-4c4e-85e1-1021b5201c82\" (UID: \"94bfd457-07b2-4c4e-85e1-1021b5201c82\") "
Oct 30 19:56:04.965303 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:04.965264    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/94bfd457-07b2-4c4e-85e1-1021b5201c82-openshift-service-ca" pod "94bfd457-07b2-4c4e-85e1-1021b5201c82" (UID: "94bfd457-07b2-4c4e-85e1-1021b5201c82"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:56:04.997223 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:04.997164    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/94bfd457-07b2-4c4e-85e1-1021b5201c82-maas-api-tls" pod "94bfd457-07b2-4c4e-85e1-1021b5201c82" (UID: "94bfd457-07b2-4c4e-85e1-1021b5201c82"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:56:04.998155 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:04.998112    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94bfd457-07b2-4c4e-85e1-1021b5201c82-kube-api-access-w4fb4" pod "94bfd457-07b2-4c4e-85e1-1021b5201c82" (UID: "94bfd457-07b2-4c4e-85e1-1021b5201c82"). InnerVolumeSpecName "kube-api-access-w4fb4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:56:04.998696 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:04.998662    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/94bfd457-07b2-4c4e-85e1-1021b5201c82-metrics-certs" pod "94bfd457-07b2-4c4e-85e1-1021b5201c82" (UID: "94bfd457-07b2-4c4e-85e1-1021b5201c82"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:56:05.067163 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:05.067115    6996 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/94bfd457-07b2-4c4e-85e1-1021b5201c82-openshift-service-ca\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:56:05.067163 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:05.067154    6996 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94bfd457-07b2-4c4e-85e1-1021b5201c82-metrics-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:56:05.067430 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:05.067171    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w4fb4\" (UniqueName: \"kubernetes.io/projected/94bfd457-07b2-4c4e-85e1-1021b5201c82-kube-api-access-w4fb4\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:56:05.067430 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:05.067188    6996 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/94bfd457-07b2-4c4e-85e1-1021b5201c82-maas-api-tls\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:56:05.439022 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:05.438987    6996 generic.go:356] "Generic (PLEG): container finished" podID="94bfd457-07b2-4c4e-85e1-1021b5201c82" containerID="9a37a76f6ee35ddc1426a7cd78e42499f987592a63661b77aa264049870659c1" exitCode=0
Oct 30 19:56:05.439372 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:05.439045    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-a29f33-5c99ccdc58-q7xcz"
Oct 30 19:56:05.439372 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:05.439079    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-a29f33-5c99ccdc58-q7xcz" event={"ID":"94bfd457-07b2-4c4e-85e1-1021b5201c82","Type":"ContainerDied","Data":"9a37a76f6ee35ddc1426a7cd78e42499f987592a63661b77aa264049870659c1"}
Oct 30 19:56:05.439372 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:05.439135    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-a29f33-5c99ccdc58-q7xcz" event={"ID":"94bfd457-07b2-4c4e-85e1-1021b5201c82","Type":"ContainerDied","Data":"415d2e510fee5412864a90e2effea869c92a49b6d13ff5c98b598f8141618cb9"}
Oct 30 19:56:05.439372 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:05.439167    6996 scope.go:122] "RemoveContainer" containerID="9a37a76f6ee35ddc1426a7cd78e42499f987592a63661b77aa264049870659c1"
Oct 30 19:56:05.449269 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:05.449250    6996 scope.go:122] "RemoveContainer" containerID="9a37a76f6ee35ddc1426a7cd78e42499f987592a63661b77aa264049870659c1"
Oct 30 19:56:05.449636 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:56:05.449615    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a37a76f6ee35ddc1426a7cd78e42499f987592a63661b77aa264049870659c1\": container with ID starting with 9a37a76f6ee35ddc1426a7cd78e42499f987592a63661b77aa264049870659c1 not found: ID does not exist" containerID="9a37a76f6ee35ddc1426a7cd78e42499f987592a63661b77aa264049870659c1"
Oct 30 19:56:05.449718 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:05.449650    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a37a76f6ee35ddc1426a7cd78e42499f987592a63661b77aa264049870659c1"} err="failed to get container status \"9a37a76f6ee35ddc1426a7cd78e42499f987592a63661b77aa264049870659c1\": rpc error: code = NotFound desc = could not find container \"9a37a76f6ee35ddc1426a7cd78e42499f987592a63661b77aa264049870659c1\": container with ID starting with 9a37a76f6ee35ddc1426a7cd78e42499f987592a63661b77aa264049870659c1 not found: ID does not exist"
Oct 30 19:56:05.482158 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:05.482113    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-a29f33-5c99ccdc58-q7xcz"]
Oct 30 19:56:05.486362 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:05.486333    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-a29f33-5c99ccdc58-q7xcz"]
Oct 30 19:56:06.792454 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:06.792418    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="94bfd457-07b2-4c4e-85e1-1021b5201c82" path="/var/lib/kubelet/pods/94bfd457-07b2-4c4e-85e1-1021b5201c82/volumes"
Oct 30 19:56:14.269812 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:14.269763    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-f03120-79bf49bdd6-z9zzs"]
Oct 30 19:56:14.301783 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:14.301257    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-f03120-7b689f64f9-vj88z"]
Oct 30 19:56:19.271033 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:19.270944    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-praxis-b-f03120-79bf49bdd6-z9zzs" podUID="eaa59c33-59f0-43c2-b8ec-68d753622ab9" containerName="payload-processing" containerID="cri-o://1468ba88c1fe4669d47383027a568dc1ea18d524d9e305555c21af3ae137d61c" gracePeriod=25
Oct 30 19:56:19.302590 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:19.302536    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-f03120-7b689f64f9-vj88z" podUID="2691319f-ce3f-4930-b1b6-2e35f8f55a64" containerName="payload-pre-processing" containerID="cri-o://c96b47b9ada5a870e82723e1c48afa0ad13a3607d4eba7cad3aff50a855e2293" gracePeriod=25
Oct 30 19:56:19.588239 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:19.588215    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-b-f03120-79bf49bdd6-z9zzs"
Oct 30 19:56:19.627728 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:19.626449    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-f03120-7b689f64f9-vj88z"
Oct 30 19:56:19.662117 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:19.661295    6996 generic.go:356] "Generic (PLEG): container finished" podID="2691319f-ce3f-4930-b1b6-2e35f8f55a64" containerID="c96b47b9ada5a870e82723e1c48afa0ad13a3607d4eba7cad3aff50a855e2293" exitCode=0
Oct 30 19:56:19.662117 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:19.661354    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-f03120-7b689f64f9-vj88z"
Oct 30 19:56:19.662117 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:19.661458    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-f03120-7b689f64f9-vj88z" event={"ID":"2691319f-ce3f-4930-b1b6-2e35f8f55a64","Type":"ContainerDied","Data":"c96b47b9ada5a870e82723e1c48afa0ad13a3607d4eba7cad3aff50a855e2293"}
Oct 30 19:56:19.662117 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:19.661527    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-f03120-7b689f64f9-vj88z" event={"ID":"2691319f-ce3f-4930-b1b6-2e35f8f55a64","Type":"ContainerDied","Data":"add9f27fadc9298e2a550922c0872e83f0e730d1a2f27d837d89a7efe4fa9cab"}
Oct 30 19:56:19.662117 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:19.661559    6996 scope.go:122] "RemoveContainer" containerID="c96b47b9ada5a870e82723e1c48afa0ad13a3607d4eba7cad3aff50a855e2293"
Oct 30 19:56:19.665251 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:19.665123    6996 generic.go:356] "Generic (PLEG): container finished" podID="eaa59c33-59f0-43c2-b8ec-68d753622ab9" containerID="1468ba88c1fe4669d47383027a568dc1ea18d524d9e305555c21af3ae137d61c" exitCode=0
Oct 30 19:56:19.665808 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:19.665767    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-b-f03120-79bf49bdd6-z9zzs"
Oct 30 19:56:19.666042 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:19.666020    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-b-f03120-79bf49bdd6-z9zzs" event={"ID":"eaa59c33-59f0-43c2-b8ec-68d753622ab9","Type":"ContainerDied","Data":"1468ba88c1fe4669d47383027a568dc1ea18d524d9e305555c21af3ae137d61c"}
Oct 30 19:56:19.666143 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:19.666087    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-b-f03120-79bf49bdd6-z9zzs" event={"ID":"eaa59c33-59f0-43c2-b8ec-68d753622ab9","Type":"ContainerDied","Data":"1b80b763fd8030c83d6171638ac7d5feb71320314a7b8b388e3bfcde74c6511a"}
Oct 30 19:56:19.667671 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:19.667646    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2691319f-ce3f-4930-b1b6-2e35f8f55a64-kube-api-access-jzb7t\" (UniqueName: \"kubernetes.io/projected/2691319f-ce3f-4930-b1b6-2e35f8f55a64-kube-api-access-jzb7t\") pod \"2691319f-ce3f-4930-b1b6-2e35f8f55a64\" (UID: \"2691319f-ce3f-4930-b1b6-2e35f8f55a64\") "
Oct 30 19:56:19.667750 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:19.667720    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eaa59c33-59f0-43c2-b8ec-68d753622ab9-kube-api-access-xdlrw\" (UniqueName: \"kubernetes.io/projected/eaa59c33-59f0-43c2-b8ec-68d753622ab9-kube-api-access-xdlrw\") pod \"eaa59c33-59f0-43c2-b8ec-68d753622ab9\" (UID: \"eaa59c33-59f0-43c2-b8ec-68d753622ab9\") "
Oct 30 19:56:19.667826 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:19.667811    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2691319f-ce3f-4930-b1b6-2e35f8f55a64-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2691319f-ce3f-4930-b1b6-2e35f8f55a64-plugins-config-volume\") pod \"2691319f-ce3f-4930-b1b6-2e35f8f55a64\" (UID: \"2691319f-ce3f-4930-b1b6-2e35f8f55a64\") "
Oct 30 19:56:19.667927 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:19.667912    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/eaa59c33-59f0-43c2-b8ec-68d753622ab9-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eaa59c33-59f0-43c2-b8ec-68d753622ab9-plugins-config-volume\") pod \"eaa59c33-59f0-43c2-b8ec-68d753622ab9\" (UID: \"eaa59c33-59f0-43c2-b8ec-68d753622ab9\") "
Oct 30 19:56:19.668931 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:19.668899    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eaa59c33-59f0-43c2-b8ec-68d753622ab9-plugins-config-volume" pod "eaa59c33-59f0-43c2-b8ec-68d753622ab9" (UID: "eaa59c33-59f0-43c2-b8ec-68d753622ab9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:56:19.671623 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:19.671597    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2691319f-ce3f-4930-b1b6-2e35f8f55a64-plugins-config-volume" pod "2691319f-ce3f-4930-b1b6-2e35f8f55a64" (UID: "2691319f-ce3f-4930-b1b6-2e35f8f55a64"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:56:19.671742 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:19.671604    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2691319f-ce3f-4930-b1b6-2e35f8f55a64-kube-api-access-jzb7t" pod "2691319f-ce3f-4930-b1b6-2e35f8f55a64" (UID: "2691319f-ce3f-4930-b1b6-2e35f8f55a64"). InnerVolumeSpecName "kube-api-access-jzb7t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:56:19.677849 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:19.677826    6996 scope.go:122] "RemoveContainer" containerID="c96b47b9ada5a870e82723e1c48afa0ad13a3607d4eba7cad3aff50a855e2293"
Oct 30 19:56:19.678228 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:56:19.678199    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c96b47b9ada5a870e82723e1c48afa0ad13a3607d4eba7cad3aff50a855e2293\": container with ID starting with c96b47b9ada5a870e82723e1c48afa0ad13a3607d4eba7cad3aff50a855e2293 not found: ID does not exist" containerID="c96b47b9ada5a870e82723e1c48afa0ad13a3607d4eba7cad3aff50a855e2293"
Oct 30 19:56:19.678303 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:19.678238    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c96b47b9ada5a870e82723e1c48afa0ad13a3607d4eba7cad3aff50a855e2293"} err="failed to get container status \"c96b47b9ada5a870e82723e1c48afa0ad13a3607d4eba7cad3aff50a855e2293\": rpc error: code = NotFound desc = could not find container \"c96b47b9ada5a870e82723e1c48afa0ad13a3607d4eba7cad3aff50a855e2293\": container with ID starting with c96b47b9ada5a870e82723e1c48afa0ad13a3607d4eba7cad3aff50a855e2293 not found: ID does not exist"
Oct 30 19:56:19.678303 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:19.678262    6996 scope.go:122] "RemoveContainer" containerID="1468ba88c1fe4669d47383027a568dc1ea18d524d9e305555c21af3ae137d61c"
Oct 30 19:56:19.682078 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:19.682048    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eaa59c33-59f0-43c2-b8ec-68d753622ab9-kube-api-access-xdlrw" pod "eaa59c33-59f0-43c2-b8ec-68d753622ab9" (UID: "eaa59c33-59f0-43c2-b8ec-68d753622ab9"). InnerVolumeSpecName "kube-api-access-xdlrw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:56:19.689428 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:19.689408    6996 scope.go:122] "RemoveContainer" containerID="1468ba88c1fe4669d47383027a568dc1ea18d524d9e305555c21af3ae137d61c"
Oct 30 19:56:19.689767 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:56:19.689730    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1468ba88c1fe4669d47383027a568dc1ea18d524d9e305555c21af3ae137d61c\": container with ID starting with 1468ba88c1fe4669d47383027a568dc1ea18d524d9e305555c21af3ae137d61c not found: ID does not exist" containerID="1468ba88c1fe4669d47383027a568dc1ea18d524d9e305555c21af3ae137d61c"
Oct 30 19:56:19.689829 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:19.689778    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1468ba88c1fe4669d47383027a568dc1ea18d524d9e305555c21af3ae137d61c"} err="failed to get container status \"1468ba88c1fe4669d47383027a568dc1ea18d524d9e305555c21af3ae137d61c\": rpc error: code = NotFound desc = could not find container \"1468ba88c1fe4669d47383027a568dc1ea18d524d9e305555c21af3ae137d61c\": container with ID starting with 1468ba88c1fe4669d47383027a568dc1ea18d524d9e305555c21af3ae137d61c not found: ID does not exist"
Oct 30 19:56:19.769662 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:19.769630    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jzb7t\" (UniqueName: \"kubernetes.io/projected/2691319f-ce3f-4930-b1b6-2e35f8f55a64-kube-api-access-jzb7t\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:56:19.769662 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:19.769659    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xdlrw\" (UniqueName: \"kubernetes.io/projected/eaa59c33-59f0-43c2-b8ec-68d753622ab9-kube-api-access-xdlrw\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:56:19.769662 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:19.769668    6996 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2691319f-ce3f-4930-b1b6-2e35f8f55a64-plugins-config-volume\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:56:19.769889 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:19.769677    6996 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eaa59c33-59f0-43c2-b8ec-68d753622ab9-plugins-config-volume\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:56:19.992682 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:19.992650    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-f03120-7b689f64f9-vj88z"]
Oct 30 19:56:19.998276 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:19.998251    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-f03120-7b689f64f9-vj88z"]
Oct 30 19:56:20.009840 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:20.009810    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-f03120-79bf49bdd6-z9zzs"]
Oct 30 19:56:20.012783 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:20.012760    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-f03120-79bf49bdd6-z9zzs"]
Oct 30 19:56:20.793849 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:20.793815    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2691319f-ce3f-4930-b1b6-2e35f8f55a64" path="/var/lib/kubelet/pods/2691319f-ce3f-4930-b1b6-2e35f8f55a64/volumes"
Oct 30 19:56:20.794243 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:20.794228    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="eaa59c33-59f0-43c2-b8ec-68d753622ab9" path="/var/lib/kubelet/pods/eaa59c33-59f0-43c2-b8ec-68d753622ab9/volumes"
Oct 30 19:56:21.136672 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:21.136640    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-0eeba5-75c6455594-8kbxt"]
Oct 30 19:56:21.137574 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:21.137552    6996 cpu_manager.go:414] "containerMap: removing container" podUID="94bfd457-07b2-4c4e-85e1-1021b5201c82" containerName="maas-api"
Oct 30 19:56:21.137643 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:21.137577    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="94bfd457-07b2-4c4e-85e1-1021b5201c82" containerName="maas-api"
Oct 30 19:56:21.137643 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:21.137598    6996 cpu_manager.go:414] "containerMap: removing container" podUID="3d767ba6-5ac3-4732-b52a-b60c2705d1de" containerName="maas-api"
Oct 30 19:56:21.137643 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:21.137606    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3d767ba6-5ac3-4732-b52a-b60c2705d1de" containerName="maas-api"
Oct 30 19:56:21.137753 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:21.137661    6996 cpu_manager.go:414] "containerMap: removing container" podUID="2691319f-ce3f-4930-b1b6-2e35f8f55a64" containerName="payload-pre-processing"
Oct 30 19:56:21.137753 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:21.137669    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2691319f-ce3f-4930-b1b6-2e35f8f55a64" containerName="payload-pre-processing"
Oct 30 19:56:21.137753 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:21.137691    6996 cpu_manager.go:414] "containerMap: removing container" podUID="eaa59c33-59f0-43c2-b8ec-68d753622ab9" containerName="payload-processing"
Oct 30 19:56:21.137753 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:21.137698    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eaa59c33-59f0-43c2-b8ec-68d753622ab9" containerName="payload-processing"
Oct 30 19:56:21.137753 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:21.137706    6996 cpu_manager.go:414] "containerMap: removing container" podUID="e7778a88-c2f0-40f6-987a-8749b0cf3378" containerName="istio-proxy"
Oct 30 19:56:21.137753 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:21.137714    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e7778a88-c2f0-40f6-987a-8749b0cf3378" containerName="istio-proxy"
Oct 30 19:56:21.137753 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:21.137736    6996 cpu_manager.go:414] "containerMap: removing container" podUID="0d1b1d3b-0290-4c57-9a3e-1afab986cb98" containerName="istio-proxy"
Oct 30 19:56:21.137753 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:21.137743    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0d1b1d3b-0290-4c57-9a3e-1afab986cb98" containerName="istio-proxy"
Oct 30 19:56:21.137986 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:21.137759    6996 cpu_manager.go:414] "containerMap: removing container" podUID="0a5af397-4eab-444b-bb54-f5f0cc2ab73a" containerName="istio-proxy"
Oct 30 19:56:21.137986 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:21.137766    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0a5af397-4eab-444b-bb54-f5f0cc2ab73a" containerName="istio-proxy"
Oct 30 19:56:21.137986 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:21.137903    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="0d1b1d3b-0290-4c57-9a3e-1afab986cb98" containerName="istio-proxy"
Oct 30 19:56:21.137986 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:21.137919    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="0a5af397-4eab-444b-bb54-f5f0cc2ab73a" containerName="istio-proxy"
Oct 30 19:56:21.137986 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:21.137929    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="94bfd457-07b2-4c4e-85e1-1021b5201c82" containerName="maas-api"
Oct 30 19:56:21.137986 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:21.137940    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="3d767ba6-5ac3-4732-b52a-b60c2705d1de" containerName="maas-api"
Oct 30 19:56:21.137986 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:21.137954    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="2691319f-ce3f-4930-b1b6-2e35f8f55a64" containerName="payload-pre-processing"
Oct 30 19:56:21.137986 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:21.137964    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="eaa59c33-59f0-43c2-b8ec-68d753622ab9" containerName="payload-processing"
Oct 30 19:56:21.137986 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:21.137972    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="e7778a88-c2f0-40f6-987a-8749b0cf3378" containerName="istio-proxy"
Oct 30 19:56:21.143329 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:21.143308    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0eeba5-75c6455594-8kbxt"
Oct 30 19:56:21.166644 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:21.166613    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-0eeba5-75c6455594-8kbxt"]
Oct 30 19:56:21.234554 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:21.217818    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-0eeba5\""
Oct 30 19:56:21.234554 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:21.217957    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-0eeba5-dockercfg-smhxf\""
Oct 30 19:56:21.303056 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:21.303005    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7t8kc\" (UniqueName: \"kubernetes.io/projected/3bc8f341-bbf4-42d4-941a-0e636de353fe-kube-api-access-7t8kc\") pod \"payload-processing-e2e-shared-a-w3-0eeba5-75c6455594-8kbxt\" (UID: \"3bc8f341-bbf4-42d4-941a-0e636de353fe\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0eeba5-75c6455594-8kbxt"
Oct 30 19:56:21.303358 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:21.303339    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3bc8f341-bbf4-42d4-941a-0e636de353fe-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-0eeba5-75c6455594-8kbxt\" (UID: \"3bc8f341-bbf4-42d4-941a-0e636de353fe\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0eeba5-75c6455594-8kbxt"
Oct 30 19:56:21.404507 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:21.404401    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7t8kc\" (UniqueName: \"kubernetes.io/projected/3bc8f341-bbf4-42d4-941a-0e636de353fe-kube-api-access-7t8kc\") pod \"payload-processing-e2e-shared-a-w3-0eeba5-75c6455594-8kbxt\" (UID: \"3bc8f341-bbf4-42d4-941a-0e636de353fe\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0eeba5-75c6455594-8kbxt"
Oct 30 19:56:21.404673 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:21.404570    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3bc8f341-bbf4-42d4-941a-0e636de353fe-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-0eeba5-75c6455594-8kbxt\" (UID: \"3bc8f341-bbf4-42d4-941a-0e636de353fe\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0eeba5-75c6455594-8kbxt"
Oct 30 19:56:21.405283 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:21.405263    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3bc8f341-bbf4-42d4-941a-0e636de353fe-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-0eeba5-75c6455594-8kbxt\" (UID: \"3bc8f341-bbf4-42d4-941a-0e636de353fe\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0eeba5-75c6455594-8kbxt"
Oct 30 19:56:21.417101 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:21.417072    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7t8kc\" (UniqueName: \"kubernetes.io/projected/3bc8f341-bbf4-42d4-941a-0e636de353fe-kube-api-access-7t8kc\") pod \"payload-processing-e2e-shared-a-w3-0eeba5-75c6455594-8kbxt\" (UID: \"3bc8f341-bbf4-42d4-941a-0e636de353fe\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0eeba5-75c6455594-8kbxt"
Oct 30 19:56:21.483084 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:21.483055    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0eeba5-75c6455594-8kbxt"
Oct 30 19:56:21.770100 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:21.770075    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-0eeba5-75c6455594-8kbxt"]
Oct 30 19:56:21.771586 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:56:21.771558    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bc8f341_bbf4_42d4_941a_0e636de353fe.slice/crio-8b285f0f30e2c27c4f08034772bb823bd7596499b2848a677bf7da8e021e6883 WatchSource:0}: Error finding container 8b285f0f30e2c27c4f08034772bb823bd7596499b2848a677bf7da8e021e6883: Status 404 returned error can't find the container with id 8b285f0f30e2c27c4f08034772bb823bd7596499b2848a677bf7da8e021e6883
Oct 30 19:56:22.689187 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:22.689154    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0eeba5-75c6455594-8kbxt" event={"ID":"3bc8f341-bbf4-42d4-941a-0e636de353fe","Type":"ContainerStarted","Data":"d6536549fd3e0ab44191c445d54cb36fdfc58da198afac152de207958d1da78d"}
Oct 30 19:56:22.689629 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:22.689213    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0eeba5-75c6455594-8kbxt" event={"ID":"3bc8f341-bbf4-42d4-941a-0e636de353fe","Type":"ContainerStarted","Data":"8b285f0f30e2c27c4f08034772bb823bd7596499b2848a677bf7da8e021e6883"}
Oct 30 19:56:22.689629 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:22.689244    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0eeba5-75c6455594-8kbxt"
Oct 30 19:56:22.709714 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:22.709658    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0eeba5-75c6455594-8kbxt" podStartSLOduration=1.7096407999999998 podStartE2EDuration="1.7096408s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:56:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:56:22.708766896 +0000 UTC m=+1458.036885050" watchObservedRunningTime="2026-10-30 19:56:22.7096408 +0000 UTC m=+1458.037758977"
Oct 30 19:56:23.705942 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:23.705901    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-c1768b-8558cb48f5-hq4x4"]
Oct 30 19:56:23.710446 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:23.710420    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-c1768b-8558cb48f5-hq4x4"
Oct 30 19:56:23.732340 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:23.732320    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-praxis-b-c1768b-serving-cert\""
Oct 30 19:56:23.734985 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:23.734953    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da82ca39-fdea-4fa0-bde2-6dae317029b3-metrics-certs\") pod \"maas-api-e2e-praxis-b-c1768b-8558cb48f5-hq4x4\" (UID: \"da82ca39-fdea-4fa0-bde2-6dae317029b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-c1768b-8558cb48f5-hq4x4"
Oct 30 19:56:23.735130 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:23.735104    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpv7m\" (UniqueName: \"kubernetes.io/projected/da82ca39-fdea-4fa0-bde2-6dae317029b3-kube-api-access-wpv7m\") pod \"maas-api-e2e-praxis-b-c1768b-8558cb48f5-hq4x4\" (UID: \"da82ca39-fdea-4fa0-bde2-6dae317029b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-c1768b-8558cb48f5-hq4x4"
Oct 30 19:56:23.735190 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:23.735159    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da82ca39-fdea-4fa0-bde2-6dae317029b3-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-c1768b-8558cb48f5-hq4x4\" (UID: \"da82ca39-fdea-4fa0-bde2-6dae317029b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-c1768b-8558cb48f5-hq4x4"
Oct 30 19:56:23.735297 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:23.735278    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da82ca39-fdea-4fa0-bde2-6dae317029b3-maas-api-tls\") pod \"maas-api-e2e-praxis-b-c1768b-8558cb48f5-hq4x4\" (UID: \"da82ca39-fdea-4fa0-bde2-6dae317029b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-c1768b-8558cb48f5-hq4x4"
Oct 30 19:56:23.749168 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:23.749143    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-c1768b-8558cb48f5-hq4x4"]
Oct 30 19:56:23.836952 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:23.836914    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da82ca39-fdea-4fa0-bde2-6dae317029b3-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-c1768b-8558cb48f5-hq4x4\" (UID: \"da82ca39-fdea-4fa0-bde2-6dae317029b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-c1768b-8558cb48f5-hq4x4"
Oct 30 19:56:23.837151 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:23.836998    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da82ca39-fdea-4fa0-bde2-6dae317029b3-maas-api-tls\") pod \"maas-api-e2e-praxis-b-c1768b-8558cb48f5-hq4x4\" (UID: \"da82ca39-fdea-4fa0-bde2-6dae317029b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-c1768b-8558cb48f5-hq4x4"
Oct 30 19:56:23.837226 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:56:23.837190    6996 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-praxis-b-c1768b-serving-cert: secret "maas-api-e2e-praxis-b-c1768b-serving-cert" not found
Oct 30 19:56:23.837279 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:23.837225    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da82ca39-fdea-4fa0-bde2-6dae317029b3-metrics-certs\") pod \"maas-api-e2e-praxis-b-c1768b-8558cb48f5-hq4x4\" (UID: \"da82ca39-fdea-4fa0-bde2-6dae317029b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-c1768b-8558cb48f5-hq4x4"
Oct 30 19:56:23.837279 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:56:23.837265    6996 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da82ca39-fdea-4fa0-bde2-6dae317029b3-maas-api-tls podName:da82ca39-fdea-4fa0-bde2-6dae317029b3 nodeName:}" failed. No retries permitted until 2026-10-30 19:56:24.337242854 +0000 UTC m=+1459.665361001 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/da82ca39-fdea-4fa0-bde2-6dae317029b3-maas-api-tls") pod "maas-api-e2e-praxis-b-c1768b-8558cb48f5-hq4x4" (UID: "da82ca39-fdea-4fa0-bde2-6dae317029b3") : secret "maas-api-e2e-praxis-b-c1768b-serving-cert" not found
Oct 30 19:56:23.837436 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:23.837416    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpv7m\" (UniqueName: \"kubernetes.io/projected/da82ca39-fdea-4fa0-bde2-6dae317029b3-kube-api-access-wpv7m\") pod \"maas-api-e2e-praxis-b-c1768b-8558cb48f5-hq4x4\" (UID: \"da82ca39-fdea-4fa0-bde2-6dae317029b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-c1768b-8558cb48f5-hq4x4"
Oct 30 19:56:23.837818 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:23.837796    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da82ca39-fdea-4fa0-bde2-6dae317029b3-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-c1768b-8558cb48f5-hq4x4\" (UID: \"da82ca39-fdea-4fa0-bde2-6dae317029b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-c1768b-8558cb48f5-hq4x4"
Oct 30 19:56:23.840418 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:23.840392    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da82ca39-fdea-4fa0-bde2-6dae317029b3-metrics-certs\") pod \"maas-api-e2e-praxis-b-c1768b-8558cb48f5-hq4x4\" (UID: \"da82ca39-fdea-4fa0-bde2-6dae317029b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-c1768b-8558cb48f5-hq4x4"
Oct 30 19:56:23.851158 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:23.851127    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpv7m\" (UniqueName: \"kubernetes.io/projected/da82ca39-fdea-4fa0-bde2-6dae317029b3-kube-api-access-wpv7m\") pod \"maas-api-e2e-praxis-b-c1768b-8558cb48f5-hq4x4\" (UID: \"da82ca39-fdea-4fa0-bde2-6dae317029b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-c1768b-8558cb48f5-hq4x4"
Oct 30 19:56:24.345101 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:24.345061    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da82ca39-fdea-4fa0-bde2-6dae317029b3-maas-api-tls\") pod \"maas-api-e2e-praxis-b-c1768b-8558cb48f5-hq4x4\" (UID: \"da82ca39-fdea-4fa0-bde2-6dae317029b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-c1768b-8558cb48f5-hq4x4"
Oct 30 19:56:24.347507 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:24.347463    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da82ca39-fdea-4fa0-bde2-6dae317029b3-maas-api-tls\") pod \"maas-api-e2e-praxis-b-c1768b-8558cb48f5-hq4x4\" (UID: \"da82ca39-fdea-4fa0-bde2-6dae317029b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-c1768b-8558cb48f5-hq4x4"
Oct 30 19:56:24.623998 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:24.623918    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-c1768b-8558cb48f5-hq4x4"
Oct 30 19:56:24.985999 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:24.985974    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-c1768b-8558cb48f5-hq4x4"]
Oct 30 19:56:24.988118 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:56:24.988087    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda82ca39_fdea_4fa0_bde2_6dae317029b3.slice/crio-07752cbac81d4940c90164ede864737c2b1027ea603480e866be663e61c13768 WatchSource:0}: Error finding container 07752cbac81d4940c90164ede864737c2b1027ea603480e866be663e61c13768: Status 404 returned error can't find the container with id 07752cbac81d4940c90164ede864737c2b1027ea603480e866be663e61c13768
Oct 30 19:56:24.989865 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:24.989840    6996 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 19:56:25.716983 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:25.716898    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-c1768b-8558cb48f5-hq4x4" event={"ID":"da82ca39-fdea-4fa0-bde2-6dae317029b3","Type":"ContainerStarted","Data":"2d0e59e82597853f28dd8d45f070cdc9940c9af054f76da105fb8026b80c359a"}
Oct 30 19:56:25.716983 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:25.716962    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-c1768b-8558cb48f5-hq4x4" event={"ID":"da82ca39-fdea-4fa0-bde2-6dae317029b3","Type":"ContainerStarted","Data":"07752cbac81d4940c90164ede864737c2b1027ea603480e866be663e61c13768"}
Oct 30 19:56:25.717161 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:25.716993    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-c1768b-8558cb48f5-hq4x4"
Oct 30 19:56:25.738956 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:25.738907    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-c1768b-8558cb48f5-hq4x4" podStartSLOduration=2.297788559 podStartE2EDuration="2.738892828s" totalImagesPullingTime="441.104269ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:56:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:56:25.737621538 +0000 UTC m=+1461.065739704" watchObservedRunningTime="2026-10-30 19:56:25.738892828 +0000 UTC m=+1461.067010982"
Oct 30 19:56:28.891211 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:28.891173    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b24a1a-6f55488c68-pk5m5"]
Oct 30 19:56:28.896248 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:28.896225    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b24a1a-6f55488c68-pk5m5"
Oct 30 19:56:28.916961 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:28.916932    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b24a1a-6f55488c68-pk5m5"]
Oct 30 19:56:28.918816 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:28.918792    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-b24a1a-serving-cert\""
Oct 30 19:56:29.035068 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:29.035030    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/33ca6985-3896-45c4-bb2f-b58fb5fb40b4-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-b24a1a-6f55488c68-pk5m5\" (UID: \"33ca6985-3896-45c4-bb2f-b58fb5fb40b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b24a1a-6f55488c68-pk5m5"
Oct 30 19:56:29.035334 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:29.035088    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33ca6985-3896-45c4-bb2f-b58fb5fb40b4-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-b24a1a-6f55488c68-pk5m5\" (UID: \"33ca6985-3896-45c4-bb2f-b58fb5fb40b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b24a1a-6f55488c68-pk5m5"
Oct 30 19:56:29.035334 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:29.035199    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxbhk\" (UniqueName: \"kubernetes.io/projected/33ca6985-3896-45c4-bb2f-b58fb5fb40b4-kube-api-access-rxbhk\") pod \"maas-api-e2e-shared-a-w2-b24a1a-6f55488c68-pk5m5\" (UID: \"33ca6985-3896-45c4-bb2f-b58fb5fb40b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b24a1a-6f55488c68-pk5m5"
Oct 30 19:56:29.035334 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:29.035238    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33ca6985-3896-45c4-bb2f-b58fb5fb40b4-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-b24a1a-6f55488c68-pk5m5\" (UID: \"33ca6985-3896-45c4-bb2f-b58fb5fb40b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b24a1a-6f55488c68-pk5m5"
Oct 30 19:56:29.141555 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:29.141449    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33ca6985-3896-45c4-bb2f-b58fb5fb40b4-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-b24a1a-6f55488c68-pk5m5\" (UID: \"33ca6985-3896-45c4-bb2f-b58fb5fb40b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b24a1a-6f55488c68-pk5m5"
Oct 30 19:56:29.141555 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:29.141547    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxbhk\" (UniqueName: \"kubernetes.io/projected/33ca6985-3896-45c4-bb2f-b58fb5fb40b4-kube-api-access-rxbhk\") pod \"maas-api-e2e-shared-a-w2-b24a1a-6f55488c68-pk5m5\" (UID: \"33ca6985-3896-45c4-bb2f-b58fb5fb40b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b24a1a-6f55488c68-pk5m5"
Oct 30 19:56:29.141990 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:29.141583    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33ca6985-3896-45c4-bb2f-b58fb5fb40b4-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-b24a1a-6f55488c68-pk5m5\" (UID: \"33ca6985-3896-45c4-bb2f-b58fb5fb40b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b24a1a-6f55488c68-pk5m5"
Oct 30 19:56:29.141990 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:29.141843    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/33ca6985-3896-45c4-bb2f-b58fb5fb40b4-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-b24a1a-6f55488c68-pk5m5\" (UID: \"33ca6985-3896-45c4-bb2f-b58fb5fb40b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b24a1a-6f55488c68-pk5m5"
Oct 30 19:56:29.143416 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:29.143385    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33ca6985-3896-45c4-bb2f-b58fb5fb40b4-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-b24a1a-6f55488c68-pk5m5\" (UID: \"33ca6985-3896-45c4-bb2f-b58fb5fb40b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b24a1a-6f55488c68-pk5m5"
Oct 30 19:56:29.147092 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:29.147057    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33ca6985-3896-45c4-bb2f-b58fb5fb40b4-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-b24a1a-6f55488c68-pk5m5\" (UID: \"33ca6985-3896-45c4-bb2f-b58fb5fb40b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b24a1a-6f55488c68-pk5m5"
Oct 30 19:56:29.156965 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:29.155468    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/33ca6985-3896-45c4-bb2f-b58fb5fb40b4-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-b24a1a-6f55488c68-pk5m5\" (UID: \"33ca6985-3896-45c4-bb2f-b58fb5fb40b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b24a1a-6f55488c68-pk5m5"
Oct 30 19:56:29.160672 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:29.160645    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxbhk\" (UniqueName: \"kubernetes.io/projected/33ca6985-3896-45c4-bb2f-b58fb5fb40b4-kube-api-access-rxbhk\") pod \"maas-api-e2e-shared-a-w2-b24a1a-6f55488c68-pk5m5\" (UID: \"33ca6985-3896-45c4-bb2f-b58fb5fb40b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b24a1a-6f55488c68-pk5m5"
Oct 30 19:56:29.227327 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:29.227292    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b24a1a-6f55488c68-pk5m5"
Oct 30 19:56:29.554777 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:29.554748    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b24a1a-6f55488c68-pk5m5"]
Oct 30 19:56:29.556516 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:56:29.556452    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33ca6985_3896_45c4_bb2f_b58fb5fb40b4.slice/crio-aa75121ce3e2b73162e98326aaa5f65d02b97661b25a5f6ebac3dbb84ca0610b WatchSource:0}: Error finding container aa75121ce3e2b73162e98326aaa5f65d02b97661b25a5f6ebac3dbb84ca0610b: Status 404 returned error can't find the container with id aa75121ce3e2b73162e98326aaa5f65d02b97661b25a5f6ebac3dbb84ca0610b
Oct 30 19:56:29.744941 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:29.744905    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b24a1a-6f55488c68-pk5m5" event={"ID":"33ca6985-3896-45c4-bb2f-b58fb5fb40b4","Type":"ContainerStarted","Data":"aa75121ce3e2b73162e98326aaa5f65d02b97661b25a5f6ebac3dbb84ca0610b"}
Oct 30 19:56:30.754267 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:30.754226    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b24a1a-6f55488c68-pk5m5" event={"ID":"33ca6985-3896-45c4-bb2f-b58fb5fb40b4","Type":"ContainerStarted","Data":"7d9978fccdad9b9b9e7c286ceaa4460a362d6d2527fd4c157ce647bf90575476"}
Oct 30 19:56:30.754722 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:30.754450    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b24a1a-6f55488c68-pk5m5"
Oct 30 19:56:30.776805 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:30.776662    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b24a1a-6f55488c68-pk5m5" podStartSLOduration=2.314397289 podStartE2EDuration="2.776643663s" totalImagesPullingTime="462.246374ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:56:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:56:30.774943822 +0000 UTC m=+1466.103061977" watchObservedRunningTime="2026-10-30 19:56:30.776643663 +0000 UTC m=+1466.104761819"
Oct 30 19:56:31.193879 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:56:31.193843    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 19:56:31.728872 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:31.728843    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-c1768b-8558cb48f5-hq4x4"
Oct 30 19:56:33.700733 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:33.700706    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0eeba5-75c6455594-8kbxt"
Oct 30 19:56:36.767912 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:36.767882    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b24a1a-6f55488c68-pk5m5"
Oct 30 19:56:37.394532 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:37.394500    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 19:56:40.210504 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:40.209183    6996 patch_prober.go:29] interesting pod/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:56:40.210504 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:40.209259    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v" podUID="b99ab818-3be9-406e-be18-c0ed26c43bea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:56:40.229593 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:40.229554    6996 patch_prober.go:29] interesting pod/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:56:40.229866 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:40.229834    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw" podUID="e41df835-fccd-45bb-bfca-ca1f81f6261a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:56:51.246442 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:51.246403    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-b7431c-5dbb549c45-hsk6h"]
Oct 30 19:56:51.252604 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:51.252578    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-b7431c-5dbb549c45-hsk6h"
Oct 30 19:56:51.274135 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:51.274105    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-b7431c-5dbb549c45-hsk6h"]
Oct 30 19:56:51.350968 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:51.350937    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w4-b7431c-serving-cert\""
Oct 30 19:56:51.397262 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:51.397227    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80f2d362-3093-42b0-a3c0-c8bf25c43a84-metrics-certs\") pod \"maas-api-e2e-shared-a-w4-b7431c-5dbb549c45-hsk6h\" (UID: \"80f2d362-3093-42b0-a3c0-c8bf25c43a84\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-b7431c-5dbb549c45-hsk6h"
Oct 30 19:56:51.397421 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:51.397353    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7qdz\" (UniqueName: \"kubernetes.io/projected/80f2d362-3093-42b0-a3c0-c8bf25c43a84-kube-api-access-n7qdz\") pod \"maas-api-e2e-shared-a-w4-b7431c-5dbb549c45-hsk6h\" (UID: \"80f2d362-3093-42b0-a3c0-c8bf25c43a84\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-b7431c-5dbb549c45-hsk6h"
Oct 30 19:56:51.397464 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:51.397432    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80f2d362-3093-42b0-a3c0-c8bf25c43a84-maas-api-tls\") pod \"maas-api-e2e-shared-a-w4-b7431c-5dbb549c45-hsk6h\" (UID: \"80f2d362-3093-42b0-a3c0-c8bf25c43a84\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-b7431c-5dbb549c45-hsk6h"
Oct 30 19:56:51.397577 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:51.397560    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80f2d362-3093-42b0-a3c0-c8bf25c43a84-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w4-b7431c-5dbb549c45-hsk6h\" (UID: \"80f2d362-3093-42b0-a3c0-c8bf25c43a84\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-b7431c-5dbb549c45-hsk6h"
Oct 30 19:56:51.499070 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:51.498970    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80f2d362-3093-42b0-a3c0-c8bf25c43a84-metrics-certs\") pod \"maas-api-e2e-shared-a-w4-b7431c-5dbb549c45-hsk6h\" (UID: \"80f2d362-3093-42b0-a3c0-c8bf25c43a84\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-b7431c-5dbb549c45-hsk6h"
Oct 30 19:56:51.499693 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:51.499665    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7qdz\" (UniqueName: \"kubernetes.io/projected/80f2d362-3093-42b0-a3c0-c8bf25c43a84-kube-api-access-n7qdz\") pod \"maas-api-e2e-shared-a-w4-b7431c-5dbb549c45-hsk6h\" (UID: \"80f2d362-3093-42b0-a3c0-c8bf25c43a84\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-b7431c-5dbb549c45-hsk6h"
Oct 30 19:56:51.500786 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:51.500750    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80f2d362-3093-42b0-a3c0-c8bf25c43a84-maas-api-tls\") pod \"maas-api-e2e-shared-a-w4-b7431c-5dbb549c45-hsk6h\" (UID: \"80f2d362-3093-42b0-a3c0-c8bf25c43a84\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-b7431c-5dbb549c45-hsk6h"
Oct 30 19:56:51.501212 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:51.501191    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80f2d362-3093-42b0-a3c0-c8bf25c43a84-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w4-b7431c-5dbb549c45-hsk6h\" (UID: \"80f2d362-3093-42b0-a3c0-c8bf25c43a84\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-b7431c-5dbb549c45-hsk6h"
Oct 30 19:56:51.502196 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:56:51.501050    6996 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-b7431c-serving-cert: secret "maas-api-e2e-shared-a-w4-b7431c-serving-cert" not found
Oct 30 19:56:51.502428 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:56:51.502412    6996 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80f2d362-3093-42b0-a3c0-c8bf25c43a84-maas-api-tls podName:80f2d362-3093-42b0-a3c0-c8bf25c43a84 nodeName:}" failed. No retries permitted until 2026-10-30 19:56:52.002389098 +0000 UTC m=+1487.330507231 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/80f2d362-3093-42b0-a3c0-c8bf25c43a84-maas-api-tls") pod "maas-api-e2e-shared-a-w4-b7431c-5dbb549c45-hsk6h" (UID: "80f2d362-3093-42b0-a3c0-c8bf25c43a84") : secret "maas-api-e2e-shared-a-w4-b7431c-serving-cert" not found
Oct 30 19:56:51.502838 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:51.502020    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80f2d362-3093-42b0-a3c0-c8bf25c43a84-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w4-b7431c-5dbb549c45-hsk6h\" (UID: \"80f2d362-3093-42b0-a3c0-c8bf25c43a84\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-b7431c-5dbb549c45-hsk6h"
Oct 30 19:56:51.518502 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:51.516403    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80f2d362-3093-42b0-a3c0-c8bf25c43a84-metrics-certs\") pod \"maas-api-e2e-shared-a-w4-b7431c-5dbb549c45-hsk6h\" (UID: \"80f2d362-3093-42b0-a3c0-c8bf25c43a84\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-b7431c-5dbb549c45-hsk6h"
Oct 30 19:56:51.520258 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:51.520227    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7qdz\" (UniqueName: \"kubernetes.io/projected/80f2d362-3093-42b0-a3c0-c8bf25c43a84-kube-api-access-n7qdz\") pod \"maas-api-e2e-shared-a-w4-b7431c-5dbb549c45-hsk6h\" (UID: \"80f2d362-3093-42b0-a3c0-c8bf25c43a84\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-b7431c-5dbb549c45-hsk6h"
Oct 30 19:56:52.015122 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:52.015080    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80f2d362-3093-42b0-a3c0-c8bf25c43a84-maas-api-tls\") pod \"maas-api-e2e-shared-a-w4-b7431c-5dbb549c45-hsk6h\" (UID: \"80f2d362-3093-42b0-a3c0-c8bf25c43a84\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-b7431c-5dbb549c45-hsk6h"
Oct 30 19:56:52.019445 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:52.019408    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80f2d362-3093-42b0-a3c0-c8bf25c43a84-maas-api-tls\") pod \"maas-api-e2e-shared-a-w4-b7431c-5dbb549c45-hsk6h\" (UID: \"80f2d362-3093-42b0-a3c0-c8bf25c43a84\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-b7431c-5dbb549c45-hsk6h"
Oct 30 19:56:52.185653 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:52.185617    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-b7431c-5dbb549c45-hsk6h"
Oct 30 19:56:52.488851 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:52.488815    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-b7431c-5dbb549c45-hsk6h"]
Oct 30 19:56:52.491288 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:56:52.491259    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80f2d362_3093_42b0_a3c0_c8bf25c43a84.slice/crio-6f68858ec028d1c2f25b549737ea3ce4318fb6ef7aaa6eb4070a10878f08ceae WatchSource:0}: Error finding container 6f68858ec028d1c2f25b549737ea3ce4318fb6ef7aaa6eb4070a10878f08ceae: Status 404 returned error can't find the container with id 6f68858ec028d1c2f25b549737ea3ce4318fb6ef7aaa6eb4070a10878f08ceae
Oct 30 19:56:53.124702 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:53.124662    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-b7431c-5dbb549c45-hsk6h" event={"ID":"80f2d362-3093-42b0-a3c0-c8bf25c43a84","Type":"ContainerStarted","Data":"3de50dc33a2c987755a08c5085df845677106e6247f55677ea258253cce66173"}
Oct 30 19:56:53.124874 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:53.124736    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-b7431c-5dbb549c45-hsk6h" event={"ID":"80f2d362-3093-42b0-a3c0-c8bf25c43a84","Type":"ContainerStarted","Data":"6f68858ec028d1c2f25b549737ea3ce4318fb6ef7aaa6eb4070a10878f08ceae"}
Oct 30 19:56:53.124874 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:53.124859    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-b7431c-5dbb549c45-hsk6h"
Oct 30 19:56:53.146154 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:53.146108    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-b7431c-5dbb549c45-hsk6h" podStartSLOduration=1.717103289 podStartE2EDuration="2.146095561s" totalImagesPullingTime="428.992272ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:56:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:56:53.143615795 +0000 UTC m=+1488.471733949" watchObservedRunningTime="2026-10-30 19:56:53.146095561 +0000 UTC m=+1488.474213714"
Oct 30 19:56:59.138840 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:56:59.138811    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-b7431c-5dbb549c45-hsk6h"
Oct 30 19:57:01.190993 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:57:01.190956    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 19:57:04.623381 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:04.623343    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-54d4c4-58c99d6b5d-v5j9h"]
Oct 30 19:57:04.628380 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:04.628341    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-54d4c4-58c99d6b5d-v5j9h"
Oct 30 19:57:04.631724 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:04.631687    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-54d4c4-serving-cert\""
Oct 30 19:57:04.641679 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:04.641655    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-54d4c4-58c99d6b5d-v5j9h"]
Oct 30 19:57:04.743689 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:04.743502    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjpvg\" (UniqueName: \"kubernetes.io/projected/4b366fcc-1688-4082-bb67-b2f05afdd019-kube-api-access-xjpvg\") pod \"maas-api-e2e-shared-b-w2-54d4c4-58c99d6b5d-v5j9h\" (UID: \"4b366fcc-1688-4082-bb67-b2f05afdd019\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-54d4c4-58c99d6b5d-v5j9h"
Oct 30 19:57:04.743689 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:04.743642    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b366fcc-1688-4082-bb67-b2f05afdd019-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-54d4c4-58c99d6b5d-v5j9h\" (UID: \"4b366fcc-1688-4082-bb67-b2f05afdd019\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-54d4c4-58c99d6b5d-v5j9h"
Oct 30 19:57:04.744436 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:04.743865    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b366fcc-1688-4082-bb67-b2f05afdd019-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-54d4c4-58c99d6b5d-v5j9h\" (UID: \"4b366fcc-1688-4082-bb67-b2f05afdd019\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-54d4c4-58c99d6b5d-v5j9h"
Oct 30 19:57:04.744436 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:04.743892    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b366fcc-1688-4082-bb67-b2f05afdd019-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-54d4c4-58c99d6b5d-v5j9h\" (UID: \"4b366fcc-1688-4082-bb67-b2f05afdd019\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-54d4c4-58c99d6b5d-v5j9h"
Oct 30 19:57:04.845101 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:04.845056    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b366fcc-1688-4082-bb67-b2f05afdd019-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-54d4c4-58c99d6b5d-v5j9h\" (UID: \"4b366fcc-1688-4082-bb67-b2f05afdd019\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-54d4c4-58c99d6b5d-v5j9h"
Oct 30 19:57:04.845101 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:04.845099    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b366fcc-1688-4082-bb67-b2f05afdd019-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-54d4c4-58c99d6b5d-v5j9h\" (UID: \"4b366fcc-1688-4082-bb67-b2f05afdd019\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-54d4c4-58c99d6b5d-v5j9h"
Oct 30 19:57:04.845371 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:04.845185    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjpvg\" (UniqueName: \"kubernetes.io/projected/4b366fcc-1688-4082-bb67-b2f05afdd019-kube-api-access-xjpvg\") pod \"maas-api-e2e-shared-b-w2-54d4c4-58c99d6b5d-v5j9h\" (UID: \"4b366fcc-1688-4082-bb67-b2f05afdd019\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-54d4c4-58c99d6b5d-v5j9h"
Oct 30 19:57:04.845371 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:04.845289    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b366fcc-1688-4082-bb67-b2f05afdd019-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-54d4c4-58c99d6b5d-v5j9h\" (UID: \"4b366fcc-1688-4082-bb67-b2f05afdd019\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-54d4c4-58c99d6b5d-v5j9h"
Oct 30 19:57:04.845889 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:04.845860    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b366fcc-1688-4082-bb67-b2f05afdd019-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-54d4c4-58c99d6b5d-v5j9h\" (UID: \"4b366fcc-1688-4082-bb67-b2f05afdd019\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-54d4c4-58c99d6b5d-v5j9h"
Oct 30 19:57:04.847610 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:04.847583    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-54d4c4-serving-cert\""
Oct 30 19:57:04.847986 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:04.847961    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b366fcc-1688-4082-bb67-b2f05afdd019-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-54d4c4-58c99d6b5d-v5j9h\" (UID: \"4b366fcc-1688-4082-bb67-b2f05afdd019\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-54d4c4-58c99d6b5d-v5j9h"
Oct 30 19:57:04.858114 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:04.858087    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b366fcc-1688-4082-bb67-b2f05afdd019-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-54d4c4-58c99d6b5d-v5j9h\" (UID: \"4b366fcc-1688-4082-bb67-b2f05afdd019\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-54d4c4-58c99d6b5d-v5j9h"
Oct 30 19:57:04.858247 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:04.858144    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjpvg\" (UniqueName: \"kubernetes.io/projected/4b366fcc-1688-4082-bb67-b2f05afdd019-kube-api-access-xjpvg\") pod \"maas-api-e2e-shared-b-w2-54d4c4-58c99d6b5d-v5j9h\" (UID: \"4b366fcc-1688-4082-bb67-b2f05afdd019\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-54d4c4-58c99d6b5d-v5j9h"
Oct 30 19:57:04.951765 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:04.951699    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-54d4c4-58c99d6b5d-v5j9h"
Oct 30 19:57:05.237334 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:05.237304    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-54d4c4-58c99d6b5d-v5j9h"]
Oct 30 19:57:05.238212 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:57:05.238188    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b366fcc_1688_4082_bb67_b2f05afdd019.slice/crio-3174097a29ddae621bc8c89748445544d49f0a944cc945e1523f1c3365a3b3c2 WatchSource:0}: Error finding container 3174097a29ddae621bc8c89748445544d49f0a944cc945e1523f1c3365a3b3c2: Status 404 returned error can't find the container with id 3174097a29ddae621bc8c89748445544d49f0a944cc945e1523f1c3365a3b3c2
Oct 30 19:57:05.317965 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:05.317923    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-54d4c4-58c99d6b5d-v5j9h" event={"ID":"4b366fcc-1688-4082-bb67-b2f05afdd019","Type":"ContainerStarted","Data":"3174097a29ddae621bc8c89748445544d49f0a944cc945e1523f1c3365a3b3c2"}
Oct 30 19:57:05.871881 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:05.866034    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-779fa3-56465c7cdd-wvqr2"]
Oct 30 19:57:05.872884 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:05.872815    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-779fa3-56465c7cdd-wvqr2"
Oct 30 19:57:05.882580 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:05.882468    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-779fa3-serving-cert\""
Oct 30 19:57:05.885885 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:05.885860    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-779fa3-56465c7cdd-wvqr2"]
Oct 30 19:57:05.995214 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:05.995093    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3bdb062c-f6d3-4d06-9c69-b9d0b8e11726-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-779fa3-56465c7cdd-wvqr2\" (UID: \"3bdb062c-f6d3-4d06-9c69-b9d0b8e11726\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-779fa3-56465c7cdd-wvqr2"
Oct 30 19:57:05.995214 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:05.995173    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3bdb062c-f6d3-4d06-9c69-b9d0b8e11726-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-779fa3-56465c7cdd-wvqr2\" (UID: \"3bdb062c-f6d3-4d06-9c69-b9d0b8e11726\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-779fa3-56465c7cdd-wvqr2"
Oct 30 19:57:05.995474 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:05.995236    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrs67\" (UniqueName: \"kubernetes.io/projected/3bdb062c-f6d3-4d06-9c69-b9d0b8e11726-kube-api-access-nrs67\") pod \"maas-api-e2e-shared-b-w3-779fa3-56465c7cdd-wvqr2\" (UID: \"3bdb062c-f6d3-4d06-9c69-b9d0b8e11726\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-779fa3-56465c7cdd-wvqr2"
Oct 30 19:57:05.995474 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:05.995307    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3bdb062c-f6d3-4d06-9c69-b9d0b8e11726-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-779fa3-56465c7cdd-wvqr2\" (UID: \"3bdb062c-f6d3-4d06-9c69-b9d0b8e11726\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-779fa3-56465c7cdd-wvqr2"
Oct 30 19:57:06.096376 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:06.096340    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrs67\" (UniqueName: \"kubernetes.io/projected/3bdb062c-f6d3-4d06-9c69-b9d0b8e11726-kube-api-access-nrs67\") pod \"maas-api-e2e-shared-b-w3-779fa3-56465c7cdd-wvqr2\" (UID: \"3bdb062c-f6d3-4d06-9c69-b9d0b8e11726\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-779fa3-56465c7cdd-wvqr2"
Oct 30 19:57:06.096564 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:06.096420    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3bdb062c-f6d3-4d06-9c69-b9d0b8e11726-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-779fa3-56465c7cdd-wvqr2\" (UID: \"3bdb062c-f6d3-4d06-9c69-b9d0b8e11726\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-779fa3-56465c7cdd-wvqr2"
Oct 30 19:57:06.096730 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:06.096713    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3bdb062c-f6d3-4d06-9c69-b9d0b8e11726-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-779fa3-56465c7cdd-wvqr2\" (UID: \"3bdb062c-f6d3-4d06-9c69-b9d0b8e11726\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-779fa3-56465c7cdd-wvqr2"
Oct 30 19:57:06.096771 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:06.096751    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3bdb062c-f6d3-4d06-9c69-b9d0b8e11726-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-779fa3-56465c7cdd-wvqr2\" (UID: \"3bdb062c-f6d3-4d06-9c69-b9d0b8e11726\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-779fa3-56465c7cdd-wvqr2"
Oct 30 19:57:06.097493 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:06.097463    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3bdb062c-f6d3-4d06-9c69-b9d0b8e11726-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-779fa3-56465c7cdd-wvqr2\" (UID: \"3bdb062c-f6d3-4d06-9c69-b9d0b8e11726\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-779fa3-56465c7cdd-wvqr2"
Oct 30 19:57:06.099627 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:06.099599    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3bdb062c-f6d3-4d06-9c69-b9d0b8e11726-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-779fa3-56465c7cdd-wvqr2\" (UID: \"3bdb062c-f6d3-4d06-9c69-b9d0b8e11726\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-779fa3-56465c7cdd-wvqr2"
Oct 30 19:57:06.099742 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:06.099644    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3bdb062c-f6d3-4d06-9c69-b9d0b8e11726-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-779fa3-56465c7cdd-wvqr2\" (UID: \"3bdb062c-f6d3-4d06-9c69-b9d0b8e11726\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-779fa3-56465c7cdd-wvqr2"
Oct 30 19:57:06.110834 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:06.110812    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrs67\" (UniqueName: \"kubernetes.io/projected/3bdb062c-f6d3-4d06-9c69-b9d0b8e11726-kube-api-access-nrs67\") pod \"maas-api-e2e-shared-b-w3-779fa3-56465c7cdd-wvqr2\" (UID: \"3bdb062c-f6d3-4d06-9c69-b9d0b8e11726\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-779fa3-56465c7cdd-wvqr2"
Oct 30 19:57:06.217896 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:06.217864    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-779fa3-56465c7cdd-wvqr2"
Oct 30 19:57:06.328564 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:06.328521    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-54d4c4-58c99d6b5d-v5j9h" event={"ID":"4b366fcc-1688-4082-bb67-b2f05afdd019","Type":"ContainerStarted","Data":"ba0ab181ce16ddbc61e5cd758944ac95a5875f97f76034b94d044c3425e5064d"}
Oct 30 19:57:06.328849 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:06.328829    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-54d4c4-58c99d6b5d-v5j9h"
Oct 30 19:57:06.354471 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:06.354421    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-54d4c4-58c99d6b5d-v5j9h" podStartSLOduration=1.9292201709999999 podStartE2EDuration="2.354407149s" totalImagesPullingTime="425.186978ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:57:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:57:06.350725365 +0000 UTC m=+1501.678843517" watchObservedRunningTime="2026-10-30 19:57:06.354407149 +0000 UTC m=+1501.682525301"
Oct 30 19:57:06.522823 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:06.522640    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-779fa3-56465c7cdd-wvqr2"]
Oct 30 19:57:06.528460 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:57:06.528431    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bdb062c_f6d3_4d06_9c69_b9d0b8e11726.slice/crio-cd6501746bf67cdd78c9feccc071ec4ce76d3185ba4b424398bfaafb29f609ed WatchSource:0}: Error finding container cd6501746bf67cdd78c9feccc071ec4ce76d3185ba4b424398bfaafb29f609ed: Status 404 returned error can't find the container with id cd6501746bf67cdd78c9feccc071ec4ce76d3185ba4b424398bfaafb29f609ed
Oct 30 19:57:06.821126 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:06.821084    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 19:57:07.343308 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:07.343220    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-779fa3-56465c7cdd-wvqr2" event={"ID":"3bdb062c-f6d3-4d06-9c69-b9d0b8e11726","Type":"ContainerStarted","Data":"93a1b24e2ea3ec2537f1b826706cd6cbcf00856ecdfefec2c01ac71fa6054358"}
Oct 30 19:57:07.343308 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:07.343296    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-779fa3-56465c7cdd-wvqr2" event={"ID":"3bdb062c-f6d3-4d06-9c69-b9d0b8e11726","Type":"ContainerStarted","Data":"cd6501746bf67cdd78c9feccc071ec4ce76d3185ba4b424398bfaafb29f609ed"}
Oct 30 19:57:07.343880 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:07.343333    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-779fa3-56465c7cdd-wvqr2"
Oct 30 19:57:07.380760 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:07.380709    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-779fa3-56465c7cdd-wvqr2" podStartSLOduration=1.865693337 podStartE2EDuration="2.38069636s" totalImagesPullingTime="515.003023ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:57:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:57:07.371647135 +0000 UTC m=+1502.699765301" watchObservedRunningTime="2026-10-30 19:57:07.38069636 +0000 UTC m=+1502.708814507"
Oct 30 19:57:12.348538 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:12.348464    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-54d4c4-58c99d6b5d-v5j9h"
Oct 30 19:57:14.365375 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:14.365341    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-779fa3-56465c7cdd-wvqr2"
Oct 30 19:57:16.506130 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:16.505979    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-c1768b-8558cb48f5-hq4x4"]
Oct 30 19:57:16.508709 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:16.506622    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-c1768b-8558cb48f5-hq4x4" podUID="da82ca39-fdea-4fa0-bde2-6dae317029b3" containerName="maas-api" containerID="cri-o://2d0e59e82597853f28dd8d45f070cdc9940c9af054f76da105fb8026b80c359a" gracePeriod=30
Oct 30 19:57:16.897924 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:16.897894    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-c1768b-8558cb48f5-hq4x4"
Oct 30 19:57:17.023805 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:17.023773    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/da82ca39-fdea-4fa0-bde2-6dae317029b3-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da82ca39-fdea-4fa0-bde2-6dae317029b3-openshift-service-ca\") pod \"da82ca39-fdea-4fa0-bde2-6dae317029b3\" (UID: \"da82ca39-fdea-4fa0-bde2-6dae317029b3\") "
Oct 30 19:57:17.023979 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:17.023815    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/da82ca39-fdea-4fa0-bde2-6dae317029b3-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da82ca39-fdea-4fa0-bde2-6dae317029b3-maas-api-tls\") pod \"da82ca39-fdea-4fa0-bde2-6dae317029b3\" (UID: \"da82ca39-fdea-4fa0-bde2-6dae317029b3\") "
Oct 30 19:57:17.023979 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:17.023900    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/da82ca39-fdea-4fa0-bde2-6dae317029b3-metrics-certs\" (UniqueName: \"kubernetes.io/secret/da82ca39-fdea-4fa0-bde2-6dae317029b3-metrics-certs\") pod \"da82ca39-fdea-4fa0-bde2-6dae317029b3\" (UID: \"da82ca39-fdea-4fa0-bde2-6dae317029b3\") "
Oct 30 19:57:17.023979 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:17.023962    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/da82ca39-fdea-4fa0-bde2-6dae317029b3-kube-api-access-wpv7m\" (UniqueName: \"kubernetes.io/projected/da82ca39-fdea-4fa0-bde2-6dae317029b3-kube-api-access-wpv7m\") pod \"da82ca39-fdea-4fa0-bde2-6dae317029b3\" (UID: \"da82ca39-fdea-4fa0-bde2-6dae317029b3\") "
Oct 30 19:57:17.024227 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:17.024197    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da82ca39-fdea-4fa0-bde2-6dae317029b3-openshift-service-ca" pod "da82ca39-fdea-4fa0-bde2-6dae317029b3" (UID: "da82ca39-fdea-4fa0-bde2-6dae317029b3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:57:17.024703 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:17.024676    6996 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da82ca39-fdea-4fa0-bde2-6dae317029b3-openshift-service-ca\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:17.026077 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:17.026046    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da82ca39-fdea-4fa0-bde2-6dae317029b3-kube-api-access-wpv7m" pod "da82ca39-fdea-4fa0-bde2-6dae317029b3" (UID: "da82ca39-fdea-4fa0-bde2-6dae317029b3"). InnerVolumeSpecName "kube-api-access-wpv7m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:57:17.026077 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:17.026059    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da82ca39-fdea-4fa0-bde2-6dae317029b3-metrics-certs" pod "da82ca39-fdea-4fa0-bde2-6dae317029b3" (UID: "da82ca39-fdea-4fa0-bde2-6dae317029b3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:57:17.026217 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:17.026138    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da82ca39-fdea-4fa0-bde2-6dae317029b3-maas-api-tls" pod "da82ca39-fdea-4fa0-bde2-6dae317029b3" (UID: "da82ca39-fdea-4fa0-bde2-6dae317029b3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:57:17.125607 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:17.125512    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wpv7m\" (UniqueName: \"kubernetes.io/projected/da82ca39-fdea-4fa0-bde2-6dae317029b3-kube-api-access-wpv7m\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:17.125607 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:17.125543    6996 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da82ca39-fdea-4fa0-bde2-6dae317029b3-maas-api-tls\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:17.125607 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:17.125554    6996 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da82ca39-fdea-4fa0-bde2-6dae317029b3-metrics-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:17.413917 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:17.413828    6996 generic.go:356] "Generic (PLEG): container finished" podID="da82ca39-fdea-4fa0-bde2-6dae317029b3" containerID="2d0e59e82597853f28dd8d45f070cdc9940c9af054f76da105fb8026b80c359a" exitCode=0
Oct 30 19:57:17.413917 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:17.413881    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-c1768b-8558cb48f5-hq4x4"
Oct 30 19:57:17.414144 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:17.413908    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-c1768b-8558cb48f5-hq4x4" event={"ID":"da82ca39-fdea-4fa0-bde2-6dae317029b3","Type":"ContainerDied","Data":"2d0e59e82597853f28dd8d45f070cdc9940c9af054f76da105fb8026b80c359a"}
Oct 30 19:57:17.414144 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:17.413961    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-c1768b-8558cb48f5-hq4x4" event={"ID":"da82ca39-fdea-4fa0-bde2-6dae317029b3","Type":"ContainerDied","Data":"07752cbac81d4940c90164ede864737c2b1027ea603480e866be663e61c13768"}
Oct 30 19:57:17.414144 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:17.413982    6996 scope.go:122] "RemoveContainer" containerID="2d0e59e82597853f28dd8d45f070cdc9940c9af054f76da105fb8026b80c359a"
Oct 30 19:57:17.425644 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:17.425629    6996 scope.go:122] "RemoveContainer" containerID="2d0e59e82597853f28dd8d45f070cdc9940c9af054f76da105fb8026b80c359a"
Oct 30 19:57:17.425929 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:57:17.425910    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d0e59e82597853f28dd8d45f070cdc9940c9af054f76da105fb8026b80c359a\": container with ID starting with 2d0e59e82597853f28dd8d45f070cdc9940c9af054f76da105fb8026b80c359a not found: ID does not exist" containerID="2d0e59e82597853f28dd8d45f070cdc9940c9af054f76da105fb8026b80c359a"
Oct 30 19:57:17.425998 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:17.425943    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d0e59e82597853f28dd8d45f070cdc9940c9af054f76da105fb8026b80c359a"} err="failed to get container status \"2d0e59e82597853f28dd8d45f070cdc9940c9af054f76da105fb8026b80c359a\": rpc error: code = NotFound desc = could not find container \"2d0e59e82597853f28dd8d45f070cdc9940c9af054f76da105fb8026b80c359a\": container with ID starting with 2d0e59e82597853f28dd8d45f070cdc9940c9af054f76da105fb8026b80c359a not found: ID does not exist"
Oct 30 19:57:17.438930 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:17.438906    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-c1768b-8558cb48f5-hq4x4"]
Oct 30 19:57:17.446210 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:17.446180    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-c1768b-8558cb48f5-hq4x4"]
Oct 30 19:57:18.793004 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:18.792966    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="da82ca39-fdea-4fa0-bde2-6dae317029b3" path="/var/lib/kubelet/pods/da82ca39-fdea-4fa0-bde2-6dae317029b3/volumes"
Oct 30 19:57:19.724649 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:19.724606    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-c1768b-8558cb48f5-hq4x4" podUID="da82ca39-fdea-4fa0-bde2-6dae317029b3" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.137:8443/health\": context deadline exceeded"
Oct 30 19:57:27.231167 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:27.231126    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-7cbcbb-6b88874567-48rg5"]
Oct 30 19:57:27.232185 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:27.232163    6996 cpu_manager.go:414] "containerMap: removing container" podUID="da82ca39-fdea-4fa0-bde2-6dae317029b3" containerName="maas-api"
Oct 30 19:57:27.232257 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:27.232189    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="da82ca39-fdea-4fa0-bde2-6dae317029b3" containerName="maas-api"
Oct 30 19:57:27.232413 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:27.232400    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="da82ca39-fdea-4fa0-bde2-6dae317029b3" containerName="maas-api"
Oct 30 19:57:27.236331 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:27.236307    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-7cbcbb-6b88874567-48rg5"
Oct 30 19:57:27.242952 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:27.242919    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w4-7cbcbb-serving-cert\""
Oct 30 19:57:27.271246 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:27.271219    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-7cbcbb-6b88874567-48rg5"]
Oct 30 19:57:27.365983 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:27.365942    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d38b232-9550-4272-8174-7c457b1f4b38-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w4-7cbcbb-6b88874567-48rg5\" (UID: \"8d38b232-9550-4272-8174-7c457b1f4b38\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-7cbcbb-6b88874567-48rg5"
Oct 30 19:57:27.366163 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:27.366070    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8d38b232-9550-4272-8174-7c457b1f4b38-maas-api-tls\") pod \"maas-api-e2e-shared-b-w4-7cbcbb-6b88874567-48rg5\" (UID: \"8d38b232-9550-4272-8174-7c457b1f4b38\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-7cbcbb-6b88874567-48rg5"
Oct 30 19:57:27.366163 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:27.366138    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9mzn\" (UniqueName: \"kubernetes.io/projected/8d38b232-9550-4272-8174-7c457b1f4b38-kube-api-access-c9mzn\") pod \"maas-api-e2e-shared-b-w4-7cbcbb-6b88874567-48rg5\" (UID: \"8d38b232-9550-4272-8174-7c457b1f4b38\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-7cbcbb-6b88874567-48rg5"
Oct 30 19:57:27.366562 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:27.366446    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d38b232-9550-4272-8174-7c457b1f4b38-metrics-certs\") pod \"maas-api-e2e-shared-b-w4-7cbcbb-6b88874567-48rg5\" (UID: \"8d38b232-9550-4272-8174-7c457b1f4b38\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-7cbcbb-6b88874567-48rg5"
Oct 30 19:57:27.470221 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:27.470190    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d38b232-9550-4272-8174-7c457b1f4b38-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w4-7cbcbb-6b88874567-48rg5\" (UID: \"8d38b232-9550-4272-8174-7c457b1f4b38\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-7cbcbb-6b88874567-48rg5"
Oct 30 19:57:27.470415 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:27.470241    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8d38b232-9550-4272-8174-7c457b1f4b38-maas-api-tls\") pod \"maas-api-e2e-shared-b-w4-7cbcbb-6b88874567-48rg5\" (UID: \"8d38b232-9550-4272-8174-7c457b1f4b38\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-7cbcbb-6b88874567-48rg5"
Oct 30 19:57:27.470415 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:27.470274    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9mzn\" (UniqueName: \"kubernetes.io/projected/8d38b232-9550-4272-8174-7c457b1f4b38-kube-api-access-c9mzn\") pod \"maas-api-e2e-shared-b-w4-7cbcbb-6b88874567-48rg5\" (UID: \"8d38b232-9550-4272-8174-7c457b1f4b38\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-7cbcbb-6b88874567-48rg5"
Oct 30 19:57:27.470415 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:27.470348    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d38b232-9550-4272-8174-7c457b1f4b38-metrics-certs\") pod \"maas-api-e2e-shared-b-w4-7cbcbb-6b88874567-48rg5\" (UID: \"8d38b232-9550-4272-8174-7c457b1f4b38\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-7cbcbb-6b88874567-48rg5"
Oct 30 19:57:27.470962 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:27.470931    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d38b232-9550-4272-8174-7c457b1f4b38-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w4-7cbcbb-6b88874567-48rg5\" (UID: \"8d38b232-9550-4272-8174-7c457b1f4b38\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-7cbcbb-6b88874567-48rg5"
Oct 30 19:57:27.472842 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:27.472822    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d38b232-9550-4272-8174-7c457b1f4b38-metrics-certs\") pod \"maas-api-e2e-shared-b-w4-7cbcbb-6b88874567-48rg5\" (UID: \"8d38b232-9550-4272-8174-7c457b1f4b38\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-7cbcbb-6b88874567-48rg5"
Oct 30 19:57:27.472923 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:27.472887    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8d38b232-9550-4272-8174-7c457b1f4b38-maas-api-tls\") pod \"maas-api-e2e-shared-b-w4-7cbcbb-6b88874567-48rg5\" (UID: \"8d38b232-9550-4272-8174-7c457b1f4b38\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-7cbcbb-6b88874567-48rg5"
Oct 30 19:57:27.486553 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:27.486499    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9mzn\" (UniqueName: \"kubernetes.io/projected/8d38b232-9550-4272-8174-7c457b1f4b38-kube-api-access-c9mzn\") pod \"maas-api-e2e-shared-b-w4-7cbcbb-6b88874567-48rg5\" (UID: \"8d38b232-9550-4272-8174-7c457b1f4b38\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-7cbcbb-6b88874567-48rg5"
Oct 30 19:57:27.553204 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:27.553157    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-7cbcbb-6b88874567-48rg5"
Oct 30 19:57:27.887789 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:27.887757    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-7cbcbb-6b88874567-48rg5"]
Oct 30 19:57:27.888890 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:57:27.888860    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d38b232_9550_4272_8174_7c457b1f4b38.slice/crio-0e7f30b10952560e49963a86d87be704de9e0ad292b41c06e4c9660be5c05473 WatchSource:0}: Error finding container 0e7f30b10952560e49963a86d87be704de9e0ad292b41c06e4c9660be5c05473: Status 404 returned error can't find the container with id 0e7f30b10952560e49963a86d87be704de9e0ad292b41c06e4c9660be5c05473
Oct 30 19:57:28.518494 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:28.518447    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-7cbcbb-6b88874567-48rg5" event={"ID":"8d38b232-9550-4272-8174-7c457b1f4b38","Type":"ContainerStarted","Data":"604a1503ebd138f50c5602c813c6dc05ea9b2ac0ed643cb881e30629e911c968"}
Oct 30 19:57:28.518867 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:28.518509    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-7cbcbb-6b88874567-48rg5" event={"ID":"8d38b232-9550-4272-8174-7c457b1f4b38","Type":"ContainerStarted","Data":"0e7f30b10952560e49963a86d87be704de9e0ad292b41c06e4c9660be5c05473"}
Oct 30 19:57:28.518867 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:28.518599    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-7cbcbb-6b88874567-48rg5"
Oct 30 19:57:28.546945 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:28.546901    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-7cbcbb-6b88874567-48rg5" podStartSLOduration=1.043945628 podStartE2EDuration="1.546886311s" totalImagesPullingTime="502.940683ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:57:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:57:28.544874689 +0000 UTC m=+1523.872992852" watchObservedRunningTime="2026-10-30 19:57:28.546886311 +0000 UTC m=+1523.875004483"
Oct 30 19:57:31.191661 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:57:31.191622    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 19:57:34.530981 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:34.530952    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-7cbcbb-6b88874567-48rg5"
Oct 30 19:57:39.156414 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:39.156376    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-981ee5-dbb54d47d-sr5mp"]
Oct 30 19:57:39.165452 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:39.162471    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-981ee5-dbb54d47d-sr5mp"
Oct 30 19:57:39.173446 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:39.173418    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w1-981ee5-serving-cert\""
Oct 30 19:57:39.175771 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:39.174386    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-981ee5-dbb54d47d-sr5mp"]
Oct 30 19:57:39.262105 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:39.262067    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/035ed3f1-789b-4a62-80e6-3f8841905629-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-981ee5-dbb54d47d-sr5mp\" (UID: \"035ed3f1-789b-4a62-80e6-3f8841905629\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-981ee5-dbb54d47d-sr5mp"
Oct 30 19:57:39.262283 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:39.262244    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/035ed3f1-789b-4a62-80e6-3f8841905629-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-981ee5-dbb54d47d-sr5mp\" (UID: \"035ed3f1-789b-4a62-80e6-3f8841905629\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-981ee5-dbb54d47d-sr5mp"
Oct 30 19:57:39.262432 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:39.262415    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f95fg\" (UniqueName: \"kubernetes.io/projected/035ed3f1-789b-4a62-80e6-3f8841905629-kube-api-access-f95fg\") pod \"maas-api-e2e-shared-b-w1-981ee5-dbb54d47d-sr5mp\" (UID: \"035ed3f1-789b-4a62-80e6-3f8841905629\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-981ee5-dbb54d47d-sr5mp"
Oct 30 19:57:39.262515 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:39.262470    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/035ed3f1-789b-4a62-80e6-3f8841905629-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-981ee5-dbb54d47d-sr5mp\" (UID: \"035ed3f1-789b-4a62-80e6-3f8841905629\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-981ee5-dbb54d47d-sr5mp"
Oct 30 19:57:39.364333 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:39.364298    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/035ed3f1-789b-4a62-80e6-3f8841905629-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-981ee5-dbb54d47d-sr5mp\" (UID: \"035ed3f1-789b-4a62-80e6-3f8841905629\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-981ee5-dbb54d47d-sr5mp"
Oct 30 19:57:39.364549 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:57:39.364444    6996 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-981ee5-serving-cert: secret "maas-api-e2e-shared-b-w1-981ee5-serving-cert" not found
Oct 30 19:57:39.364549 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:39.364468    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f95fg\" (UniqueName: \"kubernetes.io/projected/035ed3f1-789b-4a62-80e6-3f8841905629-kube-api-access-f95fg\") pod \"maas-api-e2e-shared-b-w1-981ee5-dbb54d47d-sr5mp\" (UID: \"035ed3f1-789b-4a62-80e6-3f8841905629\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-981ee5-dbb54d47d-sr5mp"
Oct 30 19:57:39.364549 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:57:39.364537    6996 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/035ed3f1-789b-4a62-80e6-3f8841905629-maas-api-tls podName:035ed3f1-789b-4a62-80e6-3f8841905629 nodeName:}" failed. No retries permitted until 2026-10-30 19:57:39.864521733 +0000 UTC m=+1535.192639877 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/035ed3f1-789b-4a62-80e6-3f8841905629-maas-api-tls") pod "maas-api-e2e-shared-b-w1-981ee5-dbb54d47d-sr5mp" (UID: "035ed3f1-789b-4a62-80e6-3f8841905629") : secret "maas-api-e2e-shared-b-w1-981ee5-serving-cert" not found
Oct 30 19:57:39.364708 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:39.364566    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/035ed3f1-789b-4a62-80e6-3f8841905629-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-981ee5-dbb54d47d-sr5mp\" (UID: \"035ed3f1-789b-4a62-80e6-3f8841905629\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-981ee5-dbb54d47d-sr5mp"
Oct 30 19:57:39.364708 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:39.364600    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/035ed3f1-789b-4a62-80e6-3f8841905629-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-981ee5-dbb54d47d-sr5mp\" (UID: \"035ed3f1-789b-4a62-80e6-3f8841905629\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-981ee5-dbb54d47d-sr5mp"
Oct 30 19:57:39.365263 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:39.365237    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/035ed3f1-789b-4a62-80e6-3f8841905629-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-981ee5-dbb54d47d-sr5mp\" (UID: \"035ed3f1-789b-4a62-80e6-3f8841905629\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-981ee5-dbb54d47d-sr5mp"
Oct 30 19:57:39.367077 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:39.367057    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/035ed3f1-789b-4a62-80e6-3f8841905629-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-981ee5-dbb54d47d-sr5mp\" (UID: \"035ed3f1-789b-4a62-80e6-3f8841905629\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-981ee5-dbb54d47d-sr5mp"
Oct 30 19:57:39.376996 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:39.376974    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f95fg\" (UniqueName: \"kubernetes.io/projected/035ed3f1-789b-4a62-80e6-3f8841905629-kube-api-access-f95fg\") pod \"maas-api-e2e-shared-b-w1-981ee5-dbb54d47d-sr5mp\" (UID: \"035ed3f1-789b-4a62-80e6-3f8841905629\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-981ee5-dbb54d47d-sr5mp"
Oct 30 19:57:39.873176 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:39.873138    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/035ed3f1-789b-4a62-80e6-3f8841905629-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-981ee5-dbb54d47d-sr5mp\" (UID: \"035ed3f1-789b-4a62-80e6-3f8841905629\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-981ee5-dbb54d47d-sr5mp"
Oct 30 19:57:39.881094 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:39.881061    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/035ed3f1-789b-4a62-80e6-3f8841905629-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-981ee5-dbb54d47d-sr5mp\" (UID: \"035ed3f1-789b-4a62-80e6-3f8841905629\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-981ee5-dbb54d47d-sr5mp"
Oct 30 19:57:40.095265 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:40.095220    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-981ee5-dbb54d47d-sr5mp"
Oct 30 19:57:40.402009 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:40.401946    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-981ee5-dbb54d47d-sr5mp"]
Oct 30 19:57:40.403591 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:57:40.403560    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod035ed3f1_789b_4a62_80e6_3f8841905629.slice/crio-8434220309ba767e180029f6c766c3dbc927f810184a6f53c22a481572e6df25 WatchSource:0}: Error finding container 8434220309ba767e180029f6c766c3dbc927f810184a6f53c22a481572e6df25: Status 404 returned error can't find the container with id 8434220309ba767e180029f6c766c3dbc927f810184a6f53c22a481572e6df25
Oct 30 19:57:40.630938 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:40.630899    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-981ee5-dbb54d47d-sr5mp" event={"ID":"035ed3f1-789b-4a62-80e6-3f8841905629","Type":"ContainerStarted","Data":"8434220309ba767e180029f6c766c3dbc927f810184a6f53c22a481572e6df25"}
Oct 30 19:57:41.639341 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:41.639302    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-981ee5-dbb54d47d-sr5mp" event={"ID":"035ed3f1-789b-4a62-80e6-3f8841905629","Type":"ContainerStarted","Data":"5873c72003f514dbb67590ea158944ad24705be25e27c36d09dc0ba40b75d41c"}
Oct 30 19:57:41.639755 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:41.639399    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-981ee5-dbb54d47d-sr5mp"
Oct 30 19:57:41.664425 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:41.664377    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-981ee5-dbb54d47d-sr5mp" podStartSLOduration=2.266436059 podStartE2EDuration="2.664361604s" totalImagesPullingTime="397.925545ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:57:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:57:41.661928128 +0000 UTC m=+1536.990046280" watchObservedRunningTime="2026-10-30 19:57:41.664361604 +0000 UTC m=+1536.992479757"
Oct 30 19:57:47.656551 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:57:47.654024    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-981ee5-dbb54d47d-sr5mp"
Oct 30 19:58:01.192005 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:58:01.191963    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 19:58:01.395537 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:01.395498    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 19:58:05.443217 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:05.443172    6996 scope.go:122] "RemoveContainer" containerID="6beab979d1bcd0f90e148c7e98e57ecfd0cf3ac9883777d5933a41a147f0e1c5"
Oct 30 19:58:15.960865 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:15.960825    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-f0e884-6cbfdf5867-g9snm"]
Oct 30 19:58:15.967110 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:15.967080    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-f0e884-6cbfdf5867-g9snm"
Oct 30 19:58:15.976981 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:15.976962    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-f0e884-serving-cert\""
Oct 30 19:58:15.984670 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:15.984641    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-f0e884-6cbfdf5867-g9snm"]
Oct 30 19:58:16.054453 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:16.054419    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/982d0d28-3106-42c2-a300-5a4e56645596-maas-api-tls\") pod \"maas-api-e2e-explicit-f0e884-6cbfdf5867-g9snm\" (UID: \"982d0d28-3106-42c2-a300-5a4e56645596\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-f0e884-6cbfdf5867-g9snm"
Oct 30 19:58:16.054633 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:16.054472    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/982d0d28-3106-42c2-a300-5a4e56645596-openshift-service-ca\") pod \"maas-api-e2e-explicit-f0e884-6cbfdf5867-g9snm\" (UID: \"982d0d28-3106-42c2-a300-5a4e56645596\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-f0e884-6cbfdf5867-g9snm"
Oct 30 19:58:16.054633 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:16.054591    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wh7s4\" (UniqueName: \"kubernetes.io/projected/982d0d28-3106-42c2-a300-5a4e56645596-kube-api-access-wh7s4\") pod \"maas-api-e2e-explicit-f0e884-6cbfdf5867-g9snm\" (UID: \"982d0d28-3106-42c2-a300-5a4e56645596\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-f0e884-6cbfdf5867-g9snm"
Oct 30 19:58:16.054776 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:16.054689    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/982d0d28-3106-42c2-a300-5a4e56645596-metrics-certs\") pod \"maas-api-e2e-explicit-f0e884-6cbfdf5867-g9snm\" (UID: \"982d0d28-3106-42c2-a300-5a4e56645596\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-f0e884-6cbfdf5867-g9snm"
Oct 30 19:58:16.159950 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:16.158630    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/982d0d28-3106-42c2-a300-5a4e56645596-maas-api-tls\") pod \"maas-api-e2e-explicit-f0e884-6cbfdf5867-g9snm\" (UID: \"982d0d28-3106-42c2-a300-5a4e56645596\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-f0e884-6cbfdf5867-g9snm"
Oct 30 19:58:16.159950 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:16.158700    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/982d0d28-3106-42c2-a300-5a4e56645596-openshift-service-ca\") pod \"maas-api-e2e-explicit-f0e884-6cbfdf5867-g9snm\" (UID: \"982d0d28-3106-42c2-a300-5a4e56645596\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-f0e884-6cbfdf5867-g9snm"
Oct 30 19:58:16.159950 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:16.158731    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wh7s4\" (UniqueName: \"kubernetes.io/projected/982d0d28-3106-42c2-a300-5a4e56645596-kube-api-access-wh7s4\") pod \"maas-api-e2e-explicit-f0e884-6cbfdf5867-g9snm\" (UID: \"982d0d28-3106-42c2-a300-5a4e56645596\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-f0e884-6cbfdf5867-g9snm"
Oct 30 19:58:16.159950 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:16.158829    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/982d0d28-3106-42c2-a300-5a4e56645596-metrics-certs\") pod \"maas-api-e2e-explicit-f0e884-6cbfdf5867-g9snm\" (UID: \"982d0d28-3106-42c2-a300-5a4e56645596\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-f0e884-6cbfdf5867-g9snm"
Oct 30 19:58:16.160645 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:16.160615    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/982d0d28-3106-42c2-a300-5a4e56645596-openshift-service-ca\") pod \"maas-api-e2e-explicit-f0e884-6cbfdf5867-g9snm\" (UID: \"982d0d28-3106-42c2-a300-5a4e56645596\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-f0e884-6cbfdf5867-g9snm"
Oct 30 19:58:16.183205 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:16.183166    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wh7s4\" (UniqueName: \"kubernetes.io/projected/982d0d28-3106-42c2-a300-5a4e56645596-kube-api-access-wh7s4\") pod \"maas-api-e2e-explicit-f0e884-6cbfdf5867-g9snm\" (UID: \"982d0d28-3106-42c2-a300-5a4e56645596\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-f0e884-6cbfdf5867-g9snm"
Oct 30 19:58:16.184668 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:16.184643    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/982d0d28-3106-42c2-a300-5a4e56645596-metrics-certs\") pod \"maas-api-e2e-explicit-f0e884-6cbfdf5867-g9snm\" (UID: \"982d0d28-3106-42c2-a300-5a4e56645596\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-f0e884-6cbfdf5867-g9snm"
Oct 30 19:58:16.188068 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:16.187959    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/982d0d28-3106-42c2-a300-5a4e56645596-maas-api-tls\") pod \"maas-api-e2e-explicit-f0e884-6cbfdf5867-g9snm\" (UID: \"982d0d28-3106-42c2-a300-5a4e56645596\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-f0e884-6cbfdf5867-g9snm"
Oct 30 19:58:16.282997 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:16.282908    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-f0e884-6cbfdf5867-g9snm"
Oct 30 19:58:16.590394 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:16.590368    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-f0e884-6cbfdf5867-g9snm"]
Oct 30 19:58:16.593541 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:58:16.593507    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod982d0d28_3106_42c2_a300_5a4e56645596.slice/crio-aaf299c08b7a0b512e824f3f55d663b2a41f050a118f9c3ed1df479a8b199935 WatchSource:0}: Error finding container aaf299c08b7a0b512e824f3f55d663b2a41f050a118f9c3ed1df479a8b199935: Status 404 returned error can't find the container with id aaf299c08b7a0b512e824f3f55d663b2a41f050a118f9c3ed1df479a8b199935
Oct 30 19:58:16.991419 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:16.991325    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-f0e884-6cbfdf5867-g9snm" event={"ID":"982d0d28-3106-42c2-a300-5a4e56645596","Type":"ContainerStarted","Data":"aaf299c08b7a0b512e824f3f55d663b2a41f050a118f9c3ed1df479a8b199935"}
Oct 30 19:58:18.001060 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:18.001020    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-f0e884-6cbfdf5867-g9snm" event={"ID":"982d0d28-3106-42c2-a300-5a4e56645596","Type":"ContainerStarted","Data":"28f4202e3f337c2a0f370c7cdfe60f6d9b9475bf29f7a7c157dfac1e637013d7"}
Oct 30 19:58:18.001540 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:18.001124    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-f0e884-6cbfdf5867-g9snm"
Oct 30 19:58:18.029677 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:18.029628    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-f0e884-6cbfdf5867-g9snm" podStartSLOduration=2.614995332 podStartE2EDuration="3.029614171s" totalImagesPullingTime="414.618839ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:58:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:58:18.026946067 +0000 UTC m=+1573.355064222" watchObservedRunningTime="2026-10-30 19:58:18.029614171 +0000 UTC m=+1573.357732325"
Oct 30 19:58:19.951051 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:19.951016    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs"]
Oct 30 19:58:19.955617 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:19.955596    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs"
Oct 30 19:58:19.967214 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:19.967191    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-779fa3-openshift-default-dockercfg-ccggk\""
Oct 30 19:58:19.994911 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:19.994868    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs"]
Oct 30 19:58:20.120771 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:20.120723    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d58ffd8-ffb2-4751-a622-11b584616c7d-credential-socket\") pod \"e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs\" (UID: \"6d58ffd8-ffb2-4751-a622-11b584616c7d\") " pod="openshift-ingress/e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs"
Oct 30 19:58:20.120956 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:20.120786    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d58ffd8-ffb2-4751-a622-11b584616c7d-istio-token\") pod \"e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs\" (UID: \"6d58ffd8-ffb2-4751-a622-11b584616c7d\") " pod="openshift-ingress/e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs"
Oct 30 19:58:20.120956 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:20.120880    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d58ffd8-ffb2-4751-a622-11b584616c7d-workload-socket\") pod \"e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs\" (UID: \"6d58ffd8-ffb2-4751-a622-11b584616c7d\") " pod="openshift-ingress/e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs"
Oct 30 19:58:20.120956 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:20.120941    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d58ffd8-ffb2-4751-a622-11b584616c7d-istiod-ca-cert\") pod \"e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs\" (UID: \"6d58ffd8-ffb2-4751-a622-11b584616c7d\") " pod="openshift-ingress/e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs"
Oct 30 19:58:20.121135 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:20.121075    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjntn\" (UniqueName: \"kubernetes.io/projected/6d58ffd8-ffb2-4751-a622-11b584616c7d-kube-api-access-tjntn\") pod \"e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs\" (UID: \"6d58ffd8-ffb2-4751-a622-11b584616c7d\") " pod="openshift-ingress/e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs"
Oct 30 19:58:20.121209 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:20.121191    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d58ffd8-ffb2-4751-a622-11b584616c7d-istio-data\") pod \"e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs\" (UID: \"6d58ffd8-ffb2-4751-a622-11b584616c7d\") " pod="openshift-ingress/e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs"
Oct 30 19:58:20.121331 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:20.121315    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d58ffd8-ffb2-4751-a622-11b584616c7d-workload-certs\") pod \"e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs\" (UID: \"6d58ffd8-ffb2-4751-a622-11b584616c7d\") " pod="openshift-ingress/e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs"
Oct 30 19:58:20.121433 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:20.121416    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d58ffd8-ffb2-4751-a622-11b584616c7d-istio-podinfo\") pod \"e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs\" (UID: \"6d58ffd8-ffb2-4751-a622-11b584616c7d\") " pod="openshift-ingress/e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs"
Oct 30 19:58:20.121513 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:20.121439    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d58ffd8-ffb2-4751-a622-11b584616c7d-istio-envoy\") pod \"e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs\" (UID: \"6d58ffd8-ffb2-4751-a622-11b584616c7d\") " pod="openshift-ingress/e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs"
Oct 30 19:58:20.222641 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:20.222539    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjntn\" (UniqueName: \"kubernetes.io/projected/6d58ffd8-ffb2-4751-a622-11b584616c7d-kube-api-access-tjntn\") pod \"e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs\" (UID: \"6d58ffd8-ffb2-4751-a622-11b584616c7d\") " pod="openshift-ingress/e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs"
Oct 30 19:58:20.222823 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:20.222645    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d58ffd8-ffb2-4751-a622-11b584616c7d-istio-data\") pod \"e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs\" (UID: \"6d58ffd8-ffb2-4751-a622-11b584616c7d\") " pod="openshift-ingress/e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs"
Oct 30 19:58:20.222823 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:20.222748    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d58ffd8-ffb2-4751-a622-11b584616c7d-workload-certs\") pod \"e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs\" (UID: \"6d58ffd8-ffb2-4751-a622-11b584616c7d\") " pod="openshift-ingress/e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs"
Oct 30 19:58:20.222959 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:20.222837    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d58ffd8-ffb2-4751-a622-11b584616c7d-istio-podinfo\") pod \"e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs\" (UID: \"6d58ffd8-ffb2-4751-a622-11b584616c7d\") " pod="openshift-ingress/e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs"
Oct 30 19:58:20.222959 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:20.222862    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d58ffd8-ffb2-4751-a622-11b584616c7d-istio-envoy\") pod \"e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs\" (UID: \"6d58ffd8-ffb2-4751-a622-11b584616c7d\") " pod="openshift-ingress/e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs"
Oct 30 19:58:20.222959 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:20.222887    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d58ffd8-ffb2-4751-a622-11b584616c7d-credential-socket\") pod \"e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs\" (UID: \"6d58ffd8-ffb2-4751-a622-11b584616c7d\") " pod="openshift-ingress/e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs"
Oct 30 19:58:20.222959 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:20.222927    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d58ffd8-ffb2-4751-a622-11b584616c7d-istio-token\") pod \"e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs\" (UID: \"6d58ffd8-ffb2-4751-a622-11b584616c7d\") " pod="openshift-ingress/e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs"
Oct 30 19:58:20.223161 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:20.222984    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d58ffd8-ffb2-4751-a622-11b584616c7d-workload-socket\") pod \"e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs\" (UID: \"6d58ffd8-ffb2-4751-a622-11b584616c7d\") " pod="openshift-ingress/e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs"
Oct 30 19:58:20.223161 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:20.223062    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d58ffd8-ffb2-4751-a622-11b584616c7d-istiod-ca-cert\") pod \"e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs\" (UID: \"6d58ffd8-ffb2-4751-a622-11b584616c7d\") " pod="openshift-ingress/e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs"
Oct 30 19:58:20.223161 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:20.223114    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d58ffd8-ffb2-4751-a622-11b584616c7d-istio-data\") pod \"e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs\" (UID: \"6d58ffd8-ffb2-4751-a622-11b584616c7d\") " pod="openshift-ingress/e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs"
Oct 30 19:58:20.223161 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:20.223144    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d58ffd8-ffb2-4751-a622-11b584616c7d-workload-certs\") pod \"e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs\" (UID: \"6d58ffd8-ffb2-4751-a622-11b584616c7d\") " pod="openshift-ingress/e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs"
Oct 30 19:58:20.223520 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:20.223465    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d58ffd8-ffb2-4751-a622-11b584616c7d-workload-socket\") pod \"e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs\" (UID: \"6d58ffd8-ffb2-4751-a622-11b584616c7d\") " pod="openshift-ingress/e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs"
Oct 30 19:58:20.223631 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:20.223531    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d58ffd8-ffb2-4751-a622-11b584616c7d-credential-socket\") pod \"e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs\" (UID: \"6d58ffd8-ffb2-4751-a622-11b584616c7d\") " pod="openshift-ingress/e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs"
Oct 30 19:58:20.223867 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:20.223846    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d58ffd8-ffb2-4751-a622-11b584616c7d-istiod-ca-cert\") pod \"e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs\" (UID: \"6d58ffd8-ffb2-4751-a622-11b584616c7d\") " pod="openshift-ingress/e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs"
Oct 30 19:58:20.225287 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:20.225265    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d58ffd8-ffb2-4751-a622-11b584616c7d-istio-envoy\") pod \"e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs\" (UID: \"6d58ffd8-ffb2-4751-a622-11b584616c7d\") " pod="openshift-ingress/e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs"
Oct 30 19:58:20.225460 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:20.225444    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d58ffd8-ffb2-4751-a622-11b584616c7d-istio-podinfo\") pod \"e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs\" (UID: \"6d58ffd8-ffb2-4751-a622-11b584616c7d\") " pod="openshift-ingress/e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs"
Oct 30 19:58:20.236859 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:20.236834    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d58ffd8-ffb2-4751-a622-11b584616c7d-istio-token\") pod \"e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs\" (UID: \"6d58ffd8-ffb2-4751-a622-11b584616c7d\") " pod="openshift-ingress/e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs"
Oct 30 19:58:20.265570 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:20.265530    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjntn\" (UniqueName: \"kubernetes.io/projected/6d58ffd8-ffb2-4751-a622-11b584616c7d-kube-api-access-tjntn\") pod \"e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs\" (UID: \"6d58ffd8-ffb2-4751-a622-11b584616c7d\") " pod="openshift-ingress/e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs"
Oct 30 19:58:20.273810 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:20.273774    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs"
Oct 30 19:58:20.574055 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:20.574029    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs"]
Oct 30 19:58:20.575858 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:58:20.575835    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d58ffd8_ffb2_4751_a622_11b584616c7d.slice/crio-5b4c72b8383048a488d9cb7d01a92724fc355c8a00443b89b65cab530e487f4a WatchSource:0}: Error finding container 5b4c72b8383048a488d9cb7d01a92724fc355c8a00443b89b65cab530e487f4a: Status 404 returned error can't find the container with id 5b4c72b8383048a488d9cb7d01a92724fc355c8a00443b89b65cab530e487f4a
Oct 30 19:58:20.578574 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:20.578540    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:58:20.578647 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:20.578609    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:58:20.578647 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:20.578641    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:58:21.026131 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:21.026045    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs" event={"ID":"6d58ffd8-ffb2-4751-a622-11b584616c7d","Type":"ContainerStarted","Data":"417c6f89ccf751106408d11b66bea18d9e8cb16687401c3ba4e55f1386ae5eb5"}
Oct 30 19:58:21.026131 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:21.026115    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs" event={"ID":"6d58ffd8-ffb2-4751-a622-11b584616c7d","Type":"ContainerStarted","Data":"5b4c72b8383048a488d9cb7d01a92724fc355c8a00443b89b65cab530e487f4a"}
Oct 30 19:58:21.051278 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:21.051198    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs" podStartSLOduration=2.051177844 podStartE2EDuration="2.051177844s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:58:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:58:21.049375735 +0000 UTC m=+1576.377493890" watchObservedRunningTime="2026-10-30 19:58:21.051177844 +0000 UTC m=+1576.379296001"
Oct 30 19:58:21.274846 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:21.274814    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs"
Oct 30 19:58:21.276697 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:21.276631    6996 patch_prober.go:29] interesting pod/e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused" start-of-body=
Oct 30 19:58:21.276817 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:21.276687    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs" podUID="6d58ffd8-ffb2-4751-a622-11b584616c7d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused"
Oct 30 19:58:22.275065 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:22.275028    6996 patch_prober.go:29] interesting pod/e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused" start-of-body=
Oct 30 19:58:22.275616 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:22.275095    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs" podUID="6d58ffd8-ffb2-4751-a622-11b584616c7d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused"
Oct 30 19:58:23.274963 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:23.274929    6996 patch_prober.go:29] interesting pod/e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused" start-of-body=
Oct 30 19:58:23.275142 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:23.274985    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs" podUID="6d58ffd8-ffb2-4751-a622-11b584616c7d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused"
Oct 30 19:58:24.014643 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:24.014615    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-f0e884-6cbfdf5867-g9snm"
Oct 30 19:58:24.275050 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:24.274926    6996 patch_prober.go:29] interesting pod/e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused" start-of-body=
Oct 30 19:58:24.275050 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:24.275020    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs" podUID="6d58ffd8-ffb2-4751-a622-11b584616c7d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused"
Oct 30 19:58:25.275547 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:25.275507    6996 patch_prober.go:29] interesting pod/e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused" start-of-body=
Oct 30 19:58:25.275923 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:25.275588    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs" podUID="6d58ffd8-ffb2-4751-a622-11b584616c7d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused"
Oct 30 19:58:26.275059 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:26.275023    6996 patch_prober.go:29] interesting pod/e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused" start-of-body=
Oct 30 19:58:26.275243 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:26.275097    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs" podUID="6d58ffd8-ffb2-4751-a622-11b584616c7d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused"
Oct 30 19:58:27.280314 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:27.280280    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs"
Oct 30 19:58:27.280851 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:27.280735    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs"
Oct 30 19:58:27.281547 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:27.281527    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs"
Oct 30 19:58:28.591117 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:28.591047    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr"]
Oct 30 19:58:28.597642 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:28.597619    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr"
Oct 30 19:58:28.611338 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:28.611180    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr"]
Oct 30 19:58:28.705143 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:28.705103    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bae90963-7525-42cb-9cdc-fdb32da947e7-workload-certs\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr\" (UID: \"bae90963-7525-42cb-9cdc-fdb32da947e7\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr"
Oct 30 19:58:28.705331 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:28.705287    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bae90963-7525-42cb-9cdc-fdb32da947e7-workload-socket\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr\" (UID: \"bae90963-7525-42cb-9cdc-fdb32da947e7\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr"
Oct 30 19:58:28.705397 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:28.705329    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bae90963-7525-42cb-9cdc-fdb32da947e7-istio-data\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr\" (UID: \"bae90963-7525-42cb-9cdc-fdb32da947e7\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr"
Oct 30 19:58:28.705397 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:28.705352    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bae90963-7525-42cb-9cdc-fdb32da947e7-istio-podinfo\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr\" (UID: \"bae90963-7525-42cb-9cdc-fdb32da947e7\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr"
Oct 30 19:58:28.705644 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:28.705416    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bae90963-7525-42cb-9cdc-fdb32da947e7-istiod-ca-cert\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr\" (UID: \"bae90963-7525-42cb-9cdc-fdb32da947e7\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr"
Oct 30 19:58:28.705644 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:28.705562    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bae90963-7525-42cb-9cdc-fdb32da947e7-credential-socket\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr\" (UID: \"bae90963-7525-42cb-9cdc-fdb32da947e7\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr"
Oct 30 19:58:28.705735 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:28.705658    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bae90963-7525-42cb-9cdc-fdb32da947e7-istio-token\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr\" (UID: \"bae90963-7525-42cb-9cdc-fdb32da947e7\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr"
Oct 30 19:58:28.705735 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:28.705719    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bae90963-7525-42cb-9cdc-fdb32da947e7-istio-envoy\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr\" (UID: \"bae90963-7525-42cb-9cdc-fdb32da947e7\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr"
Oct 30 19:58:28.705834 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:28.705768    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5cpsl\" (UniqueName: \"kubernetes.io/projected/bae90963-7525-42cb-9cdc-fdb32da947e7-kube-api-access-5cpsl\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr\" (UID: \"bae90963-7525-42cb-9cdc-fdb32da947e7\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr"
Oct 30 19:58:28.807044 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:28.806990    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bae90963-7525-42cb-9cdc-fdb32da947e7-workload-socket\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr\" (UID: \"bae90963-7525-42cb-9cdc-fdb32da947e7\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr"
Oct 30 19:58:28.807233 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:28.807062    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bae90963-7525-42cb-9cdc-fdb32da947e7-istio-data\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr\" (UID: \"bae90963-7525-42cb-9cdc-fdb32da947e7\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr"
Oct 30 19:58:28.807233 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:28.807086    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bae90963-7525-42cb-9cdc-fdb32da947e7-istio-podinfo\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr\" (UID: \"bae90963-7525-42cb-9cdc-fdb32da947e7\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr"
Oct 30 19:58:28.807233 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:28.807109    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bae90963-7525-42cb-9cdc-fdb32da947e7-istiod-ca-cert\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr\" (UID: \"bae90963-7525-42cb-9cdc-fdb32da947e7\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr"
Oct 30 19:58:28.807233 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:28.807183    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bae90963-7525-42cb-9cdc-fdb32da947e7-credential-socket\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr\" (UID: \"bae90963-7525-42cb-9cdc-fdb32da947e7\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr"
Oct 30 19:58:28.807462 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:28.807238    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bae90963-7525-42cb-9cdc-fdb32da947e7-istio-token\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr\" (UID: \"bae90963-7525-42cb-9cdc-fdb32da947e7\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr"
Oct 30 19:58:28.807462 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:28.807317    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bae90963-7525-42cb-9cdc-fdb32da947e7-istio-envoy\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr\" (UID: \"bae90963-7525-42cb-9cdc-fdb32da947e7\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr"
Oct 30 19:58:28.807462 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:28.807344    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cpsl\" (UniqueName: \"kubernetes.io/projected/bae90963-7525-42cb-9cdc-fdb32da947e7-kube-api-access-5cpsl\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr\" (UID: \"bae90963-7525-42cb-9cdc-fdb32da947e7\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr"
Oct 30 19:58:28.807462 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:28.807404    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bae90963-7525-42cb-9cdc-fdb32da947e7-workload-certs\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr\" (UID: \"bae90963-7525-42cb-9cdc-fdb32da947e7\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr"
Oct 30 19:58:28.807462 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:28.807444    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bae90963-7525-42cb-9cdc-fdb32da947e7-workload-socket\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr\" (UID: \"bae90963-7525-42cb-9cdc-fdb32da947e7\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr"
Oct 30 19:58:28.807732 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:28.807518    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bae90963-7525-42cb-9cdc-fdb32da947e7-istio-data\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr\" (UID: \"bae90963-7525-42cb-9cdc-fdb32da947e7\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr"
Oct 30 19:58:28.807732 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:28.807717    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bae90963-7525-42cb-9cdc-fdb32da947e7-credential-socket\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr\" (UID: \"bae90963-7525-42cb-9cdc-fdb32da947e7\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr"
Oct 30 19:58:28.807968 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:28.807947    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bae90963-7525-42cb-9cdc-fdb32da947e7-workload-certs\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr\" (UID: \"bae90963-7525-42cb-9cdc-fdb32da947e7\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr"
Oct 30 19:58:28.808034 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:28.807953    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bae90963-7525-42cb-9cdc-fdb32da947e7-istiod-ca-cert\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr\" (UID: \"bae90963-7525-42cb-9cdc-fdb32da947e7\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr"
Oct 30 19:58:28.809737 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:28.809708    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bae90963-7525-42cb-9cdc-fdb32da947e7-istio-envoy\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr\" (UID: \"bae90963-7525-42cb-9cdc-fdb32da947e7\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr"
Oct 30 19:58:28.810041 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:28.810023    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bae90963-7525-42cb-9cdc-fdb32da947e7-istio-podinfo\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr\" (UID: \"bae90963-7525-42cb-9cdc-fdb32da947e7\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr"
Oct 30 19:58:28.820934 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:28.820908    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bae90963-7525-42cb-9cdc-fdb32da947e7-istio-token\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr\" (UID: \"bae90963-7525-42cb-9cdc-fdb32da947e7\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr"
Oct 30 19:58:28.821030 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:28.821013    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5cpsl\" (UniqueName: \"kubernetes.io/projected/bae90963-7525-42cb-9cdc-fdb32da947e7-kube-api-access-5cpsl\") pod \"e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr\" (UID: \"bae90963-7525-42cb-9cdc-fdb32da947e7\") " pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr"
Oct 30 19:58:28.922506 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:28.922390    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr"
Oct 30 19:58:29.274098 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:29.273939    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr"]
Oct 30 19:58:29.279412 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:58:29.279379    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbae90963_7525_42cb_9cdc_fdb32da947e7.slice/crio-3f2441b9cc308c9278d7868cd6fca2d1c73cb64b61b706519320b83fae024770 WatchSource:0}: Error finding container 3f2441b9cc308c9278d7868cd6fca2d1c73cb64b61b706519320b83fae024770: Status 404 returned error can't find the container with id 3f2441b9cc308c9278d7868cd6fca2d1c73cb64b61b706519320b83fae024770
Oct 30 19:58:29.282409 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:29.282372    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:58:29.282608 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:29.282450    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:58:29.282608 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:29.282495    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:58:30.123078 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:30.123040    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr" event={"ID":"bae90963-7525-42cb-9cdc-fdb32da947e7","Type":"ContainerStarted","Data":"c1a35430b4f4cd4dde26d39e3d10c40bb8be21277769637a7e0e5d051a39ff28"}
Oct 30 19:58:30.123504 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:30.123108    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr" event={"ID":"bae90963-7525-42cb-9cdc-fdb32da947e7","Type":"ContainerStarted","Data":"3f2441b9cc308c9278d7868cd6fca2d1c73cb64b61b706519320b83fae024770"}
Oct 30 19:58:30.170232 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:30.170147    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr" podStartSLOduration=2.170126613 podStartE2EDuration="2.170126613s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:58:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:58:30.167752648 +0000 UTC m=+1585.495870803" watchObservedRunningTime="2026-10-30 19:58:30.170126613 +0000 UTC m=+1585.498244762"
Oct 30 19:58:30.922761 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:30.922633    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr"
Oct 30 19:58:30.971092 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:30.923967    6996 patch_prober.go:29] interesting pod/e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.146:15021/healthz/ready\": dial tcp 10.134.0.146:15021: connect: connection refused" start-of-body=
Oct 30 19:58:30.971092 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:30.924028    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr" podUID="bae90963-7525-42cb-9cdc-fdb32da947e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.146:15021/healthz/ready\": dial tcp 10.134.0.146:15021: connect: connection refused"
Oct 30 19:58:31.204806 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:58:31.204721    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 19:58:31.926505 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:31.923157    6996 patch_prober.go:29] interesting pod/e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.146:15021/healthz/ready\": dial tcp 10.134.0.146:15021: connect: connection refused" start-of-body=
Oct 30 19:58:31.926505 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:31.923215    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr" podUID="bae90963-7525-42cb-9cdc-fdb32da947e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.146:15021/healthz/ready\": dial tcp 10.134.0.146:15021: connect: connection refused"
Oct 30 19:58:32.923742 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:32.923704    6996 patch_prober.go:29] interesting pod/e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.146:15021/healthz/ready\": dial tcp 10.134.0.146:15021: connect: connection refused" start-of-body=
Oct 30 19:58:32.924348 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:32.924312    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr" podUID="bae90963-7525-42cb-9cdc-fdb32da947e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.146:15021/healthz/ready\": dial tcp 10.134.0.146:15021: connect: connection refused"
Oct 30 19:58:33.923175 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:33.923139    6996 patch_prober.go:29] interesting pod/e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.146:15021/healthz/ready\": dial tcp 10.134.0.146:15021: connect: connection refused" start-of-body=
Oct 30 19:58:33.923379 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:33.923200    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr" podUID="bae90963-7525-42cb-9cdc-fdb32da947e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.146:15021/healthz/ready\": dial tcp 10.134.0.146:15021: connect: connection refused"
Oct 30 19:58:34.923353 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:34.923312    6996 patch_prober.go:29] interesting pod/e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.146:15021/healthz/ready\": dial tcp 10.134.0.146:15021: connect: connection refused" start-of-body=
Oct 30 19:58:34.923859 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:34.923369    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr" podUID="bae90963-7525-42cb-9cdc-fdb32da947e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.146:15021/healthz/ready\": dial tcp 10.134.0.146:15021: connect: connection refused"
Oct 30 19:58:35.923707 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:35.923666    6996 patch_prober.go:29] interesting pod/e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.146:15021/healthz/ready\": dial tcp 10.134.0.146:15021: connect: connection refused" start-of-body=
Oct 30 19:58:35.924119 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:35.923735    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr" podUID="bae90963-7525-42cb-9cdc-fdb32da947e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.146:15021/healthz/ready\": dial tcp 10.134.0.146:15021: connect: connection refused"
Oct 30 19:58:37.964657 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:37.964602    6996 patch_prober.go:29] interesting pod/e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.146:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:58:37.965290 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:37.964665    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr" podUID="bae90963-7525-42cb-9cdc-fdb32da947e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.146:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:58:38.130211 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:38.130174    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr"
Oct 30 19:58:38.130889 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:38.130857    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr"
Oct 30 19:58:38.131695 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:38.131674    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr"
Oct 30 19:58:43.183228 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:43.183198    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8"]
Oct 30 19:58:43.187816 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:43.187799    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8"
Oct 30 19:58:43.199238 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:43.199218    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-explicit-f0e884-openshift-default-dockercfg-z5dhl\""
Oct 30 19:58:43.225924 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:43.225886    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8"]
Oct 30 19:58:43.273897 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:43.273865    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/de10228b-eec3-408f-b4cd-5250da381206-istio-data\") pod \"e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8\" (UID: \"de10228b-eec3-408f-b4cd-5250da381206\") " pod="openshift-ingress/e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8"
Oct 30 19:58:43.274060 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:43.273915    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4s9ld\" (UniqueName: \"kubernetes.io/projected/de10228b-eec3-408f-b4cd-5250da381206-kube-api-access-4s9ld\") pod \"e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8\" (UID: \"de10228b-eec3-408f-b4cd-5250da381206\") " pod="openshift-ingress/e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8"
Oct 30 19:58:43.274060 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:43.274012    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/de10228b-eec3-408f-b4cd-5250da381206-istio-envoy\") pod \"e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8\" (UID: \"de10228b-eec3-408f-b4cd-5250da381206\") " pod="openshift-ingress/e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8"
Oct 30 19:58:43.274171 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:43.274151    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/de10228b-eec3-408f-b4cd-5250da381206-istiod-ca-cert\") pod \"e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8\" (UID: \"de10228b-eec3-408f-b4cd-5250da381206\") " pod="openshift-ingress/e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8"
Oct 30 19:58:43.274211 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:43.274199    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/de10228b-eec3-408f-b4cd-5250da381206-workload-socket\") pod \"e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8\" (UID: \"de10228b-eec3-408f-b4cd-5250da381206\") " pod="openshift-ingress/e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8"
Oct 30 19:58:43.274257 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:43.274222    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/de10228b-eec3-408f-b4cd-5250da381206-istio-podinfo\") pod \"e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8\" (UID: \"de10228b-eec3-408f-b4cd-5250da381206\") " pod="openshift-ingress/e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8"
Oct 30 19:58:43.274351 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:43.274330    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/de10228b-eec3-408f-b4cd-5250da381206-credential-socket\") pod \"e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8\" (UID: \"de10228b-eec3-408f-b4cd-5250da381206\") " pod="openshift-ingress/e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8"
Oct 30 19:58:43.274419 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:43.274365    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/de10228b-eec3-408f-b4cd-5250da381206-workload-certs\") pod \"e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8\" (UID: \"de10228b-eec3-408f-b4cd-5250da381206\") " pod="openshift-ingress/e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8"
Oct 30 19:58:43.274419 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:43.274404    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de10228b-eec3-408f-b4cd-5250da381206-istio-token\") pod \"e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8\" (UID: \"de10228b-eec3-408f-b4cd-5250da381206\") " pod="openshift-ingress/e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8"
Oct 30 19:58:43.376048 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:43.376007    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/de10228b-eec3-408f-b4cd-5250da381206-istiod-ca-cert\") pod \"e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8\" (UID: \"de10228b-eec3-408f-b4cd-5250da381206\") " pod="openshift-ingress/e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8"
Oct 30 19:58:43.376212 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:43.376069    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/de10228b-eec3-408f-b4cd-5250da381206-workload-socket\") pod \"e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8\" (UID: \"de10228b-eec3-408f-b4cd-5250da381206\") " pod="openshift-ingress/e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8"
Oct 30 19:58:43.376212 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:43.376103    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/de10228b-eec3-408f-b4cd-5250da381206-istio-podinfo\") pod \"e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8\" (UID: \"de10228b-eec3-408f-b4cd-5250da381206\") " pod="openshift-ingress/e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8"
Oct 30 19:58:43.376212 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:43.376160    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/de10228b-eec3-408f-b4cd-5250da381206-credential-socket\") pod \"e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8\" (UID: \"de10228b-eec3-408f-b4cd-5250da381206\") " pod="openshift-ingress/e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8"
Oct 30 19:58:43.376212 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:43.376185    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/de10228b-eec3-408f-b4cd-5250da381206-workload-certs\") pod \"e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8\" (UID: \"de10228b-eec3-408f-b4cd-5250da381206\") " pod="openshift-ingress/e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8"
Oct 30 19:58:43.376432 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:43.376215    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de10228b-eec3-408f-b4cd-5250da381206-istio-token\") pod \"e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8\" (UID: \"de10228b-eec3-408f-b4cd-5250da381206\") " pod="openshift-ingress/e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8"
Oct 30 19:58:43.376432 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:43.376323    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/de10228b-eec3-408f-b4cd-5250da381206-istio-data\") pod \"e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8\" (UID: \"de10228b-eec3-408f-b4cd-5250da381206\") " pod="openshift-ingress/e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8"
Oct 30 19:58:43.376432 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:43.376375    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4s9ld\" (UniqueName: \"kubernetes.io/projected/de10228b-eec3-408f-b4cd-5250da381206-kube-api-access-4s9ld\") pod \"e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8\" (UID: \"de10228b-eec3-408f-b4cd-5250da381206\") " pod="openshift-ingress/e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8"
Oct 30 19:58:43.376614 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:43.376455    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/de10228b-eec3-408f-b4cd-5250da381206-istio-envoy\") pod \"e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8\" (UID: \"de10228b-eec3-408f-b4cd-5250da381206\") " pod="openshift-ingress/e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8"
Oct 30 19:58:43.376614 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:43.376581    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/de10228b-eec3-408f-b4cd-5250da381206-workload-socket\") pod \"e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8\" (UID: \"de10228b-eec3-408f-b4cd-5250da381206\") " pod="openshift-ingress/e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8"
Oct 30 19:58:43.376821 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:43.376797    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/de10228b-eec3-408f-b4cd-5250da381206-workload-certs\") pod \"e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8\" (UID: \"de10228b-eec3-408f-b4cd-5250da381206\") " pod="openshift-ingress/e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8"
Oct 30 19:58:43.376897 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:43.376809    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/de10228b-eec3-408f-b4cd-5250da381206-credential-socket\") pod \"e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8\" (UID: \"de10228b-eec3-408f-b4cd-5250da381206\") " pod="openshift-ingress/e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8"
Oct 30 19:58:43.376897 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:43.376882    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/de10228b-eec3-408f-b4cd-5250da381206-istiod-ca-cert\") pod \"e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8\" (UID: \"de10228b-eec3-408f-b4cd-5250da381206\") " pod="openshift-ingress/e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8"
Oct 30 19:58:43.377012 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:43.376883    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/de10228b-eec3-408f-b4cd-5250da381206-istio-data\") pod \"e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8\" (UID: \"de10228b-eec3-408f-b4cd-5250da381206\") " pod="openshift-ingress/e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8"
Oct 30 19:58:43.378600 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:43.378579    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/de10228b-eec3-408f-b4cd-5250da381206-istio-envoy\") pod \"e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8\" (UID: \"de10228b-eec3-408f-b4cd-5250da381206\") " pod="openshift-ingress/e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8"
Oct 30 19:58:43.378779 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:43.378759    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/de10228b-eec3-408f-b4cd-5250da381206-istio-podinfo\") pod \"e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8\" (UID: \"de10228b-eec3-408f-b4cd-5250da381206\") " pod="openshift-ingress/e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8"
Oct 30 19:58:43.389077 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:43.389053    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4s9ld\" (UniqueName: \"kubernetes.io/projected/de10228b-eec3-408f-b4cd-5250da381206-kube-api-access-4s9ld\") pod \"e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8\" (UID: \"de10228b-eec3-408f-b4cd-5250da381206\") " pod="openshift-ingress/e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8"
Oct 30 19:58:43.389909 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:43.389891    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de10228b-eec3-408f-b4cd-5250da381206-istio-token\") pod \"e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8\" (UID: \"de10228b-eec3-408f-b4cd-5250da381206\") " pod="openshift-ingress/e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8"
Oct 30 19:58:43.502990 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:43.502904    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8"
Oct 30 19:58:43.794970 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:43.794946    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8"]
Oct 30 19:58:43.797801 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:58:43.797772    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde10228b_eec3_408f_b4cd_5250da381206.slice/crio-ef2560c425acf98daa3684b535ddc7d47ba345daa473ad31708cbe475655afac WatchSource:0}: Error finding container ef2560c425acf98daa3684b535ddc7d47ba345daa473ad31708cbe475655afac: Status 404 returned error can't find the container with id ef2560c425acf98daa3684b535ddc7d47ba345daa473ad31708cbe475655afac
Oct 30 19:58:43.801002 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:43.800967    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:58:43.801093 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:43.801040    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:58:43.801093 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:43.801070    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:58:44.286124 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:44.286088    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8" event={"ID":"de10228b-eec3-408f-b4cd-5250da381206","Type":"ContainerStarted","Data":"7f3ebe7dbb29a59e73a6fa8b08a0760a1c428b0ac51552473584b3a7311d7077"}
Oct 30 19:58:44.286582 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:44.286142    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8" event={"ID":"de10228b-eec3-408f-b4cd-5250da381206","Type":"ContainerStarted","Data":"ef2560c425acf98daa3684b535ddc7d47ba345daa473ad31708cbe475655afac"}
Oct 30 19:58:44.313635 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:44.313587    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8" podStartSLOduration=1.313572252 podStartE2EDuration="1.313572252s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:58:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:58:44.310500414 +0000 UTC m=+1599.638618559" watchObservedRunningTime="2026-10-30 19:58:44.313572252 +0000 UTC m=+1599.641690406"
Oct 30 19:58:44.503489 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:44.503457    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8"
Oct 30 19:58:44.504928 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:44.504901    6996 patch_prober.go:29] interesting pod/e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.147:15021/healthz/ready\": dial tcp 10.134.0.147:15021: connect: connection refused" start-of-body=
Oct 30 19:58:44.505069 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:44.504958    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8" podUID="de10228b-eec3-408f-b4cd-5250da381206" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.147:15021/healthz/ready\": dial tcp 10.134.0.147:15021: connect: connection refused"
Oct 30 19:58:45.503358 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:45.503321    6996 patch_prober.go:29] interesting pod/e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.147:15021/healthz/ready\": dial tcp 10.134.0.147:15021: connect: connection refused" start-of-body=
Oct 30 19:58:45.503813 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:45.503378    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8" podUID="de10228b-eec3-408f-b4cd-5250da381206" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.147:15021/healthz/ready\": dial tcp 10.134.0.147:15021: connect: connection refused"
Oct 30 19:58:46.503191 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:46.503156    6996 patch_prober.go:29] interesting pod/e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.147:15021/healthz/ready\": dial tcp 10.134.0.147:15021: connect: connection refused" start-of-body=
Oct 30 19:58:46.503392 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:46.503207    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8" podUID="de10228b-eec3-408f-b4cd-5250da381206" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.147:15021/healthz/ready\": dial tcp 10.134.0.147:15021: connect: connection refused"
Oct 30 19:58:47.504035 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:47.503986    6996 patch_prober.go:29] interesting pod/e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.147:15021/healthz/ready\": dial tcp 10.134.0.147:15021: connect: connection refused" start-of-body=
Oct 30 19:58:47.504546 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:47.504055    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8" podUID="de10228b-eec3-408f-b4cd-5250da381206" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.147:15021/healthz/ready\": dial tcp 10.134.0.147:15021: connect: connection refused"
Oct 30 19:58:48.504144 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:48.504108    6996 patch_prober.go:29] interesting pod/e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.147:15021/healthz/ready\": dial tcp 10.134.0.147:15021: connect: connection refused" start-of-body=
Oct 30 19:58:48.504585 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:48.504168    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8" podUID="de10228b-eec3-408f-b4cd-5250da381206" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.147:15021/healthz/ready\": dial tcp 10.134.0.147:15021: connect: connection refused"
Oct 30 19:58:49.503324 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:49.503287    6996 patch_prober.go:29] interesting pod/e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.147:15021/healthz/ready\": dial tcp 10.134.0.147:15021: connect: connection refused" start-of-body=
Oct 30 19:58:49.503561 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:49.503341    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8" podUID="de10228b-eec3-408f-b4cd-5250da381206" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.147:15021/healthz/ready\": dial tcp 10.134.0.147:15021: connect: connection refused"
Oct 30 19:58:49.558510 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:49.555824    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-2rj4k"]
Oct 30 19:58:49.558510 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:49.556074    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-2rj4k" podUID="8d0ef765-f9f8-4f8a-82ad-0a766a663782" containerName="istio-proxy" containerID="cri-o://0e4b1db12b6da99fb8258d5b38eb0c7b49c53024421b0c92a310d962a32bf424" gracePeriod=30
Oct 30 19:58:50.508644 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:50.508613    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8"
Oct 30 19:58:50.509094 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:50.509074    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8"
Oct 30 19:58:50.509935 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:50.509916    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8"
Oct 30 19:58:55.246056 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:55.246031    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-2rj4k"
Oct 30 19:58:55.278928 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:55.278892    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8d0ef765-f9f8-4f8a-82ad-0a766a663782-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d0ef765-f9f8-4f8a-82ad-0a766a663782-workload-certs\") pod \"8d0ef765-f9f8-4f8a-82ad-0a766a663782\" (UID: \"8d0ef765-f9f8-4f8a-82ad-0a766a663782\") "
Oct 30 19:58:55.279118 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:55.279098    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8d0ef765-f9f8-4f8a-82ad-0a766a663782-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d0ef765-f9f8-4f8a-82ad-0a766a663782-credential-socket\") pod \"8d0ef765-f9f8-4f8a-82ad-0a766a663782\" (UID: \"8d0ef765-f9f8-4f8a-82ad-0a766a663782\") "
Oct 30 19:58:55.279187 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:55.279161    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d0ef765-f9f8-4f8a-82ad-0a766a663782-workload-certs" pod "8d0ef765-f9f8-4f8a-82ad-0a766a663782" (UID: "8d0ef765-f9f8-4f8a-82ad-0a766a663782"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:58:55.279251 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:55.279169    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8d0ef765-f9f8-4f8a-82ad-0a766a663782-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8d0ef765-f9f8-4f8a-82ad-0a766a663782-istio-envoy\") pod \"8d0ef765-f9f8-4f8a-82ad-0a766a663782\" (UID: \"8d0ef765-f9f8-4f8a-82ad-0a766a663782\") "
Oct 30 19:58:55.279303 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:55.279259    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8d0ef765-f9f8-4f8a-82ad-0a766a663782-kube-api-access-cvvnf\" (UniqueName: \"kubernetes.io/projected/8d0ef765-f9f8-4f8a-82ad-0a766a663782-kube-api-access-cvvnf\") pod \"8d0ef765-f9f8-4f8a-82ad-0a766a663782\" (UID: \"8d0ef765-f9f8-4f8a-82ad-0a766a663782\") "
Oct 30 19:58:55.279303 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:55.279299    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8d0ef765-f9f8-4f8a-82ad-0a766a663782-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8d0ef765-f9f8-4f8a-82ad-0a766a663782-istiod-ca-cert\") pod \"8d0ef765-f9f8-4f8a-82ad-0a766a663782\" (UID: \"8d0ef765-f9f8-4f8a-82ad-0a766a663782\") "
Oct 30 19:58:55.279407 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:55.279336    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8d0ef765-f9f8-4f8a-82ad-0a766a663782-istio-token\" (UniqueName: \"kubernetes.io/projected/8d0ef765-f9f8-4f8a-82ad-0a766a663782-istio-token\") pod \"8d0ef765-f9f8-4f8a-82ad-0a766a663782\" (UID: \"8d0ef765-f9f8-4f8a-82ad-0a766a663782\") "
Oct 30 19:58:55.279407 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:55.279392    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8d0ef765-f9f8-4f8a-82ad-0a766a663782-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d0ef765-f9f8-4f8a-82ad-0a766a663782-workload-socket\") pod \"8d0ef765-f9f8-4f8a-82ad-0a766a663782\" (UID: \"8d0ef765-f9f8-4f8a-82ad-0a766a663782\") "
Oct 30 19:58:55.279573 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:55.279426    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8d0ef765-f9f8-4f8a-82ad-0a766a663782-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8d0ef765-f9f8-4f8a-82ad-0a766a663782-istio-podinfo\") pod \"8d0ef765-f9f8-4f8a-82ad-0a766a663782\" (UID: \"8d0ef765-f9f8-4f8a-82ad-0a766a663782\") "
Oct 30 19:58:55.279573 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:55.279446    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d0ef765-f9f8-4f8a-82ad-0a766a663782-credential-socket" pod "8d0ef765-f9f8-4f8a-82ad-0a766a663782" (UID: "8d0ef765-f9f8-4f8a-82ad-0a766a663782"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:58:55.279573 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:55.279495    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8d0ef765-f9f8-4f8a-82ad-0a766a663782-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8d0ef765-f9f8-4f8a-82ad-0a766a663782-istio-data\") pod \"8d0ef765-f9f8-4f8a-82ad-0a766a663782\" (UID: \"8d0ef765-f9f8-4f8a-82ad-0a766a663782\") "
Oct 30 19:58:55.279837 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:55.279806    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d0ef765-f9f8-4f8a-82ad-0a766a663782-istiod-ca-cert" pod "8d0ef765-f9f8-4f8a-82ad-0a766a663782" (UID: "8d0ef765-f9f8-4f8a-82ad-0a766a663782"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:58:55.280189 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:55.280163    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d0ef765-f9f8-4f8a-82ad-0a766a663782-istio-data" pod "8d0ef765-f9f8-4f8a-82ad-0a766a663782" (UID: "8d0ef765-f9f8-4f8a-82ad-0a766a663782"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:58:55.280189 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:55.279974    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d0ef765-f9f8-4f8a-82ad-0a766a663782-workload-socket" pod "8d0ef765-f9f8-4f8a-82ad-0a766a663782" (UID: "8d0ef765-f9f8-4f8a-82ad-0a766a663782"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:58:55.280517 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:55.280457    6996 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d0ef765-f9f8-4f8a-82ad-0a766a663782-credential-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:55.281693 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:55.280524    6996 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8d0ef765-f9f8-4f8a-82ad-0a766a663782-istiod-ca-cert\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:55.281693 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:55.280541    6996 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d0ef765-f9f8-4f8a-82ad-0a766a663782-workload-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:55.281693 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:55.280556    6996 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8d0ef765-f9f8-4f8a-82ad-0a766a663782-istio-data\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:55.281693 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:55.280570    6996 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d0ef765-f9f8-4f8a-82ad-0a766a663782-workload-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:55.282006 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:55.281856    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d0ef765-f9f8-4f8a-82ad-0a766a663782-istio-envoy" pod "8d0ef765-f9f8-4f8a-82ad-0a766a663782" (UID: "8d0ef765-f9f8-4f8a-82ad-0a766a663782"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:58:55.282006 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:55.281875    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d0ef765-f9f8-4f8a-82ad-0a766a663782-kube-api-access-cvvnf" pod "8d0ef765-f9f8-4f8a-82ad-0a766a663782" (UID: "8d0ef765-f9f8-4f8a-82ad-0a766a663782"). InnerVolumeSpecName "kube-api-access-cvvnf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:58:55.282414 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:55.282392    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d0ef765-f9f8-4f8a-82ad-0a766a663782-istio-token" pod "8d0ef765-f9f8-4f8a-82ad-0a766a663782" (UID: "8d0ef765-f9f8-4f8a-82ad-0a766a663782"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:58:55.282629 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:55.282605    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8d0ef765-f9f8-4f8a-82ad-0a766a663782-istio-podinfo" pod "8d0ef765-f9f8-4f8a-82ad-0a766a663782" (UID: "8d0ef765-f9f8-4f8a-82ad-0a766a663782"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:58:55.364267 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:55.364175    6996 generic.go:356] "Generic (PLEG): container finished" podID="8d0ef765-f9f8-4f8a-82ad-0a766a663782" containerID="0e4b1db12b6da99fb8258d5b38eb0c7b49c53024421b0c92a310d962a32bf424" exitCode=0
Oct 30 19:58:55.364267 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:55.364230    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-2rj4k"
Oct 30 19:58:55.364510 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:55.364269    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-2rj4k" event={"ID":"8d0ef765-f9f8-4f8a-82ad-0a766a663782","Type":"ContainerDied","Data":"0e4b1db12b6da99fb8258d5b38eb0c7b49c53024421b0c92a310d962a32bf424"}
Oct 30 19:58:55.364510 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:55.364345    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-2rj4k" event={"ID":"8d0ef765-f9f8-4f8a-82ad-0a766a663782","Type":"ContainerDied","Data":"fad37487d4da79cb51c85f81b6b674c33dad4ddef481d90caef9a772cd2388db"}
Oct 30 19:58:55.364510 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:55.364380    6996 scope.go:122] "RemoveContainer" containerID="0e4b1db12b6da99fb8258d5b38eb0c7b49c53024421b0c92a310d962a32bf424"
Oct 30 19:58:55.376599 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:55.376580    6996 scope.go:122] "RemoveContainer" containerID="0e4b1db12b6da99fb8258d5b38eb0c7b49c53024421b0c92a310d962a32bf424"
Oct 30 19:58:55.376886 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:58:55.376860    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e4b1db12b6da99fb8258d5b38eb0c7b49c53024421b0c92a310d962a32bf424\": container with ID starting with 0e4b1db12b6da99fb8258d5b38eb0c7b49c53024421b0c92a310d962a32bf424 not found: ID does not exist" containerID="0e4b1db12b6da99fb8258d5b38eb0c7b49c53024421b0c92a310d962a32bf424"
Oct 30 19:58:55.376999 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:55.376891    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e4b1db12b6da99fb8258d5b38eb0c7b49c53024421b0c92a310d962a32bf424"} err="failed to get container status \"0e4b1db12b6da99fb8258d5b38eb0c7b49c53024421b0c92a310d962a32bf424\": rpc error: code = NotFound desc = could not find container \"0e4b1db12b6da99fb8258d5b38eb0c7b49c53024421b0c92a310d962a32bf424\": container with ID starting with 0e4b1db12b6da99fb8258d5b38eb0c7b49c53024421b0c92a310d962a32bf424 not found: ID does not exist"
Oct 30 19:58:55.381962 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:55.381940    6996 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8d0ef765-f9f8-4f8a-82ad-0a766a663782-istio-envoy\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:55.382065 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:55.381965    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cvvnf\" (UniqueName: \"kubernetes.io/projected/8d0ef765-f9f8-4f8a-82ad-0a766a663782-kube-api-access-cvvnf\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:55.382065 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:55.381980    6996 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d0ef765-f9f8-4f8a-82ad-0a766a663782-istio-token\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:55.382065 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:55.381993    6996 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8d0ef765-f9f8-4f8a-82ad-0a766a663782-istio-podinfo\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:55.394853 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:55.394670    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-2rj4k"]
Oct 30 19:58:55.401366 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:55.401333    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-2rj4k"]
Oct 30 19:58:56.793513 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:58:56.793448    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8d0ef765-f9f8-4f8a-82ad-0a766a663782" path="/var/lib/kubelet/pods/8d0ef765-f9f8-4f8a-82ad-0a766a663782/volumes"
Oct 30 19:59:01.190659 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:59:01.190621    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 19:59:05.571914 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:05.571878    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 19:59:21.477673 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:21.477586    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-f0e884-6cbfdf5867-g9snm"]
Oct 30 19:59:21.478140 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:21.477808    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-f0e884-6cbfdf5867-g9snm" podUID="982d0d28-3106-42c2-a300-5a4e56645596" containerName="maas-api" containerID="cri-o://28f4202e3f337c2a0f370c7cdfe60f6d9b9475bf29f7a7c157dfac1e637013d7" gracePeriod=30
Oct 30 19:59:21.892308 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:21.892276    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-f0e884-6cbfdf5867-g9snm"
Oct 30 19:59:22.014256 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:22.014214    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/982d0d28-3106-42c2-a300-5a4e56645596-kube-api-access-wh7s4\" (UniqueName: \"kubernetes.io/projected/982d0d28-3106-42c2-a300-5a4e56645596-kube-api-access-wh7s4\") pod \"982d0d28-3106-42c2-a300-5a4e56645596\" (UID: \"982d0d28-3106-42c2-a300-5a4e56645596\") "
Oct 30 19:59:22.014444 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:22.014272    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/982d0d28-3106-42c2-a300-5a4e56645596-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/982d0d28-3106-42c2-a300-5a4e56645596-openshift-service-ca\") pod \"982d0d28-3106-42c2-a300-5a4e56645596\" (UID: \"982d0d28-3106-42c2-a300-5a4e56645596\") "
Oct 30 19:59:22.014444 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:22.014359    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/982d0d28-3106-42c2-a300-5a4e56645596-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/982d0d28-3106-42c2-a300-5a4e56645596-maas-api-tls\") pod \"982d0d28-3106-42c2-a300-5a4e56645596\" (UID: \"982d0d28-3106-42c2-a300-5a4e56645596\") "
Oct 30 19:59:22.014591 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:22.014444    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/982d0d28-3106-42c2-a300-5a4e56645596-metrics-certs\" (UniqueName: \"kubernetes.io/secret/982d0d28-3106-42c2-a300-5a4e56645596-metrics-certs\") pod \"982d0d28-3106-42c2-a300-5a4e56645596\" (UID: \"982d0d28-3106-42c2-a300-5a4e56645596\") "
Oct 30 19:59:22.015099 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:22.015055    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/982d0d28-3106-42c2-a300-5a4e56645596-openshift-service-ca" pod "982d0d28-3106-42c2-a300-5a4e56645596" (UID: "982d0d28-3106-42c2-a300-5a4e56645596"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:59:22.015349 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:22.015319    6996 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/982d0d28-3106-42c2-a300-5a4e56645596-openshift-service-ca\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:22.017075 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:22.017045    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/982d0d28-3106-42c2-a300-5a4e56645596-kube-api-access-wh7s4" pod "982d0d28-3106-42c2-a300-5a4e56645596" (UID: "982d0d28-3106-42c2-a300-5a4e56645596"). InnerVolumeSpecName "kube-api-access-wh7s4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:59:22.017205 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:22.017183    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/982d0d28-3106-42c2-a300-5a4e56645596-maas-api-tls" pod "982d0d28-3106-42c2-a300-5a4e56645596" (UID: "982d0d28-3106-42c2-a300-5a4e56645596"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:59:22.017429 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:22.017414    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/982d0d28-3106-42c2-a300-5a4e56645596-metrics-certs" pod "982d0d28-3106-42c2-a300-5a4e56645596" (UID: "982d0d28-3106-42c2-a300-5a4e56645596"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:59:22.116661 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:22.116571    6996 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/982d0d28-3106-42c2-a300-5a4e56645596-metrics-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:22.116661 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:22.116603    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wh7s4\" (UniqueName: \"kubernetes.io/projected/982d0d28-3106-42c2-a300-5a4e56645596-kube-api-access-wh7s4\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:22.116661 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:22.116615    6996 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/982d0d28-3106-42c2-a300-5a4e56645596-maas-api-tls\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:22.593885 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:22.593847    6996 generic.go:356] "Generic (PLEG): container finished" podID="982d0d28-3106-42c2-a300-5a4e56645596" containerID="28f4202e3f337c2a0f370c7cdfe60f6d9b9475bf29f7a7c157dfac1e637013d7" exitCode=0
Oct 30 19:59:22.594303 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:22.593898    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-f0e884-6cbfdf5867-g9snm"
Oct 30 19:59:22.594303 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:22.593928    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-f0e884-6cbfdf5867-g9snm" event={"ID":"982d0d28-3106-42c2-a300-5a4e56645596","Type":"ContainerDied","Data":"28f4202e3f337c2a0f370c7cdfe60f6d9b9475bf29f7a7c157dfac1e637013d7"}
Oct 30 19:59:22.594303 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:22.593989    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-f0e884-6cbfdf5867-g9snm" event={"ID":"982d0d28-3106-42c2-a300-5a4e56645596","Type":"ContainerDied","Data":"aaf299c08b7a0b512e824f3f55d663b2a41f050a118f9c3ed1df479a8b199935"}
Oct 30 19:59:22.594303 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:22.594013    6996 scope.go:122] "RemoveContainer" containerID="28f4202e3f337c2a0f370c7cdfe60f6d9b9475bf29f7a7c157dfac1e637013d7"
Oct 30 19:59:22.606114 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:22.606095    6996 scope.go:122] "RemoveContainer" containerID="28f4202e3f337c2a0f370c7cdfe60f6d9b9475bf29f7a7c157dfac1e637013d7"
Oct 30 19:59:22.606406 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:59:22.606385    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28f4202e3f337c2a0f370c7cdfe60f6d9b9475bf29f7a7c157dfac1e637013d7\": container with ID starting with 28f4202e3f337c2a0f370c7cdfe60f6d9b9475bf29f7a7c157dfac1e637013d7 not found: ID does not exist" containerID="28f4202e3f337c2a0f370c7cdfe60f6d9b9475bf29f7a7c157dfac1e637013d7"
Oct 30 19:59:22.606508 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:22.606415    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28f4202e3f337c2a0f370c7cdfe60f6d9b9475bf29f7a7c157dfac1e637013d7"} err="failed to get container status \"28f4202e3f337c2a0f370c7cdfe60f6d9b9475bf29f7a7c157dfac1e637013d7\": rpc error: code = NotFound desc = could not find container \"28f4202e3f337c2a0f370c7cdfe60f6d9b9475bf29f7a7c157dfac1e637013d7\": container with ID starting with 28f4202e3f337c2a0f370c7cdfe60f6d9b9475bf29f7a7c157dfac1e637013d7 not found: ID does not exist"
Oct 30 19:59:22.621519 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:22.621491    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-f0e884-6cbfdf5867-g9snm"]
Oct 30 19:59:22.635075 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:22.635040    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-f0e884-6cbfdf5867-g9snm"]
Oct 30 19:59:22.793178 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:22.793145    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="982d0d28-3106-42c2-a300-5a4e56645596" path="/var/lib/kubelet/pods/982d0d28-3106-42c2-a300-5a4e56645596/volumes"
Oct 30 19:59:30.127629 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:30.127588    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8"]
Oct 30 19:59:30.128667 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:30.128631    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8" podUID="de10228b-eec3-408f-b4cd-5250da381206" containerName="istio-proxy" containerID="cri-o://7f3ebe7dbb29a59e73a6fa8b08a0760a1c428b0ac51552473584b3a7311d7077" gracePeriod=30
Oct 30 19:59:31.196294 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:59:31.196254    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 19:59:31.380516 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:31.380464    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-noten-8fdea8-openshift-default-98d979769-skktq"]
Oct 30 19:59:31.381522 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:31.381498    6996 cpu_manager.go:414] "containerMap: removing container" podUID="8d0ef765-f9f8-4f8a-82ad-0a766a663782" containerName="istio-proxy"
Oct 30 19:59:31.381627 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:31.381524    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8d0ef765-f9f8-4f8a-82ad-0a766a663782" containerName="istio-proxy"
Oct 30 19:59:31.381627 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:31.381573    6996 cpu_manager.go:414] "containerMap: removing container" podUID="982d0d28-3106-42c2-a300-5a4e56645596" containerName="maas-api"
Oct 30 19:59:31.381627 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:31.381581    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="982d0d28-3106-42c2-a300-5a4e56645596" containerName="maas-api"
Oct 30 19:59:31.381801 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:31.381760    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="982d0d28-3106-42c2-a300-5a4e56645596" containerName="maas-api"
Oct 30 19:59:31.381801 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:31.381791    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="8d0ef765-f9f8-4f8a-82ad-0a766a663782" containerName="istio-proxy"
Oct 30 19:59:31.385699 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:31.385672    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-8fdea8-openshift-default-98d979769-skktq"
Oct 30 19:59:31.394339 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:31.394240    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-noten-8fdea8-openshift-default-dockercfg-dqsvk\""
Oct 30 19:59:31.402526 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:31.402499    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-8fdea8-openshift-default-98d979769-skktq"]
Oct 30 19:59:31.474114 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:31.474037    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/131a2776-cefc-4c87-9fd7-133ebe2a4e27-istio-podinfo\") pod \"e2e-noten-8fdea8-openshift-default-98d979769-skktq\" (UID: \"131a2776-cefc-4c87-9fd7-133ebe2a4e27\") " pod="openshift-ingress/e2e-noten-8fdea8-openshift-default-98d979769-skktq"
Oct 30 19:59:31.474114 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:31.474073    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/131a2776-cefc-4c87-9fd7-133ebe2a4e27-istio-token\") pod \"e2e-noten-8fdea8-openshift-default-98d979769-skktq\" (UID: \"131a2776-cefc-4c87-9fd7-133ebe2a4e27\") " pod="openshift-ingress/e2e-noten-8fdea8-openshift-default-98d979769-skktq"
Oct 30 19:59:31.474313 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:31.474183    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/131a2776-cefc-4c87-9fd7-133ebe2a4e27-istio-envoy\") pod \"e2e-noten-8fdea8-openshift-default-98d979769-skktq\" (UID: \"131a2776-cefc-4c87-9fd7-133ebe2a4e27\") " pod="openshift-ingress/e2e-noten-8fdea8-openshift-default-98d979769-skktq"
Oct 30 19:59:31.474313 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:31.474215    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/131a2776-cefc-4c87-9fd7-133ebe2a4e27-istio-data\") pod \"e2e-noten-8fdea8-openshift-default-98d979769-skktq\" (UID: \"131a2776-cefc-4c87-9fd7-133ebe2a4e27\") " pod="openshift-ingress/e2e-noten-8fdea8-openshift-default-98d979769-skktq"
Oct 30 19:59:31.474313 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:31.474281    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/131a2776-cefc-4c87-9fd7-133ebe2a4e27-credential-socket\") pod \"e2e-noten-8fdea8-openshift-default-98d979769-skktq\" (UID: \"131a2776-cefc-4c87-9fd7-133ebe2a4e27\") " pod="openshift-ingress/e2e-noten-8fdea8-openshift-default-98d979769-skktq"
Oct 30 19:59:31.474313 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:31.474308    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/131a2776-cefc-4c87-9fd7-133ebe2a4e27-istiod-ca-cert\") pod \"e2e-noten-8fdea8-openshift-default-98d979769-skktq\" (UID: \"131a2776-cefc-4c87-9fd7-133ebe2a4e27\") " pod="openshift-ingress/e2e-noten-8fdea8-openshift-default-98d979769-skktq"
Oct 30 19:59:31.474453 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:31.474402    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/131a2776-cefc-4c87-9fd7-133ebe2a4e27-workload-certs\") pod \"e2e-noten-8fdea8-openshift-default-98d979769-skktq\" (UID: \"131a2776-cefc-4c87-9fd7-133ebe2a4e27\") " pod="openshift-ingress/e2e-noten-8fdea8-openshift-default-98d979769-skktq"
Oct 30 19:59:31.474453 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:31.474431    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/131a2776-cefc-4c87-9fd7-133ebe2a4e27-workload-socket\") pod \"e2e-noten-8fdea8-openshift-default-98d979769-skktq\" (UID: \"131a2776-cefc-4c87-9fd7-133ebe2a4e27\") " pod="openshift-ingress/e2e-noten-8fdea8-openshift-default-98d979769-skktq"
Oct 30 19:59:31.474554 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:31.474455    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jsvvp\" (UniqueName: \"kubernetes.io/projected/131a2776-cefc-4c87-9fd7-133ebe2a4e27-kube-api-access-jsvvp\") pod \"e2e-noten-8fdea8-openshift-default-98d979769-skktq\" (UID: \"131a2776-cefc-4c87-9fd7-133ebe2a4e27\") " pod="openshift-ingress/e2e-noten-8fdea8-openshift-default-98d979769-skktq"
Oct 30 19:59:31.576197 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:31.576154    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/131a2776-cefc-4c87-9fd7-133ebe2a4e27-istio-podinfo\") pod \"e2e-noten-8fdea8-openshift-default-98d979769-skktq\" (UID: \"131a2776-cefc-4c87-9fd7-133ebe2a4e27\") " pod="openshift-ingress/e2e-noten-8fdea8-openshift-default-98d979769-skktq"
Oct 30 19:59:31.576563 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:31.576534    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/131a2776-cefc-4c87-9fd7-133ebe2a4e27-istio-token\") pod \"e2e-noten-8fdea8-openshift-default-98d979769-skktq\" (UID: \"131a2776-cefc-4c87-9fd7-133ebe2a4e27\") " pod="openshift-ingress/e2e-noten-8fdea8-openshift-default-98d979769-skktq"
Oct 30 19:59:31.576763 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:31.576745    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/131a2776-cefc-4c87-9fd7-133ebe2a4e27-istio-envoy\") pod \"e2e-noten-8fdea8-openshift-default-98d979769-skktq\" (UID: \"131a2776-cefc-4c87-9fd7-133ebe2a4e27\") " pod="openshift-ingress/e2e-noten-8fdea8-openshift-default-98d979769-skktq"
Oct 30 19:59:31.576876 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:31.576862    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/131a2776-cefc-4c87-9fd7-133ebe2a4e27-istio-data\") pod \"e2e-noten-8fdea8-openshift-default-98d979769-skktq\" (UID: \"131a2776-cefc-4c87-9fd7-133ebe2a4e27\") " pod="openshift-ingress/e2e-noten-8fdea8-openshift-default-98d979769-skktq"
Oct 30 19:59:31.577003 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:31.576988    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/131a2776-cefc-4c87-9fd7-133ebe2a4e27-credential-socket\") pod \"e2e-noten-8fdea8-openshift-default-98d979769-skktq\" (UID: \"131a2776-cefc-4c87-9fd7-133ebe2a4e27\") " pod="openshift-ingress/e2e-noten-8fdea8-openshift-default-98d979769-skktq"
Oct 30 19:59:31.577116 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:31.577101    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/131a2776-cefc-4c87-9fd7-133ebe2a4e27-istiod-ca-cert\") pod \"e2e-noten-8fdea8-openshift-default-98d979769-skktq\" (UID: \"131a2776-cefc-4c87-9fd7-133ebe2a4e27\") " pod="openshift-ingress/e2e-noten-8fdea8-openshift-default-98d979769-skktq"
Oct 30 19:59:31.580500 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:31.577283    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/131a2776-cefc-4c87-9fd7-133ebe2a4e27-workload-certs\") pod \"e2e-noten-8fdea8-openshift-default-98d979769-skktq\" (UID: \"131a2776-cefc-4c87-9fd7-133ebe2a4e27\") " pod="openshift-ingress/e2e-noten-8fdea8-openshift-default-98d979769-skktq"
Oct 30 19:59:31.591509 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:31.580686    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/131a2776-cefc-4c87-9fd7-133ebe2a4e27-workload-socket\") pod \"e2e-noten-8fdea8-openshift-default-98d979769-skktq\" (UID: \"131a2776-cefc-4c87-9fd7-133ebe2a4e27\") " pod="openshift-ingress/e2e-noten-8fdea8-openshift-default-98d979769-skktq"
Oct 30 19:59:31.592600 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:31.592212    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/131a2776-cefc-4c87-9fd7-133ebe2a4e27-credential-socket\") pod \"e2e-noten-8fdea8-openshift-default-98d979769-skktq\" (UID: \"131a2776-cefc-4c87-9fd7-133ebe2a4e27\") " pod="openshift-ingress/e2e-noten-8fdea8-openshift-default-98d979769-skktq"
Oct 30 19:59:31.592600 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:31.592519    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/131a2776-cefc-4c87-9fd7-133ebe2a4e27-istiod-ca-cert\") pod \"e2e-noten-8fdea8-openshift-default-98d979769-skktq\" (UID: \"131a2776-cefc-4c87-9fd7-133ebe2a4e27\") " pod="openshift-ingress/e2e-noten-8fdea8-openshift-default-98d979769-skktq"
Oct 30 19:59:31.592793 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:31.592674    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/131a2776-cefc-4c87-9fd7-133ebe2a4e27-istio-data\") pod \"e2e-noten-8fdea8-openshift-default-98d979769-skktq\" (UID: \"131a2776-cefc-4c87-9fd7-133ebe2a4e27\") " pod="openshift-ingress/e2e-noten-8fdea8-openshift-default-98d979769-skktq"
Oct 30 19:59:31.598799 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:31.598747    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/131a2776-cefc-4c87-9fd7-133ebe2a4e27-workload-certs\") pod \"e2e-noten-8fdea8-openshift-default-98d979769-skktq\" (UID: \"131a2776-cefc-4c87-9fd7-133ebe2a4e27\") " pod="openshift-ingress/e2e-noten-8fdea8-openshift-default-98d979769-skktq"
Oct 30 19:59:31.599006 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:31.598983    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jsvvp\" (UniqueName: \"kubernetes.io/projected/131a2776-cefc-4c87-9fd7-133ebe2a4e27-kube-api-access-jsvvp\") pod \"e2e-noten-8fdea8-openshift-default-98d979769-skktq\" (UID: \"131a2776-cefc-4c87-9fd7-133ebe2a4e27\") " pod="openshift-ingress/e2e-noten-8fdea8-openshift-default-98d979769-skktq"
Oct 30 19:59:31.617506 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:31.599650    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/131a2776-cefc-4c87-9fd7-133ebe2a4e27-workload-socket\") pod \"e2e-noten-8fdea8-openshift-default-98d979769-skktq\" (UID: \"131a2776-cefc-4c87-9fd7-133ebe2a4e27\") " pod="openshift-ingress/e2e-noten-8fdea8-openshift-default-98d979769-skktq"
Oct 30 19:59:31.617727 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:31.612055    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/131a2776-cefc-4c87-9fd7-133ebe2a4e27-istio-podinfo\") pod \"e2e-noten-8fdea8-openshift-default-98d979769-skktq\" (UID: \"131a2776-cefc-4c87-9fd7-133ebe2a4e27\") " pod="openshift-ingress/e2e-noten-8fdea8-openshift-default-98d979769-skktq"
Oct 30 19:59:31.617823 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:31.615420    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/131a2776-cefc-4c87-9fd7-133ebe2a4e27-istio-token\") pod \"e2e-noten-8fdea8-openshift-default-98d979769-skktq\" (UID: \"131a2776-cefc-4c87-9fd7-133ebe2a4e27\") " pod="openshift-ingress/e2e-noten-8fdea8-openshift-default-98d979769-skktq"
Oct 30 19:59:31.620556 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:31.620526    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jsvvp\" (UniqueName: \"kubernetes.io/projected/131a2776-cefc-4c87-9fd7-133ebe2a4e27-kube-api-access-jsvvp\") pod \"e2e-noten-8fdea8-openshift-default-98d979769-skktq\" (UID: \"131a2776-cefc-4c87-9fd7-133ebe2a4e27\") " pod="openshift-ingress/e2e-noten-8fdea8-openshift-default-98d979769-skktq"
Oct 30 19:59:31.625719 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:31.624750    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/131a2776-cefc-4c87-9fd7-133ebe2a4e27-istio-envoy\") pod \"e2e-noten-8fdea8-openshift-default-98d979769-skktq\" (UID: \"131a2776-cefc-4c87-9fd7-133ebe2a4e27\") " pod="openshift-ingress/e2e-noten-8fdea8-openshift-default-98d979769-skktq"
Oct 30 19:59:31.702716 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:31.702681    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-8fdea8-openshift-default-98d979769-skktq"
Oct 30 19:59:32.007719 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:32.007691    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-8fdea8-openshift-default-98d979769-skktq"]
Oct 30 19:59:32.009151 ip-10-0-141-42 kubenswrapper[6996]: W1030 19:59:32.009125    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod131a2776_cefc_4c87_9fd7_133ebe2a4e27.slice/crio-a57559ec0e3573376828b3932681eccbb4f95c458c75b87a6f5007f4dc8a9cfa WatchSource:0}: Error finding container a57559ec0e3573376828b3932681eccbb4f95c458c75b87a6f5007f4dc8a9cfa: Status 404 returned error can't find the container with id a57559ec0e3573376828b3932681eccbb4f95c458c75b87a6f5007f4dc8a9cfa
Oct 30 19:59:32.012826 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:32.012776    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:59:32.012960 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:32.012940    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:59:32.013054 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:32.013032    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:59:32.738959 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:32.738913    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-8fdea8-openshift-default-98d979769-skktq" event={"ID":"131a2776-cefc-4c87-9fd7-133ebe2a4e27","Type":"ContainerStarted","Data":"45abad8bbf573e83a8a311452b2eadaec429379119b84ec144d1e3c3f015df20"}
Oct 30 19:59:32.739371 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:32.739006    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-8fdea8-openshift-default-98d979769-skktq" event={"ID":"131a2776-cefc-4c87-9fd7-133ebe2a4e27","Type":"ContainerStarted","Data":"a57559ec0e3573376828b3932681eccbb4f95c458c75b87a6f5007f4dc8a9cfa"}
Oct 30 19:59:32.768705 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:32.768647    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-noten-8fdea8-openshift-default-98d979769-skktq" podStartSLOduration=1.768633001 podStartE2EDuration="1.768633001s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:59:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:59:32.762587587 +0000 UTC m=+1648.090705740" watchObservedRunningTime="2026-10-30 19:59:32.768633001 +0000 UTC m=+1648.096751154"
Oct 30 19:59:33.703041 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:33.703005    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-noten-8fdea8-openshift-default-98d979769-skktq"
Oct 30 19:59:33.710750 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:33.710720    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-noten-8fdea8-openshift-default-98d979769-skktq"
Oct 30 19:59:33.747605 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:33.747558    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-noten-8fdea8-openshift-default-98d979769-skktq"
Oct 30 19:59:33.748515 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:33.748465    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-noten-8fdea8-openshift-default-98d979769-skktq"
Oct 30 19:59:35.535937 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:35.535910    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8"
Oct 30 19:59:35.567247 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:35.567186    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/de10228b-eec3-408f-b4cd-5250da381206-kube-api-access-4s9ld\" (UniqueName: \"kubernetes.io/projected/de10228b-eec3-408f-b4cd-5250da381206-kube-api-access-4s9ld\") pod \"de10228b-eec3-408f-b4cd-5250da381206\" (UID: \"de10228b-eec3-408f-b4cd-5250da381206\") "
Oct 30 19:59:35.567461 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:35.567410    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/de10228b-eec3-408f-b4cd-5250da381206-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/de10228b-eec3-408f-b4cd-5250da381206-workload-certs\") pod \"de10228b-eec3-408f-b4cd-5250da381206\" (UID: \"de10228b-eec3-408f-b4cd-5250da381206\") "
Oct 30 19:59:35.567587 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:35.567459    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/de10228b-eec3-408f-b4cd-5250da381206-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/de10228b-eec3-408f-b4cd-5250da381206-istio-data\") pod \"de10228b-eec3-408f-b4cd-5250da381206\" (UID: \"de10228b-eec3-408f-b4cd-5250da381206\") "
Oct 30 19:59:35.567587 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:35.567517    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/de10228b-eec3-408f-b4cd-5250da381206-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/de10228b-eec3-408f-b4cd-5250da381206-istiod-ca-cert\") pod \"de10228b-eec3-408f-b4cd-5250da381206\" (UID: \"de10228b-eec3-408f-b4cd-5250da381206\") "
Oct 30 19:59:35.567587 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:35.567554    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/de10228b-eec3-408f-b4cd-5250da381206-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/de10228b-eec3-408f-b4cd-5250da381206-istio-envoy\") pod \"de10228b-eec3-408f-b4cd-5250da381206\" (UID: \"de10228b-eec3-408f-b4cd-5250da381206\") "
Oct 30 19:59:35.567587 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:35.567579    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/de10228b-eec3-408f-b4cd-5250da381206-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/de10228b-eec3-408f-b4cd-5250da381206-credential-socket\") pod \"de10228b-eec3-408f-b4cd-5250da381206\" (UID: \"de10228b-eec3-408f-b4cd-5250da381206\") "
Oct 30 19:59:35.567827 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:35.567627    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/de10228b-eec3-408f-b4cd-5250da381206-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/de10228b-eec3-408f-b4cd-5250da381206-workload-socket\") pod \"de10228b-eec3-408f-b4cd-5250da381206\" (UID: \"de10228b-eec3-408f-b4cd-5250da381206\") "
Oct 30 19:59:35.567827 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:35.567651    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/de10228b-eec3-408f-b4cd-5250da381206-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/de10228b-eec3-408f-b4cd-5250da381206-istio-podinfo\") pod \"de10228b-eec3-408f-b4cd-5250da381206\" (UID: \"de10228b-eec3-408f-b4cd-5250da381206\") "
Oct 30 19:59:35.567827 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:35.567678    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/de10228b-eec3-408f-b4cd-5250da381206-istio-token\" (UniqueName: \"kubernetes.io/projected/de10228b-eec3-408f-b4cd-5250da381206-istio-token\") pod \"de10228b-eec3-408f-b4cd-5250da381206\" (UID: \"de10228b-eec3-408f-b4cd-5250da381206\") "
Oct 30 19:59:35.569007 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:35.568974    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de10228b-eec3-408f-b4cd-5250da381206-istiod-ca-cert" pod "de10228b-eec3-408f-b4cd-5250da381206" (UID: "de10228b-eec3-408f-b4cd-5250da381206"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:59:35.569906 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:35.569410    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de10228b-eec3-408f-b4cd-5250da381206-workload-certs" pod "de10228b-eec3-408f-b4cd-5250da381206" (UID: "de10228b-eec3-408f-b4cd-5250da381206"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:59:35.569906 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:35.569673    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de10228b-eec3-408f-b4cd-5250da381206-workload-socket" pod "de10228b-eec3-408f-b4cd-5250da381206" (UID: "de10228b-eec3-408f-b4cd-5250da381206"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:59:35.569906 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:35.569856    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de10228b-eec3-408f-b4cd-5250da381206-credential-socket" pod "de10228b-eec3-408f-b4cd-5250da381206" (UID: "de10228b-eec3-408f-b4cd-5250da381206"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:59:35.569906 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:35.569874    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de10228b-eec3-408f-b4cd-5250da381206-istio-data" pod "de10228b-eec3-408f-b4cd-5250da381206" (UID: "de10228b-eec3-408f-b4cd-5250da381206"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:59:35.570941 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:35.570914    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de10228b-eec3-408f-b4cd-5250da381206-istio-token" pod "de10228b-eec3-408f-b4cd-5250da381206" (UID: "de10228b-eec3-408f-b4cd-5250da381206"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:59:35.571744 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:35.571719    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de10228b-eec3-408f-b4cd-5250da381206-kube-api-access-4s9ld" pod "de10228b-eec3-408f-b4cd-5250da381206" (UID: "de10228b-eec3-408f-b4cd-5250da381206"). InnerVolumeSpecName "kube-api-access-4s9ld". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:59:35.572017 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:35.571994    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/de10228b-eec3-408f-b4cd-5250da381206-istio-podinfo" pod "de10228b-eec3-408f-b4cd-5250da381206" (UID: "de10228b-eec3-408f-b4cd-5250da381206"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:59:35.572959 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:35.572933    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de10228b-eec3-408f-b4cd-5250da381206-istio-envoy" pod "de10228b-eec3-408f-b4cd-5250da381206" (UID: "de10228b-eec3-408f-b4cd-5250da381206"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:59:35.703507 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:35.696325    6996 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/de10228b-eec3-408f-b4cd-5250da381206-workload-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:35.703507 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:35.696369    6996 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/de10228b-eec3-408f-b4cd-5250da381206-istio-data\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:35.703507 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:35.696384    6996 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/de10228b-eec3-408f-b4cd-5250da381206-istiod-ca-cert\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:35.703507 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:35.696397    6996 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/de10228b-eec3-408f-b4cd-5250da381206-istio-envoy\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:35.703507 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:35.696414    6996 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/de10228b-eec3-408f-b4cd-5250da381206-credential-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:35.703507 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:35.696428    6996 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/de10228b-eec3-408f-b4cd-5250da381206-workload-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:35.703507 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:35.696442    6996 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/de10228b-eec3-408f-b4cd-5250da381206-istio-podinfo\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:35.703507 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:35.696458    6996 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de10228b-eec3-408f-b4cd-5250da381206-istio-token\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:35.703507 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:35.696471    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4s9ld\" (UniqueName: \"kubernetes.io/projected/de10228b-eec3-408f-b4cd-5250da381206-kube-api-access-4s9ld\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:35.762761 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:35.762678    6996 generic.go:356] "Generic (PLEG): container finished" podID="de10228b-eec3-408f-b4cd-5250da381206" containerID="7f3ebe7dbb29a59e73a6fa8b08a0760a1c428b0ac51552473584b3a7311d7077" exitCode=0
Oct 30 19:59:35.762761 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:35.762736    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8"
Oct 30 19:59:35.762972 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:35.762762    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8" event={"ID":"de10228b-eec3-408f-b4cd-5250da381206","Type":"ContainerDied","Data":"7f3ebe7dbb29a59e73a6fa8b08a0760a1c428b0ac51552473584b3a7311d7077"}
Oct 30 19:59:35.762972 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:35.762820    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8" event={"ID":"de10228b-eec3-408f-b4cd-5250da381206","Type":"ContainerDied","Data":"ef2560c425acf98daa3684b535ddc7d47ba345daa473ad31708cbe475655afac"}
Oct 30 19:59:35.762972 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:35.762845    6996 scope.go:122] "RemoveContainer" containerID="7f3ebe7dbb29a59e73a6fa8b08a0760a1c428b0ac51552473584b3a7311d7077"
Oct 30 19:59:35.774397 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:35.774377    6996 scope.go:122] "RemoveContainer" containerID="7f3ebe7dbb29a59e73a6fa8b08a0760a1c428b0ac51552473584b3a7311d7077"
Oct 30 19:59:35.774683 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:59:35.774662    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f3ebe7dbb29a59e73a6fa8b08a0760a1c428b0ac51552473584b3a7311d7077\": container with ID starting with 7f3ebe7dbb29a59e73a6fa8b08a0760a1c428b0ac51552473584b3a7311d7077 not found: ID does not exist" containerID="7f3ebe7dbb29a59e73a6fa8b08a0760a1c428b0ac51552473584b3a7311d7077"
Oct 30 19:59:35.774743 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:35.774691    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f3ebe7dbb29a59e73a6fa8b08a0760a1c428b0ac51552473584b3a7311d7077"} err="failed to get container status \"7f3ebe7dbb29a59e73a6fa8b08a0760a1c428b0ac51552473584b3a7311d7077\": rpc error: code = NotFound desc = could not find container \"7f3ebe7dbb29a59e73a6fa8b08a0760a1c428b0ac51552473584b3a7311d7077\": container with ID starting with 7f3ebe7dbb29a59e73a6fa8b08a0760a1c428b0ac51552473584b3a7311d7077 not found: ID does not exist"
Oct 30 19:59:35.789273 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:35.789247    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8"]
Oct 30 19:59:35.795561 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:35.795537    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8"]
Oct 30 19:59:36.509892 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:36.509856    6996 patch_prober.go:29] interesting pod/e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.147:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:59:36.510162 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:36.509913    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-explicit-f0e884-openshift-default-6474bdb5df-877z8" podUID="de10228b-eec3-408f-b4cd-5250da381206" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.147:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:59:36.792526 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:36.792426    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="de10228b-eec3-408f-b4cd-5250da381206" path="/var/lib/kubelet/pods/de10228b-eec3-408f-b4cd-5250da381206/volumes"
Oct 30 19:59:46.240539 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:46.240459    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-8fdea8-openshift-default-98d979769-skktq"]
Oct 30 19:59:46.240963 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:46.240746    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-noten-8fdea8-openshift-default-98d979769-skktq" podUID="131a2776-cefc-4c87-9fd7-133ebe2a4e27" containerName="istio-proxy" containerID="cri-o://45abad8bbf573e83a8a311452b2eadaec429379119b84ec144d1e3c3f015df20" gracePeriod=30
Oct 30 19:59:48.749931 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:48.749887    6996 patch_prober.go:29] interesting pod/e2e-noten-8fdea8-openshift-default-98d979769-skktq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:59:48.750431 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:48.749959    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-noten-8fdea8-openshift-default-98d979769-skktq" podUID="131a2776-cefc-4c87-9fd7-133ebe2a4e27" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:59:50.997225 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:50.997188    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 19:59:51.529942 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:51.529857    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-8fdea8-openshift-default-98d979769-skktq"
Oct 30 19:59:51.639371 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:51.639330    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/131a2776-cefc-4c87-9fd7-133ebe2a4e27-istio-token\" (UniqueName: \"kubernetes.io/projected/131a2776-cefc-4c87-9fd7-133ebe2a4e27-istio-token\") pod \"131a2776-cefc-4c87-9fd7-133ebe2a4e27\" (UID: \"131a2776-cefc-4c87-9fd7-133ebe2a4e27\") "
Oct 30 19:59:51.639592 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:51.639448    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/131a2776-cefc-4c87-9fd7-133ebe2a4e27-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/131a2776-cefc-4c87-9fd7-133ebe2a4e27-istio-data\") pod \"131a2776-cefc-4c87-9fd7-133ebe2a4e27\" (UID: \"131a2776-cefc-4c87-9fd7-133ebe2a4e27\") "
Oct 30 19:59:51.639592 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:51.639530    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/131a2776-cefc-4c87-9fd7-133ebe2a4e27-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/131a2776-cefc-4c87-9fd7-133ebe2a4e27-workload-socket\") pod \"131a2776-cefc-4c87-9fd7-133ebe2a4e27\" (UID: \"131a2776-cefc-4c87-9fd7-133ebe2a4e27\") "
Oct 30 19:59:51.639592 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:51.639576    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/131a2776-cefc-4c87-9fd7-133ebe2a4e27-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/131a2776-cefc-4c87-9fd7-133ebe2a4e27-istio-podinfo\") pod \"131a2776-cefc-4c87-9fd7-133ebe2a4e27\" (UID: \"131a2776-cefc-4c87-9fd7-133ebe2a4e27\") "
Oct 30 19:59:51.639718 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:51.639646    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/131a2776-cefc-4c87-9fd7-133ebe2a4e27-kube-api-access-jsvvp\" (UniqueName: \"kubernetes.io/projected/131a2776-cefc-4c87-9fd7-133ebe2a4e27-kube-api-access-jsvvp\") pod \"131a2776-cefc-4c87-9fd7-133ebe2a4e27\" (UID: \"131a2776-cefc-4c87-9fd7-133ebe2a4e27\") "
Oct 30 19:59:51.640039 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:51.639984    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/131a2776-cefc-4c87-9fd7-133ebe2a4e27-workload-socket" pod "131a2776-cefc-4c87-9fd7-133ebe2a4e27" (UID: "131a2776-cefc-4c87-9fd7-133ebe2a4e27"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:59:51.640039 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:51.640010    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/131a2776-cefc-4c87-9fd7-133ebe2a4e27-istio-data" pod "131a2776-cefc-4c87-9fd7-133ebe2a4e27" (UID: "131a2776-cefc-4c87-9fd7-133ebe2a4e27"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:59:51.640532 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:51.640291    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/131a2776-cefc-4c87-9fd7-133ebe2a4e27-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/131a2776-cefc-4c87-9fd7-133ebe2a4e27-workload-certs\") pod \"131a2776-cefc-4c87-9fd7-133ebe2a4e27\" (UID: \"131a2776-cefc-4c87-9fd7-133ebe2a4e27\") "
Oct 30 19:59:51.640532 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:51.640345    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/131a2776-cefc-4c87-9fd7-133ebe2a4e27-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/131a2776-cefc-4c87-9fd7-133ebe2a4e27-istio-envoy\") pod \"131a2776-cefc-4c87-9fd7-133ebe2a4e27\" (UID: \"131a2776-cefc-4c87-9fd7-133ebe2a4e27\") "
Oct 30 19:59:51.640532 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:51.640374    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/131a2776-cefc-4c87-9fd7-133ebe2a4e27-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/131a2776-cefc-4c87-9fd7-133ebe2a4e27-istiod-ca-cert\") pod \"131a2776-cefc-4c87-9fd7-133ebe2a4e27\" (UID: \"131a2776-cefc-4c87-9fd7-133ebe2a4e27\") "
Oct 30 19:59:51.640700 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:51.640637    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/131a2776-cefc-4c87-9fd7-133ebe2a4e27-workload-certs" pod "131a2776-cefc-4c87-9fd7-133ebe2a4e27" (UID: "131a2776-cefc-4c87-9fd7-133ebe2a4e27"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:59:51.640856 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:51.640831    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/131a2776-cefc-4c87-9fd7-133ebe2a4e27-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/131a2776-cefc-4c87-9fd7-133ebe2a4e27-credential-socket\") pod \"131a2776-cefc-4c87-9fd7-133ebe2a4e27\" (UID: \"131a2776-cefc-4c87-9fd7-133ebe2a4e27\") "
Oct 30 19:59:51.641334 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:51.641308    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/131a2776-cefc-4c87-9fd7-133ebe2a4e27-credential-socket" pod "131a2776-cefc-4c87-9fd7-133ebe2a4e27" (UID: "131a2776-cefc-4c87-9fd7-133ebe2a4e27"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:59:51.641568 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:51.641519    6996 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/131a2776-cefc-4c87-9fd7-133ebe2a4e27-credential-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:51.641649 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:51.641579    6996 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/131a2776-cefc-4c87-9fd7-133ebe2a4e27-istio-data\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:51.641649 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:51.641594    6996 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/131a2776-cefc-4c87-9fd7-133ebe2a4e27-workload-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:51.641649 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:51.641609    6996 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/131a2776-cefc-4c87-9fd7-133ebe2a4e27-workload-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:51.641649 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:51.641637    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/131a2776-cefc-4c87-9fd7-133ebe2a4e27-istiod-ca-cert" pod "131a2776-cefc-4c87-9fd7-133ebe2a4e27" (UID: "131a2776-cefc-4c87-9fd7-133ebe2a4e27"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:59:51.642211 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:51.642191    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/131a2776-cefc-4c87-9fd7-133ebe2a4e27-istio-token" pod "131a2776-cefc-4c87-9fd7-133ebe2a4e27" (UID: "131a2776-cefc-4c87-9fd7-133ebe2a4e27"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:59:51.642356 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:51.642331    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/131a2776-cefc-4c87-9fd7-133ebe2a4e27-istio-podinfo" pod "131a2776-cefc-4c87-9fd7-133ebe2a4e27" (UID: "131a2776-cefc-4c87-9fd7-133ebe2a4e27"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:59:51.642787 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:51.642766    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/131a2776-cefc-4c87-9fd7-133ebe2a4e27-istio-envoy" pod "131a2776-cefc-4c87-9fd7-133ebe2a4e27" (UID: "131a2776-cefc-4c87-9fd7-133ebe2a4e27"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:59:51.642959 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:51.642938    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/131a2776-cefc-4c87-9fd7-133ebe2a4e27-kube-api-access-jsvvp" pod "131a2776-cefc-4c87-9fd7-133ebe2a4e27" (UID: "131a2776-cefc-4c87-9fd7-133ebe2a4e27"). InnerVolumeSpecName "kube-api-access-jsvvp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:59:51.742570 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:51.742519    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jsvvp\" (UniqueName: \"kubernetes.io/projected/131a2776-cefc-4c87-9fd7-133ebe2a4e27-kube-api-access-jsvvp\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:51.742570 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:51.742558    6996 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/131a2776-cefc-4c87-9fd7-133ebe2a4e27-istio-envoy\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:51.742570 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:51.742567    6996 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/131a2776-cefc-4c87-9fd7-133ebe2a4e27-istiod-ca-cert\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:51.742570 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:51.742578    6996 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/131a2776-cefc-4c87-9fd7-133ebe2a4e27-istio-token\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:51.742945 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:51.742587    6996 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/131a2776-cefc-4c87-9fd7-133ebe2a4e27-istio-podinfo\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:51.913036 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:51.913000    6996 generic.go:356] "Generic (PLEG): container finished" podID="131a2776-cefc-4c87-9fd7-133ebe2a4e27" containerID="45abad8bbf573e83a8a311452b2eadaec429379119b84ec144d1e3c3f015df20" exitCode=0
Oct 30 19:59:51.913223 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:51.913044    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-8fdea8-openshift-default-98d979769-skktq"
Oct 30 19:59:51.913223 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:51.913081    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-8fdea8-openshift-default-98d979769-skktq" event={"ID":"131a2776-cefc-4c87-9fd7-133ebe2a4e27","Type":"ContainerDied","Data":"45abad8bbf573e83a8a311452b2eadaec429379119b84ec144d1e3c3f015df20"}
Oct 30 19:59:51.913223 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:51.913134    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-8fdea8-openshift-default-98d979769-skktq" event={"ID":"131a2776-cefc-4c87-9fd7-133ebe2a4e27","Type":"ContainerDied","Data":"a57559ec0e3573376828b3932681eccbb4f95c458c75b87a6f5007f4dc8a9cfa"}
Oct 30 19:59:51.913223 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:51.913156    6996 scope.go:122] "RemoveContainer" containerID="45abad8bbf573e83a8a311452b2eadaec429379119b84ec144d1e3c3f015df20"
Oct 30 19:59:51.927527 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:51.927498    6996 scope.go:122] "RemoveContainer" containerID="45abad8bbf573e83a8a311452b2eadaec429379119b84ec144d1e3c3f015df20"
Oct 30 19:59:51.928802 ip-10-0-141-42 kubenswrapper[6996]: E1030 19:59:51.928779    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45abad8bbf573e83a8a311452b2eadaec429379119b84ec144d1e3c3f015df20\": container with ID starting with 45abad8bbf573e83a8a311452b2eadaec429379119b84ec144d1e3c3f015df20 not found: ID does not exist" containerID="45abad8bbf573e83a8a311452b2eadaec429379119b84ec144d1e3c3f015df20"
Oct 30 19:59:51.928905 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:51.928811    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45abad8bbf573e83a8a311452b2eadaec429379119b84ec144d1e3c3f015df20"} err="failed to get container status \"45abad8bbf573e83a8a311452b2eadaec429379119b84ec144d1e3c3f015df20\": rpc error: code = NotFound desc = could not find container \"45abad8bbf573e83a8a311452b2eadaec429379119b84ec144d1e3c3f015df20\": container with ID starting with 45abad8bbf573e83a8a311452b2eadaec429379119b84ec144d1e3c3f015df20 not found: ID does not exist"
Oct 30 19:59:52.028992 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:52.028948    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-8fdea8-openshift-default-98d979769-skktq"]
Oct 30 19:59:52.060504 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:52.048020    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-noten-8fdea8-openshift-default-98d979769-skktq"]
Oct 30 19:59:52.793415 ip-10-0-141-42 kubenswrapper[6996]: I1030 19:59:52.793382    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="131a2776-cefc-4c87-9fd7-133ebe2a4e27" path="/var/lib/kubelet/pods/131a2776-cefc-4c87-9fd7-133ebe2a4e27/volumes"
Oct 30 20:00:01.199020 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:00:01.198984    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 20:00:05.497269 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:05.497234    6996 scope.go:122] "RemoveContainer" containerID="456b89a23267803472b4c6bd5cb8e187fd6a560a049e0080719f056e9d5fff33"
Oct 30 20:00:31.233783 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:00:31.233742    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 20:00:36.349170 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:36.349135    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 20:00:45.153602 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:45.153562    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-4d2317-75c7f66df4-4784l"]
Oct 30 20:00:45.154581 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:45.154557    6996 cpu_manager.go:414] "containerMap: removing container" podUID="de10228b-eec3-408f-b4cd-5250da381206" containerName="istio-proxy"
Oct 30 20:00:45.154581 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:45.154581    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="de10228b-eec3-408f-b4cd-5250da381206" containerName="istio-proxy"
Oct 30 20:00:45.154774 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:45.154654    6996 cpu_manager.go:414] "containerMap: removing container" podUID="131a2776-cefc-4c87-9fd7-133ebe2a4e27" containerName="istio-proxy"
Oct 30 20:00:45.154774 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:45.154663    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="131a2776-cefc-4c87-9fd7-133ebe2a4e27" containerName="istio-proxy"
Oct 30 20:00:45.154897 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:45.154832    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="131a2776-cefc-4c87-9fd7-133ebe2a4e27" containerName="istio-proxy"
Oct 30 20:00:45.154897 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:45.154852    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="de10228b-eec3-408f-b4cd-5250da381206" containerName="istio-proxy"
Oct 30 20:00:45.159417 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:45.159396    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-4d2317-75c7f66df4-4784l"
Oct 30 20:00:45.185668 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:45.185643    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-4d2317-75c7f66df4-4784l"]
Oct 30 20:00:45.206030 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:45.205973    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w0-4d2317-dockercfg-29g2n\""
Oct 30 20:00:45.210136 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:45.209813    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rd6j7\" (UniqueName: \"kubernetes.io/projected/f6e147bc-ed53-4e12-b8f3-23a9e0ab12e2-kube-api-access-rd6j7\") pod \"payload-processing-e2e-shared-a-w0-4d2317-75c7f66df4-4784l\" (UID: \"f6e147bc-ed53-4e12-b8f3-23a9e0ab12e2\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-4d2317-75c7f66df4-4784l"
Oct 30 20:00:45.210136 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:45.210000    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6e147bc-ed53-4e12-b8f3-23a9e0ab12e2-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w0-4d2317-75c7f66df4-4784l\" (UID: \"f6e147bc-ed53-4e12-b8f3-23a9e0ab12e2\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-4d2317-75c7f66df4-4784l"
Oct 30 20:00:45.214688 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:45.214671    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w0-4d2317\""
Oct 30 20:00:45.318493 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:45.318435    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6e147bc-ed53-4e12-b8f3-23a9e0ab12e2-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w0-4d2317-75c7f66df4-4784l\" (UID: \"f6e147bc-ed53-4e12-b8f3-23a9e0ab12e2\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-4d2317-75c7f66df4-4784l"
Oct 30 20:00:45.340810 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:45.325828    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6e147bc-ed53-4e12-b8f3-23a9e0ab12e2-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w0-4d2317-75c7f66df4-4784l\" (UID: \"f6e147bc-ed53-4e12-b8f3-23a9e0ab12e2\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-4d2317-75c7f66df4-4784l"
Oct 30 20:00:45.341310 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:45.341281    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rd6j7\" (UniqueName: \"kubernetes.io/projected/f6e147bc-ed53-4e12-b8f3-23a9e0ab12e2-kube-api-access-rd6j7\") pod \"payload-processing-e2e-shared-a-w0-4d2317-75c7f66df4-4784l\" (UID: \"f6e147bc-ed53-4e12-b8f3-23a9e0ab12e2\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-4d2317-75c7f66df4-4784l"
Oct 30 20:00:45.374409 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:45.374378    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rd6j7\" (UniqueName: \"kubernetes.io/projected/f6e147bc-ed53-4e12-b8f3-23a9e0ab12e2-kube-api-access-rd6j7\") pod \"payload-processing-e2e-shared-a-w0-4d2317-75c7f66df4-4784l\" (UID: \"f6e147bc-ed53-4e12-b8f3-23a9e0ab12e2\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-4d2317-75c7f66df4-4784l"
Oct 30 20:00:45.461599 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:45.461512    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88"]
Oct 30 20:00:45.466351 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:45.466322    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88"
Oct 30 20:00:45.472027 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:45.471996    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w1-7eaeed-openshift-default-dockercfg-7hrlh\""
Oct 30 20:00:45.478163 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:45.478135    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-4d2317-75c7f66df4-4784l"
Oct 30 20:00:45.486702 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:45.486639    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88"]
Oct 30 20:00:45.545036 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:45.544993    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d-istio-token\") pod \"e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88\" (UID: \"a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d\") " pod="openshift-ingress/e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88"
Oct 30 20:00:45.545268 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:45.545060    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d-credential-socket\") pod \"e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88\" (UID: \"a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d\") " pod="openshift-ingress/e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88"
Oct 30 20:00:45.545268 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:45.545134    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d-workload-certs\") pod \"e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88\" (UID: \"a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d\") " pod="openshift-ingress/e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88"
Oct 30 20:00:45.545268 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:45.545199    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d-istio-envoy\") pod \"e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88\" (UID: \"a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d\") " pod="openshift-ingress/e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88"
Oct 30 20:00:45.545268 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:45.545234    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d-workload-socket\") pod \"e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88\" (UID: \"a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d\") " pod="openshift-ingress/e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88"
Oct 30 20:00:45.545637 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:45.545330    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d-istiod-ca-cert\") pod \"e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88\" (UID: \"a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d\") " pod="openshift-ingress/e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88"
Oct 30 20:00:45.545637 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:45.545358    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d-istio-podinfo\") pod \"e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88\" (UID: \"a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d\") " pod="openshift-ingress/e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88"
Oct 30 20:00:45.545637 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:45.545381    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plk5w\" (UniqueName: \"kubernetes.io/projected/a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d-kube-api-access-plk5w\") pod \"e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88\" (UID: \"a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d\") " pod="openshift-ingress/e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88"
Oct 30 20:00:45.545829 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:45.545644    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d-istio-data\") pod \"e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88\" (UID: \"a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d\") " pod="openshift-ingress/e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88"
Oct 30 20:00:45.647405 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:45.647367    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d-istio-data\") pod \"e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88\" (UID: \"a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d\") " pod="openshift-ingress/e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88"
Oct 30 20:00:45.647641 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:45.647454    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d-istio-token\") pod \"e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88\" (UID: \"a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d\") " pod="openshift-ingress/e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88"
Oct 30 20:00:45.647641 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:45.647518    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d-credential-socket\") pod \"e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88\" (UID: \"a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d\") " pod="openshift-ingress/e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88"
Oct 30 20:00:45.647641 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:45.647538    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d-workload-certs\") pod \"e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88\" (UID: \"a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d\") " pod="openshift-ingress/e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88"
Oct 30 20:00:45.647641 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:45.647584    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d-istio-envoy\") pod \"e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88\" (UID: \"a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d\") " pod="openshift-ingress/e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88"
Oct 30 20:00:45.647641 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:45.647607    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d-workload-socket\") pod \"e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88\" (UID: \"a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d\") " pod="openshift-ingress/e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88"
Oct 30 20:00:45.647933 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:45.647693    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d-istiod-ca-cert\") pod \"e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88\" (UID: \"a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d\") " pod="openshift-ingress/e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88"
Oct 30 20:00:45.647933 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:45.647716    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d-istio-podinfo\") pod \"e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88\" (UID: \"a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d\") " pod="openshift-ingress/e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88"
Oct 30 20:00:45.647933 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:45.647745    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plk5w\" (UniqueName: \"kubernetes.io/projected/a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d-kube-api-access-plk5w\") pod \"e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88\" (UID: \"a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d\") " pod="openshift-ingress/e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88"
Oct 30 20:00:45.647933 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:45.647881    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d-istio-data\") pod \"e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88\" (UID: \"a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d\") " pod="openshift-ingress/e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88"
Oct 30 20:00:45.647933 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:45.647926    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d-credential-socket\") pod \"e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88\" (UID: \"a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d\") " pod="openshift-ingress/e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88"
Oct 30 20:00:45.648193 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:45.648009    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d-workload-certs\") pod \"e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88\" (UID: \"a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d\") " pod="openshift-ingress/e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88"
Oct 30 20:00:45.648259 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:45.648219    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d-workload-socket\") pod \"e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88\" (UID: \"a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d\") " pod="openshift-ingress/e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88"
Oct 30 20:00:45.648422 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:45.648402    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d-istiod-ca-cert\") pod \"e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88\" (UID: \"a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d\") " pod="openshift-ingress/e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88"
Oct 30 20:00:45.649851 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:45.649827    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d-istio-envoy\") pod \"e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88\" (UID: \"a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d\") " pod="openshift-ingress/e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88"
Oct 30 20:00:45.650342 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:45.650322    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d-istio-podinfo\") pod \"e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88\" (UID: \"a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d\") " pod="openshift-ingress/e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88"
Oct 30 20:00:45.670955 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:45.670925    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-plk5w\" (UniqueName: \"kubernetes.io/projected/a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d-kube-api-access-plk5w\") pod \"e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88\" (UID: \"a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d\") " pod="openshift-ingress/e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88"
Oct 30 20:00:45.673256 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:45.673237    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d-istio-token\") pod \"e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88\" (UID: \"a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d\") " pod="openshift-ingress/e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88"
Oct 30 20:00:45.781970 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:45.781944    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-4d2317-75c7f66df4-4784l"]
Oct 30 20:00:45.782878 ip-10-0-141-42 kubenswrapper[6996]: W1030 20:00:45.782848    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6e147bc_ed53_4e12_b8f3_23a9e0ab12e2.slice/crio-5351e9a7cfa46714bc2c0e309be7eb6577e5f0b5cec4c048cbe7e9c2b83ed640 WatchSource:0}: Error finding container 5351e9a7cfa46714bc2c0e309be7eb6577e5f0b5cec4c048cbe7e9c2b83ed640: Status 404 returned error can't find the container with id 5351e9a7cfa46714bc2c0e309be7eb6577e5f0b5cec4c048cbe7e9c2b83ed640
Oct 30 20:00:45.782967 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:45.782939    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88"
Oct 30 20:00:46.084554 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:46.084524    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88"]
Oct 30 20:00:46.085737 ip-10-0-141-42 kubenswrapper[6996]: W1030 20:00:46.085714    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1c3b8a0_7dde_4d9c_97e2_8ca9a326a31d.slice/crio-c1b4c1fee9cedcddeeeb95ffc147cffc1b11f2f49221a406864b950a5becaab4 WatchSource:0}: Error finding container c1b4c1fee9cedcddeeeb95ffc147cffc1b11f2f49221a406864b950a5becaab4: Status 404 returned error can't find the container with id c1b4c1fee9cedcddeeeb95ffc147cffc1b11f2f49221a406864b950a5becaab4
Oct 30 20:00:46.088513 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:46.088466    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 20:00:46.088592 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:46.088549    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 20:00:46.088630 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:46.088588    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 20:00:46.349868 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:46.349772    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88" event={"ID":"a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d","Type":"ContainerStarted","Data":"069a1a3cdad8370d2251b30bc1ee006721315f513f0639354695499a67eb5f71"}
Oct 30 20:00:46.349868 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:46.349845    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88" event={"ID":"a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d","Type":"ContainerStarted","Data":"c1b4c1fee9cedcddeeeb95ffc147cffc1b11f2f49221a406864b950a5becaab4"}
Oct 30 20:00:46.351472 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:46.351444    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-4d2317-75c7f66df4-4784l" event={"ID":"f6e147bc-ed53-4e12-b8f3-23a9e0ab12e2","Type":"ContainerStarted","Data":"72d820e41dcd9bed1cd811b9cc99af216e115f32384a4a5db944bedf663acc08"}
Oct 30 20:00:46.351619 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:46.351526    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-4d2317-75c7f66df4-4784l" event={"ID":"f6e147bc-ed53-4e12-b8f3-23a9e0ab12e2","Type":"ContainerStarted","Data":"5351e9a7cfa46714bc2c0e309be7eb6577e5f0b5cec4c048cbe7e9c2b83ed640"}
Oct 30 20:00:46.351619 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:46.351566    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-4d2317-75c7f66df4-4784l"
Oct 30 20:00:46.388227 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:46.383040    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88" podStartSLOduration=1.383018628 podStartE2EDuration="1.383018628s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:00:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:00:46.380952014 +0000 UTC m=+1721.709070169" watchObservedRunningTime="2026-10-30 20:00:46.383018628 +0000 UTC m=+1721.711136783"
Oct 30 20:00:46.406648 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:46.406578    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-4d2317-75c7f66df4-4784l" podStartSLOduration=1.406555853 podStartE2EDuration="1.406555853s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:00:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:00:46.402301574 +0000 UTC m=+1721.730419732" watchObservedRunningTime="2026-10-30 20:00:46.406555853 +0000 UTC m=+1721.734674017"
Oct 30 20:00:46.796407 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:46.796327    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88"
Oct 30 20:00:47.785298 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:47.785257    6996 patch_prober.go:29] interesting pod/e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused" start-of-body=
Oct 30 20:00:47.785749 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:47.785315    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88" podUID="a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused"
Oct 30 20:00:48.784005 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:48.783965    6996 patch_prober.go:29] interesting pod/e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused" start-of-body=
Oct 30 20:00:48.784273 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:48.784036    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88" podUID="a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused"
Oct 30 20:00:49.784013 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:49.783975    6996 patch_prober.go:29] interesting pod/e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused" start-of-body=
Oct 30 20:00:49.784548 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:49.784031    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88" podUID="a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused"
Oct 30 20:00:50.784298 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:50.784254    6996 patch_prober.go:29] interesting pod/e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused" start-of-body=
Oct 30 20:00:50.784764 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:50.784347    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88" podUID="a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused"
Oct 30 20:00:51.783793 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:51.783754    6996 patch_prober.go:29] interesting pod/e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused" start-of-body=
Oct 30 20:00:51.784055 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:51.784007    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88" podUID="a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused"
Oct 30 20:00:52.831604 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:52.831568    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88"
Oct 30 20:00:52.833017 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:52.832989    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88"
Oct 30 20:00:52.833788 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:52.833766    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88"
Oct 30 20:00:56.962411 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:56.962375    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-4d2317-7d6cc8fd8d-ltsxv"]
Oct 30 20:00:56.967222 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:56.967195    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-4d2317-7d6cc8fd8d-ltsxv"
Oct 30 20:00:56.973311 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:56.973284    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w0-4d2317-serving-cert\""
Oct 30 20:00:56.990058 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:56.990032    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-4d2317-7d6cc8fd8d-ltsxv"]
Oct 30 20:00:57.045495 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:57.045459    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnsk5\" (UniqueName: \"kubernetes.io/projected/babec534-131b-46e9-9706-cdf7fb2f60c8-kube-api-access-tnsk5\") pod \"maas-api-e2e-shared-a-w0-4d2317-7d6cc8fd8d-ltsxv\" (UID: \"babec534-131b-46e9-9706-cdf7fb2f60c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-4d2317-7d6cc8fd8d-ltsxv"
Oct 30 20:00:57.045680 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:57.045504    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/babec534-131b-46e9-9706-cdf7fb2f60c8-maas-api-tls\") pod \"maas-api-e2e-shared-a-w0-4d2317-7d6cc8fd8d-ltsxv\" (UID: \"babec534-131b-46e9-9706-cdf7fb2f60c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-4d2317-7d6cc8fd8d-ltsxv"
Oct 30 20:00:57.045680 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:57.045652    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/babec534-131b-46e9-9706-cdf7fb2f60c8-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w0-4d2317-7d6cc8fd8d-ltsxv\" (UID: \"babec534-131b-46e9-9706-cdf7fb2f60c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-4d2317-7d6cc8fd8d-ltsxv"
Oct 30 20:00:57.045831 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:57.045811    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/babec534-131b-46e9-9706-cdf7fb2f60c8-metrics-certs\") pod \"maas-api-e2e-shared-a-w0-4d2317-7d6cc8fd8d-ltsxv\" (UID: \"babec534-131b-46e9-9706-cdf7fb2f60c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-4d2317-7d6cc8fd8d-ltsxv"
Oct 30 20:00:57.146927 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:57.146878    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/babec534-131b-46e9-9706-cdf7fb2f60c8-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w0-4d2317-7d6cc8fd8d-ltsxv\" (UID: \"babec534-131b-46e9-9706-cdf7fb2f60c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-4d2317-7d6cc8fd8d-ltsxv"
Oct 30 20:00:57.147120 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:57.147023    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/babec534-131b-46e9-9706-cdf7fb2f60c8-metrics-certs\") pod \"maas-api-e2e-shared-a-w0-4d2317-7d6cc8fd8d-ltsxv\" (UID: \"babec534-131b-46e9-9706-cdf7fb2f60c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-4d2317-7d6cc8fd8d-ltsxv"
Oct 30 20:00:57.147266 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:57.147245    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnsk5\" (UniqueName: \"kubernetes.io/projected/babec534-131b-46e9-9706-cdf7fb2f60c8-kube-api-access-tnsk5\") pod \"maas-api-e2e-shared-a-w0-4d2317-7d6cc8fd8d-ltsxv\" (UID: \"babec534-131b-46e9-9706-cdf7fb2f60c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-4d2317-7d6cc8fd8d-ltsxv"
Oct 30 20:00:57.147355 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:57.147287    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/babec534-131b-46e9-9706-cdf7fb2f60c8-maas-api-tls\") pod \"maas-api-e2e-shared-a-w0-4d2317-7d6cc8fd8d-ltsxv\" (UID: \"babec534-131b-46e9-9706-cdf7fb2f60c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-4d2317-7d6cc8fd8d-ltsxv"
Oct 30 20:00:57.148014 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:57.147987    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/babec534-131b-46e9-9706-cdf7fb2f60c8-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w0-4d2317-7d6cc8fd8d-ltsxv\" (UID: \"babec534-131b-46e9-9706-cdf7fb2f60c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-4d2317-7d6cc8fd8d-ltsxv"
Oct 30 20:00:57.150272 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:57.150241    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/babec534-131b-46e9-9706-cdf7fb2f60c8-maas-api-tls\") pod \"maas-api-e2e-shared-a-w0-4d2317-7d6cc8fd8d-ltsxv\" (UID: \"babec534-131b-46e9-9706-cdf7fb2f60c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-4d2317-7d6cc8fd8d-ltsxv"
Oct 30 20:00:57.150362 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:57.150245    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/babec534-131b-46e9-9706-cdf7fb2f60c8-metrics-certs\") pod \"maas-api-e2e-shared-a-w0-4d2317-7d6cc8fd8d-ltsxv\" (UID: \"babec534-131b-46e9-9706-cdf7fb2f60c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-4d2317-7d6cc8fd8d-ltsxv"
Oct 30 20:00:57.164821 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:57.164797    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnsk5\" (UniqueName: \"kubernetes.io/projected/babec534-131b-46e9-9706-cdf7fb2f60c8-kube-api-access-tnsk5\") pod \"maas-api-e2e-shared-a-w0-4d2317-7d6cc8fd8d-ltsxv\" (UID: \"babec534-131b-46e9-9706-cdf7fb2f60c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-4d2317-7d6cc8fd8d-ltsxv"
Oct 30 20:00:57.288549 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:57.287518    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-4d2317-7d6cc8fd8d-ltsxv"
Oct 30 20:00:57.381534 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:57.381504    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-4d2317-75c7f66df4-4784l"
Oct 30 20:00:57.601775 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:57.601737    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-4d2317-7d6cc8fd8d-ltsxv"]
Oct 30 20:00:57.601934 ip-10-0-141-42 kubenswrapper[6996]: W1030 20:00:57.601746    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbabec534_131b_46e9_9706_cdf7fb2f60c8.slice/crio-5cdf80125b39b08b9012b24d1d929c7978e495eb3e9d61168ac9cc6d80954ffa WatchSource:0}: Error finding container 5cdf80125b39b08b9012b24d1d929c7978e495eb3e9d61168ac9cc6d80954ffa: Status 404 returned error can't find the container with id 5cdf80125b39b08b9012b24d1d929c7978e495eb3e9d61168ac9cc6d80954ffa
Oct 30 20:00:58.455639 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:58.455542    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-4d2317-7d6cc8fd8d-ltsxv" event={"ID":"babec534-131b-46e9-9706-cdf7fb2f60c8","Type":"ContainerStarted","Data":"5733f8a8774c51cd8af27b5e721ebe34b9ba4d2e870c38698b54fe584c8397d1"}
Oct 30 20:00:58.455639 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:58.455609    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-4d2317-7d6cc8fd8d-ltsxv" event={"ID":"babec534-131b-46e9-9706-cdf7fb2f60c8","Type":"ContainerStarted","Data":"5cdf80125b39b08b9012b24d1d929c7978e495eb3e9d61168ac9cc6d80954ffa"}
Oct 30 20:00:58.456142 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:58.455771    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-4d2317-7d6cc8fd8d-ltsxv"
Oct 30 20:00:58.492976 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:00:58.492926    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-4d2317-7d6cc8fd8d-ltsxv" podStartSLOduration=2.00503811 podStartE2EDuration="2.492914372s" totalImagesPullingTime="487.876262ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:00:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:00:58.48932008 +0000 UTC m=+1733.817438235" watchObservedRunningTime="2026-10-30 20:00:58.492914372 +0000 UTC m=+1733.821032525"
Oct 30 20:01:01.215215 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:01:01.214370    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 20:01:04.470095 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:01:04.470067    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-4d2317-7d6cc8fd8d-ltsxv"
Oct 30 20:01:15.401060 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:01:15.401019    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 20:01:31.199218 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:01:31.199177    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 20:01:38.772032 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:01:38.771991    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-c9bce8-66c79b8d4c-tr289"]
Oct 30 20:01:38.777782 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:01:38.777758    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c9bce8-66c79b8d4c-tr289"
Oct 30 20:01:38.794032 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:01:38.794002    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-a-c9bce8-serving-cert\""
Oct 30 20:01:38.803264 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:01:38.803229    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-c9bce8-66c79b8d4c-tr289"]
Oct 30 20:01:38.899651 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:01:38.899609    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/86effdca-6a15-4388-b069-9096d8c586c5-metrics-certs\") pod \"maas-api-e2e-extmod-a-c9bce8-66c79b8d4c-tr289\" (UID: \"86effdca-6a15-4388-b069-9096d8c586c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c9bce8-66c79b8d4c-tr289"
Oct 30 20:01:38.899856 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:01:38.899687    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/86effdca-6a15-4388-b069-9096d8c586c5-maas-api-tls\") pod \"maas-api-e2e-extmod-a-c9bce8-66c79b8d4c-tr289\" (UID: \"86effdca-6a15-4388-b069-9096d8c586c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c9bce8-66c79b8d4c-tr289"
Oct 30 20:01:38.899856 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:01:38.899728    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/86effdca-6a15-4388-b069-9096d8c586c5-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-c9bce8-66c79b8d4c-tr289\" (UID: \"86effdca-6a15-4388-b069-9096d8c586c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c9bce8-66c79b8d4c-tr289"
Oct 30 20:01:38.900027 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:01:38.899979    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddnb6\" (UniqueName: \"kubernetes.io/projected/86effdca-6a15-4388-b069-9096d8c586c5-kube-api-access-ddnb6\") pod \"maas-api-e2e-extmod-a-c9bce8-66c79b8d4c-tr289\" (UID: \"86effdca-6a15-4388-b069-9096d8c586c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c9bce8-66c79b8d4c-tr289"
Oct 30 20:01:39.001360 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:01:39.001313    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/86effdca-6a15-4388-b069-9096d8c586c5-maas-api-tls\") pod \"maas-api-e2e-extmod-a-c9bce8-66c79b8d4c-tr289\" (UID: \"86effdca-6a15-4388-b069-9096d8c586c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c9bce8-66c79b8d4c-tr289"
Oct 30 20:01:39.001616 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:01:39.001371    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/86effdca-6a15-4388-b069-9096d8c586c5-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-c9bce8-66c79b8d4c-tr289\" (UID: \"86effdca-6a15-4388-b069-9096d8c586c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c9bce8-66c79b8d4c-tr289"
Oct 30 20:01:39.001616 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:01:39.001589    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddnb6\" (UniqueName: \"kubernetes.io/projected/86effdca-6a15-4388-b069-9096d8c586c5-kube-api-access-ddnb6\") pod \"maas-api-e2e-extmod-a-c9bce8-66c79b8d4c-tr289\" (UID: \"86effdca-6a15-4388-b069-9096d8c586c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c9bce8-66c79b8d4c-tr289"
Oct 30 20:01:39.001768 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:01:39.001752    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/86effdca-6a15-4388-b069-9096d8c586c5-metrics-certs\") pod \"maas-api-e2e-extmod-a-c9bce8-66c79b8d4c-tr289\" (UID: \"86effdca-6a15-4388-b069-9096d8c586c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c9bce8-66c79b8d4c-tr289"
Oct 30 20:01:39.002256 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:01:39.002202    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/86effdca-6a15-4388-b069-9096d8c586c5-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-c9bce8-66c79b8d4c-tr289\" (UID: \"86effdca-6a15-4388-b069-9096d8c586c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c9bce8-66c79b8d4c-tr289"
Oct 30 20:01:39.003931 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:01:39.003911    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/86effdca-6a15-4388-b069-9096d8c586c5-maas-api-tls\") pod \"maas-api-e2e-extmod-a-c9bce8-66c79b8d4c-tr289\" (UID: \"86effdca-6a15-4388-b069-9096d8c586c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c9bce8-66c79b8d4c-tr289"
Oct 30 20:01:39.004260 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:01:39.004237    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/86effdca-6a15-4388-b069-9096d8c586c5-metrics-certs\") pod \"maas-api-e2e-extmod-a-c9bce8-66c79b8d4c-tr289\" (UID: \"86effdca-6a15-4388-b069-9096d8c586c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c9bce8-66c79b8d4c-tr289"
Oct 30 20:01:39.014951 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:01:39.014926    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddnb6\" (UniqueName: \"kubernetes.io/projected/86effdca-6a15-4388-b069-9096d8c586c5-kube-api-access-ddnb6\") pod \"maas-api-e2e-extmod-a-c9bce8-66c79b8d4c-tr289\" (UID: \"86effdca-6a15-4388-b069-9096d8c586c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c9bce8-66c79b8d4c-tr289"
Oct 30 20:01:39.111624 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:01:39.108722    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c9bce8-66c79b8d4c-tr289"
Oct 30 20:01:39.464120 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:01:39.464092    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-c9bce8-66c79b8d4c-tr289"]
Oct 30 20:01:39.465192 ip-10-0-141-42 kubenswrapper[6996]: W1030 20:01:39.465163    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86effdca_6a15_4388_b069_9096d8c586c5.slice/crio-2ba306816f598215ba2dd142575f7b562002971efd4158890b0839813fb85e10 WatchSource:0}: Error finding container 2ba306816f598215ba2dd142575f7b562002971efd4158890b0839813fb85e10: Status 404 returned error can't find the container with id 2ba306816f598215ba2dd142575f7b562002971efd4158890b0839813fb85e10
Oct 30 20:01:39.467205 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:01:39.467179    6996 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 20:01:39.756472 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:01:39.756382    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c9bce8-66c79b8d4c-tr289" event={"ID":"86effdca-6a15-4388-b069-9096d8c586c5","Type":"ContainerStarted","Data":"2ba306816f598215ba2dd142575f7b562002971efd4158890b0839813fb85e10"}
Oct 30 20:01:40.766585 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:01:40.766537    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c9bce8-66c79b8d4c-tr289" event={"ID":"86effdca-6a15-4388-b069-9096d8c586c5","Type":"ContainerStarted","Data":"b479fd5b18f596ba8ccc8c241e6cfab4af0efa821b1f0e7562e332ac852870fc"}
Oct 30 20:01:40.767021 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:01:40.766732    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c9bce8-66c79b8d4c-tr289"
Oct 30 20:01:40.794166 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:01:40.794116    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c9bce8-66c79b8d4c-tr289" podStartSLOduration=2.189841794 podStartE2EDuration="2.794102459s" totalImagesPullingTime="604.260665ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:01:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:01:40.790017676 +0000 UTC m=+1776.118135830" watchObservedRunningTime="2026-10-30 20:01:40.794102459 +0000 UTC m=+1776.122220613"
Oct 30 20:01:46.797125 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:01:46.797095    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c9bce8-66c79b8d4c-tr289"
Oct 30 20:01:53.610686 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:01:53.610650    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-8c8241-79657d945b-5gwzv"]
Oct 30 20:01:53.615671 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:01:53.615652    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-8c8241-79657d945b-5gwzv"
Oct 30 20:01:53.631448 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:01:53.631422    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-8c8241-79657d945b-5gwzv"]
Oct 30 20:01:53.671063 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:01:53.671005    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-a-8c8241\""
Oct 30 20:01:53.671593 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:01:53.671569    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-a-8c8241-dockercfg-b9d6j\""
Oct 30 20:01:53.776514 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:01:53.772971    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56aa21ff-1941-413f-99da-31997009371a-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-8c8241-79657d945b-5gwzv\" (UID: \"56aa21ff-1941-413f-99da-31997009371a\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-8c8241-79657d945b-5gwzv"
Oct 30 20:01:53.776514 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:01:53.773512    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b586r\" (UniqueName: \"kubernetes.io/projected/56aa21ff-1941-413f-99da-31997009371a-kube-api-access-b586r\") pod \"payload-processing-e2e-extmod-a-8c8241-79657d945b-5gwzv\" (UID: \"56aa21ff-1941-413f-99da-31997009371a\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-8c8241-79657d945b-5gwzv"
Oct 30 20:01:53.874805 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:01:53.874709    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b586r\" (UniqueName: \"kubernetes.io/projected/56aa21ff-1941-413f-99da-31997009371a-kube-api-access-b586r\") pod \"payload-processing-e2e-extmod-a-8c8241-79657d945b-5gwzv\" (UID: \"56aa21ff-1941-413f-99da-31997009371a\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-8c8241-79657d945b-5gwzv"
Oct 30 20:01:53.875284 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:01:53.875251    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56aa21ff-1941-413f-99da-31997009371a-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-8c8241-79657d945b-5gwzv\" (UID: \"56aa21ff-1941-413f-99da-31997009371a\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-8c8241-79657d945b-5gwzv"
Oct 30 20:01:53.876137 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:01:53.876113    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56aa21ff-1941-413f-99da-31997009371a-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-8c8241-79657d945b-5gwzv\" (UID: \"56aa21ff-1941-413f-99da-31997009371a\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-8c8241-79657d945b-5gwzv"
Oct 30 20:01:53.893435 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:01:53.893398    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b586r\" (UniqueName: \"kubernetes.io/projected/56aa21ff-1941-413f-99da-31997009371a-kube-api-access-b586r\") pod \"payload-processing-e2e-extmod-a-8c8241-79657d945b-5gwzv\" (UID: \"56aa21ff-1941-413f-99da-31997009371a\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-8c8241-79657d945b-5gwzv"
Oct 30 20:01:53.938225 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:01:53.938197    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-8c8241-79657d945b-5gwzv"
Oct 30 20:01:54.238017 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:01:54.237984    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-8c8241-79657d945b-5gwzv"]
Oct 30 20:01:54.240617 ip-10-0-141-42 kubenswrapper[6996]: W1030 20:01:54.240583    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56aa21ff_1941_413f_99da_31997009371a.slice/crio-4526ef898d1ac2bb51d6b7dc9d2a6bced43e1b1995d7f5fd9c08007fabd76cda WatchSource:0}: Error finding container 4526ef898d1ac2bb51d6b7dc9d2a6bced43e1b1995d7f5fd9c08007fabd76cda: Status 404 returned error can't find the container with id 4526ef898d1ac2bb51d6b7dc9d2a6bced43e1b1995d7f5fd9c08007fabd76cda
Oct 30 20:01:54.898178 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:01:54.898140    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-8c8241-79657d945b-5gwzv" event={"ID":"56aa21ff-1941-413f-99da-31997009371a","Type":"ContainerStarted","Data":"94224f04750c1764f71a2c26afa4305d1a3767cce74e01166d9de07f1305274c"}
Oct 30 20:01:54.898681 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:01:54.898207    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-8c8241-79657d945b-5gwzv"
Oct 30 20:01:54.898681 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:01:54.898228    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-8c8241-79657d945b-5gwzv" event={"ID":"56aa21ff-1941-413f-99da-31997009371a","Type":"ContainerStarted","Data":"4526ef898d1ac2bb51d6b7dc9d2a6bced43e1b1995d7f5fd9c08007fabd76cda"}
Oct 30 20:01:54.928831 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:01:54.928754    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-extmod-a-8c8241-79657d945b-5gwzv" podStartSLOduration=1.92873765 podStartE2EDuration="1.92873765s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:01:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:01:54.927964585 +0000 UTC m=+1790.256082739" watchObservedRunningTime="2026-10-30 20:01:54.92873765 +0000 UTC m=+1790.256855805"
Oct 30 20:02:01.198067 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:02:01.198022    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 20:02:05.909163 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:05.909134    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-8c8241-79657d945b-5gwzv"
Oct 30 20:02:07.373029 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:07.372989    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 20:02:09.291188 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:09.291150    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j"]
Oct 30 20:02:09.295992 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:09.295972    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j"
Oct 30 20:02:09.306603 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:09.306581    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w4-7cbcbb-openshift-default-dockercfg-ggq85\""
Oct 30 20:02:09.314431 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:09.314405    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j"]
Oct 30 20:02:09.415171 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:09.415122    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/974a4f8f-5f4a-4338-80b3-847f6e5ca080-istio-podinfo\") pod \"e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j\" (UID: \"974a4f8f-5f4a-4338-80b3-847f6e5ca080\") " pod="openshift-ingress/e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j"
Oct 30 20:02:09.415171 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:09.415174    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/974a4f8f-5f4a-4338-80b3-847f6e5ca080-istiod-ca-cert\") pod \"e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j\" (UID: \"974a4f8f-5f4a-4338-80b3-847f6e5ca080\") " pod="openshift-ingress/e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j"
Oct 30 20:02:09.415409 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:09.415207    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/974a4f8f-5f4a-4338-80b3-847f6e5ca080-istio-data\") pod \"e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j\" (UID: \"974a4f8f-5f4a-4338-80b3-847f6e5ca080\") " pod="openshift-ingress/e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j"
Oct 30 20:02:09.415409 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:09.415348    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/974a4f8f-5f4a-4338-80b3-847f6e5ca080-workload-socket\") pod \"e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j\" (UID: \"974a4f8f-5f4a-4338-80b3-847f6e5ca080\") " pod="openshift-ingress/e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j"
Oct 30 20:02:09.415526 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:09.415474    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/974a4f8f-5f4a-4338-80b3-847f6e5ca080-istio-envoy\") pod \"e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j\" (UID: \"974a4f8f-5f4a-4338-80b3-847f6e5ca080\") " pod="openshift-ingress/e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j"
Oct 30 20:02:09.415579 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:09.415562    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/974a4f8f-5f4a-4338-80b3-847f6e5ca080-credential-socket\") pod \"e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j\" (UID: \"974a4f8f-5f4a-4338-80b3-847f6e5ca080\") " pod="openshift-ingress/e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j"
Oct 30 20:02:09.415619 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:09.415592    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/974a4f8f-5f4a-4338-80b3-847f6e5ca080-workload-certs\") pod \"e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j\" (UID: \"974a4f8f-5f4a-4338-80b3-847f6e5ca080\") " pod="openshift-ingress/e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j"
Oct 30 20:02:09.415653 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:09.415639    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xv4wt\" (UniqueName: \"kubernetes.io/projected/974a4f8f-5f4a-4338-80b3-847f6e5ca080-kube-api-access-xv4wt\") pod \"e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j\" (UID: \"974a4f8f-5f4a-4338-80b3-847f6e5ca080\") " pod="openshift-ingress/e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j"
Oct 30 20:02:09.415710 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:09.415698    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/974a4f8f-5f4a-4338-80b3-847f6e5ca080-istio-token\") pod \"e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j\" (UID: \"974a4f8f-5f4a-4338-80b3-847f6e5ca080\") " pod="openshift-ingress/e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j"
Oct 30 20:02:09.516814 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:09.516785    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/974a4f8f-5f4a-4338-80b3-847f6e5ca080-istio-podinfo\") pod \"e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j\" (UID: \"974a4f8f-5f4a-4338-80b3-847f6e5ca080\") " pod="openshift-ingress/e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j"
Oct 30 20:02:09.516991 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:09.516819    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/974a4f8f-5f4a-4338-80b3-847f6e5ca080-istiod-ca-cert\") pod \"e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j\" (UID: \"974a4f8f-5f4a-4338-80b3-847f6e5ca080\") " pod="openshift-ingress/e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j"
Oct 30 20:02:09.516991 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:09.516856    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/974a4f8f-5f4a-4338-80b3-847f6e5ca080-istio-data\") pod \"e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j\" (UID: \"974a4f8f-5f4a-4338-80b3-847f6e5ca080\") " pod="openshift-ingress/e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j"
Oct 30 20:02:09.516991 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:09.516915    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/974a4f8f-5f4a-4338-80b3-847f6e5ca080-workload-socket\") pod \"e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j\" (UID: \"974a4f8f-5f4a-4338-80b3-847f6e5ca080\") " pod="openshift-ingress/e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j"
Oct 30 20:02:09.517129 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:09.517049    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/974a4f8f-5f4a-4338-80b3-847f6e5ca080-istio-envoy\") pod \"e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j\" (UID: \"974a4f8f-5f4a-4338-80b3-847f6e5ca080\") " pod="openshift-ingress/e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j"
Oct 30 20:02:09.517129 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:09.517083    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/974a4f8f-5f4a-4338-80b3-847f6e5ca080-credential-socket\") pod \"e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j\" (UID: \"974a4f8f-5f4a-4338-80b3-847f6e5ca080\") " pod="openshift-ingress/e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j"
Oct 30 20:02:09.517129 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:09.517113    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/974a4f8f-5f4a-4338-80b3-847f6e5ca080-workload-certs\") pod \"e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j\" (UID: \"974a4f8f-5f4a-4338-80b3-847f6e5ca080\") " pod="openshift-ingress/e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j"
Oct 30 20:02:09.517287 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:09.517144    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xv4wt\" (UniqueName: \"kubernetes.io/projected/974a4f8f-5f4a-4338-80b3-847f6e5ca080-kube-api-access-xv4wt\") pod \"e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j\" (UID: \"974a4f8f-5f4a-4338-80b3-847f6e5ca080\") " pod="openshift-ingress/e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j"
Oct 30 20:02:09.517287 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:09.517231    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/974a4f8f-5f4a-4338-80b3-847f6e5ca080-istio-token\") pod \"e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j\" (UID: \"974a4f8f-5f4a-4338-80b3-847f6e5ca080\") " pod="openshift-ingress/e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j"
Oct 30 20:02:09.517393 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:09.517365    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/974a4f8f-5f4a-4338-80b3-847f6e5ca080-workload-socket\") pod \"e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j\" (UID: \"974a4f8f-5f4a-4338-80b3-847f6e5ca080\") " pod="openshift-ingress/e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j"
Oct 30 20:02:09.517594 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:09.517567    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/974a4f8f-5f4a-4338-80b3-847f6e5ca080-credential-socket\") pod \"e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j\" (UID: \"974a4f8f-5f4a-4338-80b3-847f6e5ca080\") " pod="openshift-ingress/e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j"
Oct 30 20:02:09.517696 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:09.517607    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/974a4f8f-5f4a-4338-80b3-847f6e5ca080-workload-certs\") pod \"e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j\" (UID: \"974a4f8f-5f4a-4338-80b3-847f6e5ca080\") " pod="openshift-ingress/e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j"
Oct 30 20:02:09.517696 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:09.517619    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/974a4f8f-5f4a-4338-80b3-847f6e5ca080-istiod-ca-cert\") pod \"e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j\" (UID: \"974a4f8f-5f4a-4338-80b3-847f6e5ca080\") " pod="openshift-ingress/e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j"
Oct 30 20:02:09.517821 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:09.517801    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/974a4f8f-5f4a-4338-80b3-847f6e5ca080-istio-data\") pod \"e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j\" (UID: \"974a4f8f-5f4a-4338-80b3-847f6e5ca080\") " pod="openshift-ingress/e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j"
Oct 30 20:02:09.519445 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:09.519420    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/974a4f8f-5f4a-4338-80b3-847f6e5ca080-istio-podinfo\") pod \"e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j\" (UID: \"974a4f8f-5f4a-4338-80b3-847f6e5ca080\") " pod="openshift-ingress/e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j"
Oct 30 20:02:09.519557 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:09.519442    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/974a4f8f-5f4a-4338-80b3-847f6e5ca080-istio-envoy\") pod \"e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j\" (UID: \"974a4f8f-5f4a-4338-80b3-847f6e5ca080\") " pod="openshift-ingress/e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j"
Oct 30 20:02:09.532517 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:09.532453    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xv4wt\" (UniqueName: \"kubernetes.io/projected/974a4f8f-5f4a-4338-80b3-847f6e5ca080-kube-api-access-xv4wt\") pod \"e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j\" (UID: \"974a4f8f-5f4a-4338-80b3-847f6e5ca080\") " pod="openshift-ingress/e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j"
Oct 30 20:02:09.532517 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:09.532494    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/974a4f8f-5f4a-4338-80b3-847f6e5ca080-istio-token\") pod \"e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j\" (UID: \"974a4f8f-5f4a-4338-80b3-847f6e5ca080\") " pod="openshift-ingress/e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j"
Oct 30 20:02:09.610426 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:09.610394    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j"
Oct 30 20:02:09.918553 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:09.918525    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j"]
Oct 30 20:02:09.919365 ip-10-0-141-42 kubenswrapper[6996]: W1030 20:02:09.919334    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod974a4f8f_5f4a_4338_80b3_847f6e5ca080.slice/crio-2ee44a32cb31b4246ae7f348a515246d1787ab91d1ba7efd289b9bbdba72f052 WatchSource:0}: Error finding container 2ee44a32cb31b4246ae7f348a515246d1787ab91d1ba7efd289b9bbdba72f052: Status 404 returned error can't find the container with id 2ee44a32cb31b4246ae7f348a515246d1787ab91d1ba7efd289b9bbdba72f052
Oct 30 20:02:09.922335 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:09.922302    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 20:02:09.922447 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:09.922385    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 20:02:09.922447 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:09.922432    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 20:02:10.051675 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:10.051633    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j" event={"ID":"974a4f8f-5f4a-4338-80b3-847f6e5ca080","Type":"ContainerStarted","Data":"a90814e71ccbaffcfb0f994387b99d9674dc19f86bf129593e0ffdd822862aac"}
Oct 30 20:02:10.051872 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:10.051699    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j" event={"ID":"974a4f8f-5f4a-4338-80b3-847f6e5ca080","Type":"ContainerStarted","Data":"2ee44a32cb31b4246ae7f348a515246d1787ab91d1ba7efd289b9bbdba72f052"}
Oct 30 20:02:10.079454 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:10.079398    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j" podStartSLOduration=1.079382498 podStartE2EDuration="1.079382498s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:02:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:02:10.076654469 +0000 UTC m=+1805.404772622" watchObservedRunningTime="2026-10-30 20:02:10.079382498 +0000 UTC m=+1805.407500651"
Oct 30 20:02:10.610995 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:10.610959    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j"
Oct 30 20:02:10.612285 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:10.612256    6996 patch_prober.go:29] interesting pod/e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused" start-of-body=
Oct 30 20:02:10.612423 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:10.612303    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j" podUID="974a4f8f-5f4a-4338-80b3-847f6e5ca080" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused"
Oct 30 20:02:11.611740 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:11.611704    6996 patch_prober.go:29] interesting pod/e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused" start-of-body=
Oct 30 20:02:11.612143 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:11.611778    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j" podUID="974a4f8f-5f4a-4338-80b3-847f6e5ca080" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused"
Oct 30 20:02:12.613949 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:12.613908    6996 patch_prober.go:29] interesting pod/e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused" start-of-body=
Oct 30 20:02:12.614639 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:12.614597    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j" podUID="974a4f8f-5f4a-4338-80b3-847f6e5ca080" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused"
Oct 30 20:02:13.611331 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:13.611289    6996 patch_prober.go:29] interesting pod/e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused" start-of-body=
Oct 30 20:02:13.611619 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:13.611348    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j" podUID="974a4f8f-5f4a-4338-80b3-847f6e5ca080" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused"
Oct 30 20:02:14.611290 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:14.611245    6996 patch_prober.go:29] interesting pod/e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused" start-of-body=
Oct 30 20:02:14.611812 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:14.611321    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j" podUID="974a4f8f-5f4a-4338-80b3-847f6e5ca080" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused"
Oct 30 20:02:15.611700 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:15.611659    6996 patch_prober.go:29] interesting pod/e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused" start-of-body=
Oct 30 20:02:15.612106 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:15.611713    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j" podUID="974a4f8f-5f4a-4338-80b3-847f6e5ca080" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused"
Oct 30 20:02:16.686667 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:16.686623    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j"
Oct 30 20:02:16.687221 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:16.686929    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j"
Oct 30 20:02:16.687903 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:16.687881    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j"
Oct 30 20:02:20.538850 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:20.538813    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8"]
Oct 30 20:02:20.553177 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:20.553147    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8"
Oct 30 20:02:20.576896 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:20.576866    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8"]
Oct 30 20:02:20.608260 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:20.608223    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w0-75b361-openshift-default-dockercfg-mcqdv\""
Oct 30 20:02:20.682556 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:20.682522    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a98a3fe9-bb1b-4a7f-af39-8486b0053f39-workload-socket\") pod \"e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8\" (UID: \"a98a3fe9-bb1b-4a7f-af39-8486b0053f39\") " pod="openshift-ingress/e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8"
Oct 30 20:02:20.682719 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:20.682640    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a98a3fe9-bb1b-4a7f-af39-8486b0053f39-workload-certs\") pod \"e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8\" (UID: \"a98a3fe9-bb1b-4a7f-af39-8486b0053f39\") " pod="openshift-ingress/e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8"
Oct 30 20:02:20.682761 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:20.682719    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a98a3fe9-bb1b-4a7f-af39-8486b0053f39-istio-token\") pod \"e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8\" (UID: \"a98a3fe9-bb1b-4a7f-af39-8486b0053f39\") " pod="openshift-ingress/e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8"
Oct 30 20:02:20.682797 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:20.682779    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a98a3fe9-bb1b-4a7f-af39-8486b0053f39-credential-socket\") pod \"e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8\" (UID: \"a98a3fe9-bb1b-4a7f-af39-8486b0053f39\") " pod="openshift-ingress/e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8"
Oct 30 20:02:20.682833 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:20.682818    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a98a3fe9-bb1b-4a7f-af39-8486b0053f39-istio-envoy\") pod \"e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8\" (UID: \"a98a3fe9-bb1b-4a7f-af39-8486b0053f39\") " pod="openshift-ingress/e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8"
Oct 30 20:02:20.682874 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:20.682863    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a98a3fe9-bb1b-4a7f-af39-8486b0053f39-istiod-ca-cert\") pod \"e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8\" (UID: \"a98a3fe9-bb1b-4a7f-af39-8486b0053f39\") " pod="openshift-ingress/e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8"
Oct 30 20:02:20.682972 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:20.682958    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a98a3fe9-bb1b-4a7f-af39-8486b0053f39-istio-podinfo\") pod \"e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8\" (UID: \"a98a3fe9-bb1b-4a7f-af39-8486b0053f39\") " pod="openshift-ingress/e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8"
Oct 30 20:02:20.683071 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:20.683050    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bn84m\" (UniqueName: \"kubernetes.io/projected/a98a3fe9-bb1b-4a7f-af39-8486b0053f39-kube-api-access-bn84m\") pod \"e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8\" (UID: \"a98a3fe9-bb1b-4a7f-af39-8486b0053f39\") " pod="openshift-ingress/e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8"
Oct 30 20:02:20.683158 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:20.683145    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a98a3fe9-bb1b-4a7f-af39-8486b0053f39-istio-data\") pod \"e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8\" (UID: \"a98a3fe9-bb1b-4a7f-af39-8486b0053f39\") " pod="openshift-ingress/e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8"
Oct 30 20:02:20.784457 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:20.784426    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a98a3fe9-bb1b-4a7f-af39-8486b0053f39-istio-envoy\") pod \"e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8\" (UID: \"a98a3fe9-bb1b-4a7f-af39-8486b0053f39\") " pod="openshift-ingress/e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8"
Oct 30 20:02:20.784685 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:20.784492    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a98a3fe9-bb1b-4a7f-af39-8486b0053f39-istiod-ca-cert\") pod \"e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8\" (UID: \"a98a3fe9-bb1b-4a7f-af39-8486b0053f39\") " pod="openshift-ingress/e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8"
Oct 30 20:02:20.784685 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:20.784537    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a98a3fe9-bb1b-4a7f-af39-8486b0053f39-istio-podinfo\") pod \"e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8\" (UID: \"a98a3fe9-bb1b-4a7f-af39-8486b0053f39\") " pod="openshift-ingress/e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8"
Oct 30 20:02:20.784819 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:20.784687    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bn84m\" (UniqueName: \"kubernetes.io/projected/a98a3fe9-bb1b-4a7f-af39-8486b0053f39-kube-api-access-bn84m\") pod \"e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8\" (UID: \"a98a3fe9-bb1b-4a7f-af39-8486b0053f39\") " pod="openshift-ingress/e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8"
Oct 30 20:02:20.784819 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:20.784791    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a98a3fe9-bb1b-4a7f-af39-8486b0053f39-istio-data\") pod \"e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8\" (UID: \"a98a3fe9-bb1b-4a7f-af39-8486b0053f39\") " pod="openshift-ingress/e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8"
Oct 30 20:02:20.784933 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:20.784831    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a98a3fe9-bb1b-4a7f-af39-8486b0053f39-workload-socket\") pod \"e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8\" (UID: \"a98a3fe9-bb1b-4a7f-af39-8486b0053f39\") " pod="openshift-ingress/e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8"
Oct 30 20:02:20.784996 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:20.784930    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a98a3fe9-bb1b-4a7f-af39-8486b0053f39-workload-certs\") pod \"e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8\" (UID: \"a98a3fe9-bb1b-4a7f-af39-8486b0053f39\") " pod="openshift-ingress/e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8"
Oct 30 20:02:20.785208 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:20.785181    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a98a3fe9-bb1b-4a7f-af39-8486b0053f39-istio-data\") pod \"e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8\" (UID: \"a98a3fe9-bb1b-4a7f-af39-8486b0053f39\") " pod="openshift-ingress/e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8"
Oct 30 20:02:20.785402 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:20.785277    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a98a3fe9-bb1b-4a7f-af39-8486b0053f39-workload-socket\") pod \"e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8\" (UID: \"a98a3fe9-bb1b-4a7f-af39-8486b0053f39\") " pod="openshift-ingress/e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8"
Oct 30 20:02:20.785525 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:20.785290    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a98a3fe9-bb1b-4a7f-af39-8486b0053f39-istiod-ca-cert\") pod \"e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8\" (UID: \"a98a3fe9-bb1b-4a7f-af39-8486b0053f39\") " pod="openshift-ingress/e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8"
Oct 30 20:02:20.785525 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:20.785301    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a98a3fe9-bb1b-4a7f-af39-8486b0053f39-istio-token\") pod \"e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8\" (UID: \"a98a3fe9-bb1b-4a7f-af39-8486b0053f39\") " pod="openshift-ingress/e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8"
Oct 30 20:02:20.785525 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:20.785515    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a98a3fe9-bb1b-4a7f-af39-8486b0053f39-credential-socket\") pod \"e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8\" (UID: \"a98a3fe9-bb1b-4a7f-af39-8486b0053f39\") " pod="openshift-ingress/e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8"
Oct 30 20:02:20.785661 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:20.785323    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a98a3fe9-bb1b-4a7f-af39-8486b0053f39-workload-certs\") pod \"e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8\" (UID: \"a98a3fe9-bb1b-4a7f-af39-8486b0053f39\") " pod="openshift-ingress/e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8"
Oct 30 20:02:20.785918 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:20.785896    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a98a3fe9-bb1b-4a7f-af39-8486b0053f39-credential-socket\") pod \"e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8\" (UID: \"a98a3fe9-bb1b-4a7f-af39-8486b0053f39\") " pod="openshift-ingress/e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8"
Oct 30 20:02:20.786984 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:20.786960    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a98a3fe9-bb1b-4a7f-af39-8486b0053f39-istio-envoy\") pod \"e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8\" (UID: \"a98a3fe9-bb1b-4a7f-af39-8486b0053f39\") " pod="openshift-ingress/e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8"
Oct 30 20:02:20.787310 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:20.787290    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a98a3fe9-bb1b-4a7f-af39-8486b0053f39-istio-podinfo\") pod \"e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8\" (UID: \"a98a3fe9-bb1b-4a7f-af39-8486b0053f39\") " pod="openshift-ingress/e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8"
Oct 30 20:02:20.809890 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:20.809827    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bn84m\" (UniqueName: \"kubernetes.io/projected/a98a3fe9-bb1b-4a7f-af39-8486b0053f39-kube-api-access-bn84m\") pod \"e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8\" (UID: \"a98a3fe9-bb1b-4a7f-af39-8486b0053f39\") " pod="openshift-ingress/e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8"
Oct 30 20:02:20.811593 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:20.811571    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a98a3fe9-bb1b-4a7f-af39-8486b0053f39-istio-token\") pod \"e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8\" (UID: \"a98a3fe9-bb1b-4a7f-af39-8486b0053f39\") " pod="openshift-ingress/e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8"
Oct 30 20:02:20.869814 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:20.869786    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8"
Oct 30 20:02:21.212856 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:21.212831    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8"]
Oct 30 20:02:21.214517 ip-10-0-141-42 kubenswrapper[6996]: W1030 20:02:21.214449    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda98a3fe9_bb1b_4a7f_af39_8486b0053f39.slice/crio-ecd6d3b32e73e84fd34ee2649dbd186f14006292532554ee3e56ebe9bbee1084 WatchSource:0}: Error finding container ecd6d3b32e73e84fd34ee2649dbd186f14006292532554ee3e56ebe9bbee1084: Status 404 returned error can't find the container with id ecd6d3b32e73e84fd34ee2649dbd186f14006292532554ee3e56ebe9bbee1084
Oct 30 20:02:21.217144 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:21.217111    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 20:02:21.217236 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:21.217176    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 20:02:21.217236 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:21.217207    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 20:02:21.229084 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:21.229053    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8" event={"ID":"a98a3fe9-bb1b-4a7f-af39-8486b0053f39","Type":"ContainerStarted","Data":"ecd6d3b32e73e84fd34ee2649dbd186f14006292532554ee3e56ebe9bbee1084"}
Oct 30 20:02:22.267461 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:22.263060    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8" event={"ID":"a98a3fe9-bb1b-4a7f-af39-8486b0053f39","Type":"ContainerStarted","Data":"0a3b23b37f92712ac1f5d1951e138d3eaeee7b2af6d12e8be9f96126b1e7b811"}
Oct 30 20:02:22.290878 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:22.290823    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8" podStartSLOduration=2.2908078 podStartE2EDuration="2.2908078s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:02:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:02:22.287115498 +0000 UTC m=+1817.615233652" watchObservedRunningTime="2026-10-30 20:02:22.2908078 +0000 UTC m=+1817.618925954"
Oct 30 20:02:22.870820 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:22.870786    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8"
Oct 30 20:02:22.875665 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:22.875627    6996 patch_prober.go:29] interesting pod/e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.155:15021/healthz/ready\": dial tcp 10.134.0.155:15021: connect: connection refused" start-of-body=
Oct 30 20:02:22.875931 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:22.875898    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8" podUID="a98a3fe9-bb1b-4a7f-af39-8486b0053f39" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.155:15021/healthz/ready\": dial tcp 10.134.0.155:15021: connect: connection refused"
Oct 30 20:02:23.871270 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:23.871234    6996 patch_prober.go:29] interesting pod/e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.155:15021/healthz/ready\": dial tcp 10.134.0.155:15021: connect: connection refused" start-of-body=
Oct 30 20:02:23.871270 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:23.871307    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8" podUID="a98a3fe9-bb1b-4a7f-af39-8486b0053f39" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.155:15021/healthz/ready\": dial tcp 10.134.0.155:15021: connect: connection refused"
Oct 30 20:02:24.770427 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:24.769531    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg"]
Oct 30 20:02:24.770427 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:24.769795    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg" podUID="7d78490f-9250-4c76-9786-4d5499c34beb" containerName="istio-proxy" containerID="cri-o://a5e61c5578da65f9c92d5742d91400330c00050beda09e673ecd29f0bdd2eb4c" gracePeriod=30
Oct 30 20:02:24.870087 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:24.870053    6996 patch_prober.go:29] interesting pod/e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.155:15021/healthz/ready\": dial tcp 10.134.0.155:15021: connect: connection refused" start-of-body=
Oct 30 20:02:24.870308 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:24.870287    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8" podUID="a98a3fe9-bb1b-4a7f-af39-8486b0053f39" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.155:15021/healthz/ready\": dial tcp 10.134.0.155:15021: connect: connection refused"
Oct 30 20:02:25.870399 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:25.870364    6996 patch_prober.go:29] interesting pod/e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.155:15021/healthz/ready\": dial tcp 10.134.0.155:15021: connect: connection refused" start-of-body=
Oct 30 20:02:25.870948 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:25.870447    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8" podUID="a98a3fe9-bb1b-4a7f-af39-8486b0053f39" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.155:15021/healthz/ready\": dial tcp 10.134.0.155:15021: connect: connection refused"
Oct 30 20:02:26.884427 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:26.883165    6996 patch_prober.go:29] interesting pod/e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.155:15021/healthz/ready\": dial tcp 10.134.0.155:15021: connect: connection refused" start-of-body=
Oct 30 20:02:26.884427 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:26.883233    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8" podUID="a98a3fe9-bb1b-4a7f-af39-8486b0053f39" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.155:15021/healthz/ready\": dial tcp 10.134.0.155:15021: connect: connection refused"
Oct 30 20:02:27.897226 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:27.897193    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8"
Oct 30 20:02:27.898244 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:27.898216    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8"
Oct 30 20:02:27.903740 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:27.903713    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8"
Oct 30 20:02:30.065582 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:30.065558    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg"
Oct 30 20:02:30.128647 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:30.128074    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7d78490f-9250-4c76-9786-4d5499c34beb-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d78490f-9250-4c76-9786-4d5499c34beb-workload-socket\") pod \"7d78490f-9250-4c76-9786-4d5499c34beb\" (UID: \"7d78490f-9250-4c76-9786-4d5499c34beb\") "
Oct 30 20:02:30.128647 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:30.128136    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7d78490f-9250-4c76-9786-4d5499c34beb-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d78490f-9250-4c76-9786-4d5499c34beb-credential-socket\") pod \"7d78490f-9250-4c76-9786-4d5499c34beb\" (UID: \"7d78490f-9250-4c76-9786-4d5499c34beb\") "
Oct 30 20:02:30.128647 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:30.128176    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7d78490f-9250-4c76-9786-4d5499c34beb-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d78490f-9250-4c76-9786-4d5499c34beb-istio-podinfo\") pod \"7d78490f-9250-4c76-9786-4d5499c34beb\" (UID: \"7d78490f-9250-4c76-9786-4d5499c34beb\") "
Oct 30 20:02:30.128647 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:30.128277    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7d78490f-9250-4c76-9786-4d5499c34beb-kube-api-access-9s7h5\" (UniqueName: \"kubernetes.io/projected/7d78490f-9250-4c76-9786-4d5499c34beb-kube-api-access-9s7h5\") pod \"7d78490f-9250-4c76-9786-4d5499c34beb\" (UID: \"7d78490f-9250-4c76-9786-4d5499c34beb\") "
Oct 30 20:02:30.128647 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:30.128300    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7d78490f-9250-4c76-9786-4d5499c34beb-istio-token\" (UniqueName: \"kubernetes.io/projected/7d78490f-9250-4c76-9786-4d5499c34beb-istio-token\") pod \"7d78490f-9250-4c76-9786-4d5499c34beb\" (UID: \"7d78490f-9250-4c76-9786-4d5499c34beb\") "
Oct 30 20:02:30.128647 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:30.128330    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7d78490f-9250-4c76-9786-4d5499c34beb-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d78490f-9250-4c76-9786-4d5499c34beb-istio-envoy\") pod \"7d78490f-9250-4c76-9786-4d5499c34beb\" (UID: \"7d78490f-9250-4c76-9786-4d5499c34beb\") "
Oct 30 20:02:30.128647 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:30.128391    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d78490f-9250-4c76-9786-4d5499c34beb-workload-socket" pod "7d78490f-9250-4c76-9786-4d5499c34beb" (UID: "7d78490f-9250-4c76-9786-4d5499c34beb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:02:30.128647 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:30.128413    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7d78490f-9250-4c76-9786-4d5499c34beb-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d78490f-9250-4c76-9786-4d5499c34beb-istio-data\") pod \"7d78490f-9250-4c76-9786-4d5499c34beb\" (UID: \"7d78490f-9250-4c76-9786-4d5499c34beb\") "
Oct 30 20:02:30.129184 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:30.128817    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7d78490f-9250-4c76-9786-4d5499c34beb-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d78490f-9250-4c76-9786-4d5499c34beb-istiod-ca-cert\") pod \"7d78490f-9250-4c76-9786-4d5499c34beb\" (UID: \"7d78490f-9250-4c76-9786-4d5499c34beb\") "
Oct 30 20:02:30.129184 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:30.128884    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7d78490f-9250-4c76-9786-4d5499c34beb-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d78490f-9250-4c76-9786-4d5499c34beb-workload-certs\") pod \"7d78490f-9250-4c76-9786-4d5499c34beb\" (UID: \"7d78490f-9250-4c76-9786-4d5499c34beb\") "
Oct 30 20:02:30.129333 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:30.129308    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d78490f-9250-4c76-9786-4d5499c34beb-istio-data" pod "7d78490f-9250-4c76-9786-4d5499c34beb" (UID: "7d78490f-9250-4c76-9786-4d5499c34beb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:02:30.129643 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:30.129610    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d78490f-9250-4c76-9786-4d5499c34beb-workload-certs" pod "7d78490f-9250-4c76-9786-4d5499c34beb" (UID: "7d78490f-9250-4c76-9786-4d5499c34beb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:02:30.129792 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:30.129696    6996 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d78490f-9250-4c76-9786-4d5499c34beb-istio-data\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:02:30.129792 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:30.129792    6996 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d78490f-9250-4c76-9786-4d5499c34beb-workload-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:02:30.129964 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:30.129873    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d78490f-9250-4c76-9786-4d5499c34beb-credential-socket" pod "7d78490f-9250-4c76-9786-4d5499c34beb" (UID: "7d78490f-9250-4c76-9786-4d5499c34beb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:02:30.130185 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:30.130163    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d78490f-9250-4c76-9786-4d5499c34beb-istiod-ca-cert" pod "7d78490f-9250-4c76-9786-4d5499c34beb" (UID: "7d78490f-9250-4c76-9786-4d5499c34beb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:02:30.137793 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:30.137632    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d78490f-9250-4c76-9786-4d5499c34beb-istio-envoy" pod "7d78490f-9250-4c76-9786-4d5499c34beb" (UID: "7d78490f-9250-4c76-9786-4d5499c34beb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:02:30.139583 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:30.138235    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d78490f-9250-4c76-9786-4d5499c34beb-istio-token" pod "7d78490f-9250-4c76-9786-4d5499c34beb" (UID: "7d78490f-9250-4c76-9786-4d5499c34beb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:02:30.168820 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:30.168790    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d78490f-9250-4c76-9786-4d5499c34beb-kube-api-access-9s7h5" pod "7d78490f-9250-4c76-9786-4d5499c34beb" (UID: "7d78490f-9250-4c76-9786-4d5499c34beb"). InnerVolumeSpecName "kube-api-access-9s7h5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:02:30.172669 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:30.172636    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7d78490f-9250-4c76-9786-4d5499c34beb-istio-podinfo" pod "7d78490f-9250-4c76-9786-4d5499c34beb" (UID: "7d78490f-9250-4c76-9786-4d5499c34beb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 20:02:30.231272 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:30.231243    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9s7h5\" (UniqueName: \"kubernetes.io/projected/7d78490f-9250-4c76-9786-4d5499c34beb-kube-api-access-9s7h5\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:02:30.231272 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:30.231269    6996 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d78490f-9250-4c76-9786-4d5499c34beb-istio-token\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:02:30.231463 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:30.231281    6996 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d78490f-9250-4c76-9786-4d5499c34beb-istio-envoy\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:02:30.231463 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:30.231290    6996 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d78490f-9250-4c76-9786-4d5499c34beb-istiod-ca-cert\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:02:30.231463 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:30.231300    6996 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d78490f-9250-4c76-9786-4d5499c34beb-workload-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:02:30.231463 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:30.231310    6996 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d78490f-9250-4c76-9786-4d5499c34beb-credential-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:02:30.231463 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:30.231318    6996 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d78490f-9250-4c76-9786-4d5499c34beb-istio-podinfo\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:02:30.340258 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:30.340177    6996 generic.go:356] "Generic (PLEG): container finished" podID="7d78490f-9250-4c76-9786-4d5499c34beb" containerID="a5e61c5578da65f9c92d5742d91400330c00050beda09e673ecd29f0bdd2eb4c" exitCode=0
Oct 30 20:02:30.340258 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:30.340229    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg"
Oct 30 20:02:30.340454 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:30.340232    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg" event={"ID":"7d78490f-9250-4c76-9786-4d5499c34beb","Type":"ContainerDied","Data":"a5e61c5578da65f9c92d5742d91400330c00050beda09e673ecd29f0bdd2eb4c"}
Oct 30 20:02:30.340454 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:30.340354    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg" event={"ID":"7d78490f-9250-4c76-9786-4d5499c34beb","Type":"ContainerDied","Data":"6c9603f51e8a633d0ef173c9155c1c8359fded0654c2c91bda1cb827698a0b98"}
Oct 30 20:02:30.340454 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:30.340383    6996 scope.go:122] "RemoveContainer" containerID="a5e61c5578da65f9c92d5742d91400330c00050beda09e673ecd29f0bdd2eb4c"
Oct 30 20:02:30.351915 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:30.351893    6996 scope.go:122] "RemoveContainer" containerID="a5e61c5578da65f9c92d5742d91400330c00050beda09e673ecd29f0bdd2eb4c"
Oct 30 20:02:30.352204 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:02:30.352183    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5e61c5578da65f9c92d5742d91400330c00050beda09e673ecd29f0bdd2eb4c\": container with ID starting with a5e61c5578da65f9c92d5742d91400330c00050beda09e673ecd29f0bdd2eb4c not found: ID does not exist" containerID="a5e61c5578da65f9c92d5742d91400330c00050beda09e673ecd29f0bdd2eb4c"
Oct 30 20:02:30.352282 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:30.352214    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5e61c5578da65f9c92d5742d91400330c00050beda09e673ecd29f0bdd2eb4c"} err="failed to get container status \"a5e61c5578da65f9c92d5742d91400330c00050beda09e673ecd29f0bdd2eb4c\": rpc error: code = NotFound desc = could not find container \"a5e61c5578da65f9c92d5742d91400330c00050beda09e673ecd29f0bdd2eb4c\": container with ID starting with a5e61c5578da65f9c92d5742d91400330c00050beda09e673ecd29f0bdd2eb4c not found: ID does not exist"
Oct 30 20:02:30.366185 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:30.366164    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg"]
Oct 30 20:02:30.371305 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:30.371279    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-6wnbg"]
Oct 30 20:02:30.793909 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:30.793837    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7d78490f-9250-4c76-9786-4d5499c34beb" path="/var/lib/kubelet/pods/7d78490f-9250-4c76-9786-4d5499c34beb/volumes"
Oct 30 20:02:31.191901 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:02:31.191861    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 20:02:49.040037 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:49.040001    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5"]
Oct 30 20:02:49.040627 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:49.040241    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5" podUID="7938d051-21e1-4dfc-8d60-a3162c8398c1" containerName="istio-proxy" containerID="cri-o://01ea146cc38a96596da921936b295813275892534276527fd5e244488861f496" gracePeriod=30
Oct 30 20:02:54.322091 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:54.322059    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5"
Oct 30 20:02:54.333849 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:54.333817    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7938d051-21e1-4dfc-8d60-a3162c8398c1-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7938d051-21e1-4dfc-8d60-a3162c8398c1-credential-socket\") pod \"7938d051-21e1-4dfc-8d60-a3162c8398c1\" (UID: \"7938d051-21e1-4dfc-8d60-a3162c8398c1\") "
Oct 30 20:02:54.333976 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:54.333882    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7938d051-21e1-4dfc-8d60-a3162c8398c1-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7938d051-21e1-4dfc-8d60-a3162c8398c1-workload-certs\") pod \"7938d051-21e1-4dfc-8d60-a3162c8398c1\" (UID: \"7938d051-21e1-4dfc-8d60-a3162c8398c1\") "
Oct 30 20:02:54.333976 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:54.333929    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7938d051-21e1-4dfc-8d60-a3162c8398c1-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7938d051-21e1-4dfc-8d60-a3162c8398c1-istio-data\") pod \"7938d051-21e1-4dfc-8d60-a3162c8398c1\" (UID: \"7938d051-21e1-4dfc-8d60-a3162c8398c1\") "
Oct 30 20:02:54.334087 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:54.334063    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7938d051-21e1-4dfc-8d60-a3162c8398c1-credential-socket" pod "7938d051-21e1-4dfc-8d60-a3162c8398c1" (UID: "7938d051-21e1-4dfc-8d60-a3162c8398c1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:02:54.334144 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:54.334073    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7938d051-21e1-4dfc-8d60-a3162c8398c1-kube-api-access-6zk67\" (UniqueName: \"kubernetes.io/projected/7938d051-21e1-4dfc-8d60-a3162c8398c1-kube-api-access-6zk67\") pod \"7938d051-21e1-4dfc-8d60-a3162c8398c1\" (UID: \"7938d051-21e1-4dfc-8d60-a3162c8398c1\") "
Oct 30 20:02:54.334205 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:54.334194    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7938d051-21e1-4dfc-8d60-a3162c8398c1-istio-token\" (UniqueName: \"kubernetes.io/projected/7938d051-21e1-4dfc-8d60-a3162c8398c1-istio-token\") pod \"7938d051-21e1-4dfc-8d60-a3162c8398c1\" (UID: \"7938d051-21e1-4dfc-8d60-a3162c8398c1\") "
Oct 30 20:02:54.334262 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:54.334195    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7938d051-21e1-4dfc-8d60-a3162c8398c1-workload-certs" pod "7938d051-21e1-4dfc-8d60-a3162c8398c1" (UID: "7938d051-21e1-4dfc-8d60-a3162c8398c1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:02:54.334262 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:54.334226    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7938d051-21e1-4dfc-8d60-a3162c8398c1-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7938d051-21e1-4dfc-8d60-a3162c8398c1-istio-envoy\") pod \"7938d051-21e1-4dfc-8d60-a3162c8398c1\" (UID: \"7938d051-21e1-4dfc-8d60-a3162c8398c1\") "
Oct 30 20:02:54.334365 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:54.334274    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7938d051-21e1-4dfc-8d60-a3162c8398c1-istio-data" pod "7938d051-21e1-4dfc-8d60-a3162c8398c1" (UID: "7938d051-21e1-4dfc-8d60-a3162c8398c1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:02:54.334365 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:54.334293    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7938d051-21e1-4dfc-8d60-a3162c8398c1-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7938d051-21e1-4dfc-8d60-a3162c8398c1-istio-podinfo\") pod \"7938d051-21e1-4dfc-8d60-a3162c8398c1\" (UID: \"7938d051-21e1-4dfc-8d60-a3162c8398c1\") "
Oct 30 20:02:54.334365 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:54.334346    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7938d051-21e1-4dfc-8d60-a3162c8398c1-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7938d051-21e1-4dfc-8d60-a3162c8398c1-workload-socket\") pod \"7938d051-21e1-4dfc-8d60-a3162c8398c1\" (UID: \"7938d051-21e1-4dfc-8d60-a3162c8398c1\") "
Oct 30 20:02:54.334542 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:54.334404    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7938d051-21e1-4dfc-8d60-a3162c8398c1-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7938d051-21e1-4dfc-8d60-a3162c8398c1-istiod-ca-cert\") pod \"7938d051-21e1-4dfc-8d60-a3162c8398c1\" (UID: \"7938d051-21e1-4dfc-8d60-a3162c8398c1\") "
Oct 30 20:02:54.334678 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:54.334653    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7938d051-21e1-4dfc-8d60-a3162c8398c1-workload-socket" pod "7938d051-21e1-4dfc-8d60-a3162c8398c1" (UID: "7938d051-21e1-4dfc-8d60-a3162c8398c1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:02:54.334844 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:54.334815    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7938d051-21e1-4dfc-8d60-a3162c8398c1-istiod-ca-cert" pod "7938d051-21e1-4dfc-8d60-a3162c8398c1" (UID: "7938d051-21e1-4dfc-8d60-a3162c8398c1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:02:54.335123 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:54.335101    6996 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7938d051-21e1-4dfc-8d60-a3162c8398c1-workload-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:02:54.335123 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:54.335123    6996 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7938d051-21e1-4dfc-8d60-a3162c8398c1-istio-data\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:02:54.335265 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:54.335132    6996 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7938d051-21e1-4dfc-8d60-a3162c8398c1-workload-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:02:54.335265 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:54.335141    6996 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7938d051-21e1-4dfc-8d60-a3162c8398c1-istiod-ca-cert\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:02:54.335265 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:54.335149    6996 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7938d051-21e1-4dfc-8d60-a3162c8398c1-credential-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:02:54.336392 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:54.336367    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7938d051-21e1-4dfc-8d60-a3162c8398c1-istio-podinfo" pod "7938d051-21e1-4dfc-8d60-a3162c8398c1" (UID: "7938d051-21e1-4dfc-8d60-a3162c8398c1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 20:02:54.336392 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:54.336382    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7938d051-21e1-4dfc-8d60-a3162c8398c1-kube-api-access-6zk67" pod "7938d051-21e1-4dfc-8d60-a3162c8398c1" (UID: "7938d051-21e1-4dfc-8d60-a3162c8398c1"). InnerVolumeSpecName "kube-api-access-6zk67". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:02:54.336693 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:54.336672    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7938d051-21e1-4dfc-8d60-a3162c8398c1-istio-envoy" pod "7938d051-21e1-4dfc-8d60-a3162c8398c1" (UID: "7938d051-21e1-4dfc-8d60-a3162c8398c1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:02:54.336776 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:54.336719    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7938d051-21e1-4dfc-8d60-a3162c8398c1-istio-token" pod "7938d051-21e1-4dfc-8d60-a3162c8398c1" (UID: "7938d051-21e1-4dfc-8d60-a3162c8398c1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:02:54.436227 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:54.436192    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6zk67\" (UniqueName: \"kubernetes.io/projected/7938d051-21e1-4dfc-8d60-a3162c8398c1-kube-api-access-6zk67\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:02:54.436227 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:54.436219    6996 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7938d051-21e1-4dfc-8d60-a3162c8398c1-istio-token\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:02:54.436227 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:54.436229    6996 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7938d051-21e1-4dfc-8d60-a3162c8398c1-istio-envoy\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:02:54.436449 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:54.436237    6996 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7938d051-21e1-4dfc-8d60-a3162c8398c1-istio-podinfo\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:02:54.549704 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:54.549663    6996 generic.go:356] "Generic (PLEG): container finished" podID="7938d051-21e1-4dfc-8d60-a3162c8398c1" containerID="01ea146cc38a96596da921936b295813275892534276527fd5e244488861f496" exitCode=0
Oct 30 20:02:54.549867 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:54.549715    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5"
Oct 30 20:02:54.549867 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:54.549749    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5" event={"ID":"7938d051-21e1-4dfc-8d60-a3162c8398c1","Type":"ContainerDied","Data":"01ea146cc38a96596da921936b295813275892534276527fd5e244488861f496"}
Oct 30 20:02:54.549867 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:54.549829    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5" event={"ID":"7938d051-21e1-4dfc-8d60-a3162c8398c1","Type":"ContainerDied","Data":"7bc6bdf44639b6996bcf54ce042e812237d3d642a8c8945423165d0ed5e530c4"}
Oct 30 20:02:54.549867 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:54.549854    6996 scope.go:122] "RemoveContainer" containerID="01ea146cc38a96596da921936b295813275892534276527fd5e244488861f496"
Oct 30 20:02:54.561927 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:54.561910    6996 scope.go:122] "RemoveContainer" containerID="01ea146cc38a96596da921936b295813275892534276527fd5e244488861f496"
Oct 30 20:02:54.562194 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:02:54.562168    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01ea146cc38a96596da921936b295813275892534276527fd5e244488861f496\": container with ID starting with 01ea146cc38a96596da921936b295813275892534276527fd5e244488861f496 not found: ID does not exist" containerID="01ea146cc38a96596da921936b295813275892534276527fd5e244488861f496"
Oct 30 20:02:54.562294 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:54.562202    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01ea146cc38a96596da921936b295813275892534276527fd5e244488861f496"} err="failed to get container status \"01ea146cc38a96596da921936b295813275892534276527fd5e244488861f496\": rpc error: code = NotFound desc = could not find container \"01ea146cc38a96596da921936b295813275892534276527fd5e244488861f496\": container with ID starting with 01ea146cc38a96596da921936b295813275892534276527fd5e244488861f496 not found: ID does not exist"
Oct 30 20:02:54.579202 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:54.579147    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5"]
Oct 30 20:02:54.582410 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:54.582391    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-klpl5"]
Oct 30 20:02:54.793757 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:02:54.793720    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7938d051-21e1-4dfc-8d60-a3162c8398c1" path="/var/lib/kubelet/pods/7938d051-21e1-4dfc-8d60-a3162c8398c1/volumes"
Oct 30 20:03:01.192196 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:03:01.192155    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 20:03:02.445941 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:02.445905    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l"]
Oct 30 20:03:02.446348 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:02.446114    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l" podUID="9584be0d-abe4-4783-8849-f4b79f3094db" containerName="istio-proxy" containerID="cri-o://391c0de62fd01a10bb0e6642a23af18309b2acae2d4ebe39f9c032df75b9cca5" gracePeriod=30
Oct 30 20:03:06.143890 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:06.143856    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-969878-d8f4759b7-kdsjv"]
Oct 30 20:03:06.144573 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:06.144558    6996 cpu_manager.go:414] "containerMap: removing container" podUID="7d78490f-9250-4c76-9786-4d5499c34beb" containerName="istio-proxy"
Oct 30 20:03:06.144633 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:06.144575    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7d78490f-9250-4c76-9786-4d5499c34beb" containerName="istio-proxy"
Oct 30 20:03:06.144633 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:06.144599    6996 cpu_manager.go:414] "containerMap: removing container" podUID="7938d051-21e1-4dfc-8d60-a3162c8398c1" containerName="istio-proxy"
Oct 30 20:03:06.144633 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:06.144605    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7938d051-21e1-4dfc-8d60-a3162c8398c1" containerName="istio-proxy"
Oct 30 20:03:06.144731 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:06.144719    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="7d78490f-9250-4c76-9786-4d5499c34beb" containerName="istio-proxy"
Oct 30 20:03:06.144764 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:06.144732    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="7938d051-21e1-4dfc-8d60-a3162c8398c1" containerName="istio-proxy"
Oct 30 20:03:06.150749 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:06.150725    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-969878-d8f4759b7-kdsjv"
Oct 30 20:03:06.160159 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:06.159914    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzrfx\" (UniqueName: \"kubernetes.io/projected/89a06251-4c13-4b46-99fb-6b3db0c4ddf6-kube-api-access-nzrfx\") pod \"maas-api-e2e-extmod-b-969878-d8f4759b7-kdsjv\" (UID: \"89a06251-4c13-4b46-99fb-6b3db0c4ddf6\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-969878-d8f4759b7-kdsjv"
Oct 30 20:03:06.160159 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:06.159989    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/89a06251-4c13-4b46-99fb-6b3db0c4ddf6-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-969878-d8f4759b7-kdsjv\" (UID: \"89a06251-4c13-4b46-99fb-6b3db0c4ddf6\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-969878-d8f4759b7-kdsjv"
Oct 30 20:03:06.160159 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:06.160023    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89a06251-4c13-4b46-99fb-6b3db0c4ddf6-maas-api-tls\") pod \"maas-api-e2e-extmod-b-969878-d8f4759b7-kdsjv\" (UID: \"89a06251-4c13-4b46-99fb-6b3db0c4ddf6\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-969878-d8f4759b7-kdsjv"
Oct 30 20:03:06.160159 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:06.160046    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-b-969878-serving-cert\""
Oct 30 20:03:06.160159 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:06.160105    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89a06251-4c13-4b46-99fb-6b3db0c4ddf6-metrics-certs\") pod \"maas-api-e2e-extmod-b-969878-d8f4759b7-kdsjv\" (UID: \"89a06251-4c13-4b46-99fb-6b3db0c4ddf6\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-969878-d8f4759b7-kdsjv"
Oct 30 20:03:06.175540 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:06.171014    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-969878-d8f4759b7-kdsjv"]
Oct 30 20:03:06.261655 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:06.261622    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzrfx\" (UniqueName: \"kubernetes.io/projected/89a06251-4c13-4b46-99fb-6b3db0c4ddf6-kube-api-access-nzrfx\") pod \"maas-api-e2e-extmod-b-969878-d8f4759b7-kdsjv\" (UID: \"89a06251-4c13-4b46-99fb-6b3db0c4ddf6\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-969878-d8f4759b7-kdsjv"
Oct 30 20:03:06.261850 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:06.261667    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/89a06251-4c13-4b46-99fb-6b3db0c4ddf6-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-969878-d8f4759b7-kdsjv\" (UID: \"89a06251-4c13-4b46-99fb-6b3db0c4ddf6\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-969878-d8f4759b7-kdsjv"
Oct 30 20:03:06.261850 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:06.261688    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89a06251-4c13-4b46-99fb-6b3db0c4ddf6-maas-api-tls\") pod \"maas-api-e2e-extmod-b-969878-d8f4759b7-kdsjv\" (UID: \"89a06251-4c13-4b46-99fb-6b3db0c4ddf6\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-969878-d8f4759b7-kdsjv"
Oct 30 20:03:06.261850 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:06.261743    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89a06251-4c13-4b46-99fb-6b3db0c4ddf6-metrics-certs\") pod \"maas-api-e2e-extmod-b-969878-d8f4759b7-kdsjv\" (UID: \"89a06251-4c13-4b46-99fb-6b3db0c4ddf6\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-969878-d8f4759b7-kdsjv"
Oct 30 20:03:06.262018 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:03:06.261849    6996 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-extmod-b-969878-serving-cert: secret "maas-api-e2e-extmod-b-969878-serving-cert" not found
Oct 30 20:03:06.262018 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:03:06.261926    6996 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/89a06251-4c13-4b46-99fb-6b3db0c4ddf6-maas-api-tls podName:89a06251-4c13-4b46-99fb-6b3db0c4ddf6 nodeName:}" failed. No retries permitted until 2026-10-30 20:03:06.761904758 +0000 UTC m=+1862.090022913 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/89a06251-4c13-4b46-99fb-6b3db0c4ddf6-maas-api-tls") pod "maas-api-e2e-extmod-b-969878-d8f4759b7-kdsjv" (UID: "89a06251-4c13-4b46-99fb-6b3db0c4ddf6") : secret "maas-api-e2e-extmod-b-969878-serving-cert" not found
Oct 30 20:03:06.262361 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:06.262337    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/89a06251-4c13-4b46-99fb-6b3db0c4ddf6-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-969878-d8f4759b7-kdsjv\" (UID: \"89a06251-4c13-4b46-99fb-6b3db0c4ddf6\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-969878-d8f4759b7-kdsjv"
Oct 30 20:03:06.264293 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:06.264270    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89a06251-4c13-4b46-99fb-6b3db0c4ddf6-metrics-certs\") pod \"maas-api-e2e-extmod-b-969878-d8f4759b7-kdsjv\" (UID: \"89a06251-4c13-4b46-99fb-6b3db0c4ddf6\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-969878-d8f4759b7-kdsjv"
Oct 30 20:03:06.282749 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:06.282678    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzrfx\" (UniqueName: \"kubernetes.io/projected/89a06251-4c13-4b46-99fb-6b3db0c4ddf6-kube-api-access-nzrfx\") pod \"maas-api-e2e-extmod-b-969878-d8f4759b7-kdsjv\" (UID: \"89a06251-4c13-4b46-99fb-6b3db0c4ddf6\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-969878-d8f4759b7-kdsjv"
Oct 30 20:03:06.768676 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:06.768640    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89a06251-4c13-4b46-99fb-6b3db0c4ddf6-maas-api-tls\") pod \"maas-api-e2e-extmod-b-969878-d8f4759b7-kdsjv\" (UID: \"89a06251-4c13-4b46-99fb-6b3db0c4ddf6\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-969878-d8f4759b7-kdsjv"
Oct 30 20:03:06.771100 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:06.771073    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89a06251-4c13-4b46-99fb-6b3db0c4ddf6-maas-api-tls\") pod \"maas-api-e2e-extmod-b-969878-d8f4759b7-kdsjv\" (UID: \"89a06251-4c13-4b46-99fb-6b3db0c4ddf6\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-969878-d8f4759b7-kdsjv"
Oct 30 20:03:06.780073 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:06.780051    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-969878-d8f4759b7-kdsjv"
Oct 30 20:03:06.904525 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:06.902266    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 20:03:07.165240 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:07.162659    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-969878-d8f4759b7-kdsjv"]
Oct 30 20:03:07.670931 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:07.670775    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-969878-d8f4759b7-kdsjv" event={"ID":"89a06251-4c13-4b46-99fb-6b3db0c4ddf6","Type":"ContainerStarted","Data":"edc3339fd991b871b6ed3140d97d370514456eb0ea45affc101c2f5e903abc95"}
Oct 30 20:03:07.672834 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:07.672808    6996 generic.go:356] "Generic (PLEG): container finished" podID="9584be0d-abe4-4783-8849-f4b79f3094db" containerID="391c0de62fd01a10bb0e6642a23af18309b2acae2d4ebe39f9c032df75b9cca5" exitCode=0
Oct 30 20:03:07.672965 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:07.672861    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l" event={"ID":"9584be0d-abe4-4783-8849-f4b79f3094db","Type":"ContainerDied","Data":"391c0de62fd01a10bb0e6642a23af18309b2acae2d4ebe39f9c032df75b9cca5"}
Oct 30 20:03:07.920357 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:07.920029    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l"
Oct 30 20:03:08.048436 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:08.047338    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9584be0d-abe4-4783-8849-f4b79f3094db-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9584be0d-abe4-4783-8849-f4b79f3094db-workload-certs\") pod \"9584be0d-abe4-4783-8849-f4b79f3094db\" (UID: \"9584be0d-abe4-4783-8849-f4b79f3094db\") "
Oct 30 20:03:08.048436 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:08.047429    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9584be0d-abe4-4783-8849-f4b79f3094db-kube-api-access-k8jsh\" (UniqueName: \"kubernetes.io/projected/9584be0d-abe4-4783-8849-f4b79f3094db-kube-api-access-k8jsh\") pod \"9584be0d-abe4-4783-8849-f4b79f3094db\" (UID: \"9584be0d-abe4-4783-8849-f4b79f3094db\") "
Oct 30 20:03:08.048436 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:08.047470    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9584be0d-abe4-4783-8849-f4b79f3094db-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9584be0d-abe4-4783-8849-f4b79f3094db-istiod-ca-cert\") pod \"9584be0d-abe4-4783-8849-f4b79f3094db\" (UID: \"9584be0d-abe4-4783-8849-f4b79f3094db\") "
Oct 30 20:03:08.048436 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:08.047585    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9584be0d-abe4-4783-8849-f4b79f3094db-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9584be0d-abe4-4783-8849-f4b79f3094db-istio-podinfo\") pod \"9584be0d-abe4-4783-8849-f4b79f3094db\" (UID: \"9584be0d-abe4-4783-8849-f4b79f3094db\") "
Oct 30 20:03:08.048436 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:08.047613    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9584be0d-abe4-4783-8849-f4b79f3094db-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9584be0d-abe4-4783-8849-f4b79f3094db-istio-data\") pod \"9584be0d-abe4-4783-8849-f4b79f3094db\" (UID: \"9584be0d-abe4-4783-8849-f4b79f3094db\") "
Oct 30 20:03:08.048436 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:08.047635    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9584be0d-abe4-4783-8849-f4b79f3094db-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9584be0d-abe4-4783-8849-f4b79f3094db-credential-socket\") pod \"9584be0d-abe4-4783-8849-f4b79f3094db\" (UID: \"9584be0d-abe4-4783-8849-f4b79f3094db\") "
Oct 30 20:03:08.048436 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:08.047717    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9584be0d-abe4-4783-8849-f4b79f3094db-istio-token\" (UniqueName: \"kubernetes.io/projected/9584be0d-abe4-4783-8849-f4b79f3094db-istio-token\") pod \"9584be0d-abe4-4783-8849-f4b79f3094db\" (UID: \"9584be0d-abe4-4783-8849-f4b79f3094db\") "
Oct 30 20:03:08.048436 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:08.047804    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9584be0d-abe4-4783-8849-f4b79f3094db-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9584be0d-abe4-4783-8849-f4b79f3094db-workload-socket\") pod \"9584be0d-abe4-4783-8849-f4b79f3094db\" (UID: \"9584be0d-abe4-4783-8849-f4b79f3094db\") "
Oct 30 20:03:08.048436 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:08.047832    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9584be0d-abe4-4783-8849-f4b79f3094db-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9584be0d-abe4-4783-8849-f4b79f3094db-istio-envoy\") pod \"9584be0d-abe4-4783-8849-f4b79f3094db\" (UID: \"9584be0d-abe4-4783-8849-f4b79f3094db\") "
Oct 30 20:03:08.049022 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:08.048823    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9584be0d-abe4-4783-8849-f4b79f3094db-credential-socket" pod "9584be0d-abe4-4783-8849-f4b79f3094db" (UID: "9584be0d-abe4-4783-8849-f4b79f3094db"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:03:08.049022 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:08.048914    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9584be0d-abe4-4783-8849-f4b79f3094db-istio-data" pod "9584be0d-abe4-4783-8849-f4b79f3094db" (UID: "9584be0d-abe4-4783-8849-f4b79f3094db"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:03:08.049022 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:08.048974    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9584be0d-abe4-4783-8849-f4b79f3094db-workload-certs" pod "9584be0d-abe4-4783-8849-f4b79f3094db" (UID: "9584be0d-abe4-4783-8849-f4b79f3094db"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:03:08.049022 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:08.049011    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9584be0d-abe4-4783-8849-f4b79f3094db-istiod-ca-cert" pod "9584be0d-abe4-4783-8849-f4b79f3094db" (UID: "9584be0d-abe4-4783-8849-f4b79f3094db"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:03:08.049225 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:08.049113    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9584be0d-abe4-4783-8849-f4b79f3094db-workload-socket" pod "9584be0d-abe4-4783-8849-f4b79f3094db" (UID: "9584be0d-abe4-4783-8849-f4b79f3094db"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:03:08.050626 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:08.050598    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9584be0d-abe4-4783-8849-f4b79f3094db-istio-envoy" pod "9584be0d-abe4-4783-8849-f4b79f3094db" (UID: "9584be0d-abe4-4783-8849-f4b79f3094db"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:03:08.051170 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:08.051143    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9584be0d-abe4-4783-8849-f4b79f3094db-istio-podinfo" pod "9584be0d-abe4-4783-8849-f4b79f3094db" (UID: "9584be0d-abe4-4783-8849-f4b79f3094db"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 20:03:08.051725 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:08.051700    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9584be0d-abe4-4783-8849-f4b79f3094db-istio-token" pod "9584be0d-abe4-4783-8849-f4b79f3094db" (UID: "9584be0d-abe4-4783-8849-f4b79f3094db"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:03:08.051869 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:08.051849    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9584be0d-abe4-4783-8849-f4b79f3094db-kube-api-access-k8jsh" pod "9584be0d-abe4-4783-8849-f4b79f3094db" (UID: "9584be0d-abe4-4783-8849-f4b79f3094db"). InnerVolumeSpecName "kube-api-access-k8jsh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:03:08.150133 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:08.150041    6996 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9584be0d-abe4-4783-8849-f4b79f3094db-istio-token\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:03:08.150133 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:08.150083    6996 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9584be0d-abe4-4783-8849-f4b79f3094db-workload-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:03:08.150133 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:08.150097    6996 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9584be0d-abe4-4783-8849-f4b79f3094db-istio-envoy\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:03:08.150133 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:08.150109    6996 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9584be0d-abe4-4783-8849-f4b79f3094db-workload-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:03:08.150133 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:08.150125    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k8jsh\" (UniqueName: \"kubernetes.io/projected/9584be0d-abe4-4783-8849-f4b79f3094db-kube-api-access-k8jsh\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:03:08.150565 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:08.150141    6996 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9584be0d-abe4-4783-8849-f4b79f3094db-istiod-ca-cert\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:03:08.150565 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:08.150155    6996 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9584be0d-abe4-4783-8849-f4b79f3094db-istio-podinfo\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:03:08.150565 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:08.150167    6996 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9584be0d-abe4-4783-8849-f4b79f3094db-istio-data\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:03:08.150565 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:08.150180    6996 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9584be0d-abe4-4783-8849-f4b79f3094db-credential-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:03:08.682105 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:08.682068    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-969878-d8f4759b7-kdsjv" event={"ID":"89a06251-4c13-4b46-99fb-6b3db0c4ddf6","Type":"ContainerStarted","Data":"99b2de11388801ed32596c535fe72ae5288e673c9cd4367f0c854a38c3762861"}
Oct 30 20:03:08.682673 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:08.682298    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-969878-d8f4759b7-kdsjv"
Oct 30 20:03:08.684047 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:08.684019    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l" event={"ID":"9584be0d-abe4-4783-8849-f4b79f3094db","Type":"ContainerDied","Data":"2aaca1e06eee25cceb1c97204eb643c3e9b8b22cc17d94e2fd5242aa84c24054"}
Oct 30 20:03:08.684179 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:08.684051    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l"
Oct 30 20:03:08.684179 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:08.684077    6996 scope.go:122] "RemoveContainer" containerID="391c0de62fd01a10bb0e6642a23af18309b2acae2d4ebe39f9c032df75b9cca5"
Oct 30 20:03:08.711550 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:08.703707    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-969878-d8f4759b7-kdsjv" podStartSLOduration=1.990215032 podStartE2EDuration="2.70369001s" totalImagesPullingTime="713.474978ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:03:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:03:08.701649216 +0000 UTC m=+1864.029767368" watchObservedRunningTime="2026-10-30 20:03:08.70369001 +0000 UTC m=+1864.031808164"
Oct 30 20:03:08.776501 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:08.771558    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l"]
Oct 30 20:03:08.777550 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:08.777518    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-7g96l"]
Oct 30 20:03:08.805097 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:08.803262    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9584be0d-abe4-4783-8849-f4b79f3094db" path="/var/lib/kubelet/pods/9584be0d-abe4-4783-8849-f4b79f3094db/volumes"
Oct 30 20:03:11.416630 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:11.416599    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-4d5758-6985bf95bc-twp4p"]
Oct 30 20:03:11.417294 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:11.417279    6996 cpu_manager.go:414] "containerMap: removing container" podUID="9584be0d-abe4-4783-8849-f4b79f3094db" containerName="istio-proxy"
Oct 30 20:03:11.417347 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:11.417296    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9584be0d-abe4-4783-8849-f4b79f3094db" containerName="istio-proxy"
Oct 30 20:03:11.417418 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:11.417409    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="9584be0d-abe4-4783-8849-f4b79f3094db" containerName="istio-proxy"
Oct 30 20:03:11.422153 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:11.422129    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-4d5758-6985bf95bc-twp4p"
Oct 30 20:03:11.432747 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:11.432720    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-b-4d5758\""
Oct 30 20:03:11.433015 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:11.432996    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-b-4d5758-dockercfg-jdrbp\""
Oct 30 20:03:11.437514 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:11.437491    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-4d5758-6985bf95bc-twp4p"]
Oct 30 20:03:11.636589 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:11.636542    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbmnf\" (UniqueName: \"kubernetes.io/projected/316882f2-58f3-404a-a515-310efc6f2462-kube-api-access-nbmnf\") pod \"payload-pre-processing-e2e-extmod-b-4d5758-6985bf95bc-twp4p\" (UID: \"316882f2-58f3-404a-a515-310efc6f2462\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-4d5758-6985bf95bc-twp4p"
Oct 30 20:03:11.636868 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:11.636846    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/316882f2-58f3-404a-a515-310efc6f2462-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-4d5758-6985bf95bc-twp4p\" (UID: \"316882f2-58f3-404a-a515-310efc6f2462\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-4d5758-6985bf95bc-twp4p"
Oct 30 20:03:11.737929 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:11.737844    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/316882f2-58f3-404a-a515-310efc6f2462-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-4d5758-6985bf95bc-twp4p\" (UID: \"316882f2-58f3-404a-a515-310efc6f2462\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-4d5758-6985bf95bc-twp4p"
Oct 30 20:03:11.738084 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:11.737951    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbmnf\" (UniqueName: \"kubernetes.io/projected/316882f2-58f3-404a-a515-310efc6f2462-kube-api-access-nbmnf\") pod \"payload-pre-processing-e2e-extmod-b-4d5758-6985bf95bc-twp4p\" (UID: \"316882f2-58f3-404a-a515-310efc6f2462\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-4d5758-6985bf95bc-twp4p"
Oct 30 20:03:11.738590 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:11.738570    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/316882f2-58f3-404a-a515-310efc6f2462-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-4d5758-6985bf95bc-twp4p\" (UID: \"316882f2-58f3-404a-a515-310efc6f2462\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-4d5758-6985bf95bc-twp4p"
Oct 30 20:03:11.750640 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:11.750608    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbmnf\" (UniqueName: \"kubernetes.io/projected/316882f2-58f3-404a-a515-310efc6f2462-kube-api-access-nbmnf\") pod \"payload-pre-processing-e2e-extmod-b-4d5758-6985bf95bc-twp4p\" (UID: \"316882f2-58f3-404a-a515-310efc6f2462\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-4d5758-6985bf95bc-twp4p"
Oct 30 20:03:12.036614 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:12.036534    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-4d5758-6985bf95bc-twp4p"
Oct 30 20:03:12.323942 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:12.323909    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-4d5758-6985bf95bc-twp4p"]
Oct 30 20:03:12.324954 ip-10-0-141-42 kubenswrapper[6996]: W1030 20:03:12.324919    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod316882f2_58f3_404a_a515_310efc6f2462.slice/crio-cb1acc42cd62b2cd63a762ce76ca5cff98716c3db71d24ebe540434547e821b4 WatchSource:0}: Error finding container cb1acc42cd62b2cd63a762ce76ca5cff98716c3db71d24ebe540434547e821b4: Status 404 returned error can't find the container with id cb1acc42cd62b2cd63a762ce76ca5cff98716c3db71d24ebe540434547e821b4
Oct 30 20:03:12.717222 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:12.717136    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-4d5758-6985bf95bc-twp4p" event={"ID":"316882f2-58f3-404a-a515-310efc6f2462","Type":"ContainerStarted","Data":"5e03fe9f19e192b5b4a3d1cec291e20f83467107868ac65fc820e1ffd10c34c6"}
Oct 30 20:03:12.717222 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:12.717198    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-4d5758-6985bf95bc-twp4p" event={"ID":"316882f2-58f3-404a-a515-310efc6f2462","Type":"ContainerStarted","Data":"cb1acc42cd62b2cd63a762ce76ca5cff98716c3db71d24ebe540434547e821b4"}
Oct 30 20:03:12.717644 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:12.717227    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-4d5758-6985bf95bc-twp4p"
Oct 30 20:03:12.739142 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:12.739097    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-4d5758-6985bf95bc-twp4p" podStartSLOduration=1.7390837019999998 podStartE2EDuration="1.739083702s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:03:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:03:12.737585384 +0000 UTC m=+1868.065703537" watchObservedRunningTime="2026-10-30 20:03:12.739083702 +0000 UTC m=+1868.067201855"
Oct 30 20:03:14.699682 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:14.699646    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-969878-d8f4759b7-kdsjv"
Oct 30 20:03:23.728025 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:23.727990    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-4d5758-6985bf95bc-twp4p"
Oct 30 20:03:31.337651 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:03:31.337585    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 20:03:34.551173 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:34.551140    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-4d5758-766c8d5467-z6cst"]
Oct 30 20:03:34.556918 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:34.556893    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-4d5758-766c8d5467-z6cst"
Oct 30 20:03:34.566109 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:34.566079    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-b-4d5758-serving-cert\""
Oct 30 20:03:34.571313 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:34.571282    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-4d5758-766c8d5467-z6cst"]
Oct 30 20:03:34.627762 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:34.627726    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/24beb51d-3611-47ca-8061-b17dbfc16c80-maas-api-tls\") pod \"maas-api-e2e-extmod-b-4d5758-766c8d5467-z6cst\" (UID: \"24beb51d-3611-47ca-8061-b17dbfc16c80\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-4d5758-766c8d5467-z6cst"
Oct 30 20:03:34.627963 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:34.627885    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/24beb51d-3611-47ca-8061-b17dbfc16c80-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-4d5758-766c8d5467-z6cst\" (UID: \"24beb51d-3611-47ca-8061-b17dbfc16c80\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-4d5758-766c8d5467-z6cst"
Oct 30 20:03:34.627963 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:34.627933    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7xpb\" (UniqueName: \"kubernetes.io/projected/24beb51d-3611-47ca-8061-b17dbfc16c80-kube-api-access-k7xpb\") pod \"maas-api-e2e-extmod-b-4d5758-766c8d5467-z6cst\" (UID: \"24beb51d-3611-47ca-8061-b17dbfc16c80\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-4d5758-766c8d5467-z6cst"
Oct 30 20:03:34.628126 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:34.628110    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24beb51d-3611-47ca-8061-b17dbfc16c80-metrics-certs\") pod \"maas-api-e2e-extmod-b-4d5758-766c8d5467-z6cst\" (UID: \"24beb51d-3611-47ca-8061-b17dbfc16c80\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-4d5758-766c8d5467-z6cst"
Oct 30 20:03:34.729347 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:34.729306    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24beb51d-3611-47ca-8061-b17dbfc16c80-metrics-certs\") pod \"maas-api-e2e-extmod-b-4d5758-766c8d5467-z6cst\" (UID: \"24beb51d-3611-47ca-8061-b17dbfc16c80\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-4d5758-766c8d5467-z6cst"
Oct 30 20:03:34.729571 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:34.729523    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/24beb51d-3611-47ca-8061-b17dbfc16c80-maas-api-tls\") pod \"maas-api-e2e-extmod-b-4d5758-766c8d5467-z6cst\" (UID: \"24beb51d-3611-47ca-8061-b17dbfc16c80\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-4d5758-766c8d5467-z6cst"
Oct 30 20:03:34.729653 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:34.729621    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/24beb51d-3611-47ca-8061-b17dbfc16c80-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-4d5758-766c8d5467-z6cst\" (UID: \"24beb51d-3611-47ca-8061-b17dbfc16c80\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-4d5758-766c8d5467-z6cst"
Oct 30 20:03:34.729713 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:34.729662    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7xpb\" (UniqueName: \"kubernetes.io/projected/24beb51d-3611-47ca-8061-b17dbfc16c80-kube-api-access-k7xpb\") pod \"maas-api-e2e-extmod-b-4d5758-766c8d5467-z6cst\" (UID: \"24beb51d-3611-47ca-8061-b17dbfc16c80\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-4d5758-766c8d5467-z6cst"
Oct 30 20:03:34.729713 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:03:34.729685    6996 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-extmod-b-4d5758-serving-cert: secret "maas-api-e2e-extmod-b-4d5758-serving-cert" not found
Oct 30 20:03:34.729823 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:03:34.729776    6996 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24beb51d-3611-47ca-8061-b17dbfc16c80-maas-api-tls podName:24beb51d-3611-47ca-8061-b17dbfc16c80 nodeName:}" failed. No retries permitted until 2026-10-30 20:03:35.229751113 +0000 UTC m=+1890.557869244 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/24beb51d-3611-47ca-8061-b17dbfc16c80-maas-api-tls") pod "maas-api-e2e-extmod-b-4d5758-766c8d5467-z6cst" (UID: "24beb51d-3611-47ca-8061-b17dbfc16c80") : secret "maas-api-e2e-extmod-b-4d5758-serving-cert" not found
Oct 30 20:03:34.730246 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:34.730221    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/24beb51d-3611-47ca-8061-b17dbfc16c80-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-4d5758-766c8d5467-z6cst\" (UID: \"24beb51d-3611-47ca-8061-b17dbfc16c80\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-4d5758-766c8d5467-z6cst"
Oct 30 20:03:34.735145 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:34.735116    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24beb51d-3611-47ca-8061-b17dbfc16c80-metrics-certs\") pod \"maas-api-e2e-extmod-b-4d5758-766c8d5467-z6cst\" (UID: \"24beb51d-3611-47ca-8061-b17dbfc16c80\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-4d5758-766c8d5467-z6cst"
Oct 30 20:03:34.744383 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:34.744350    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7xpb\" (UniqueName: \"kubernetes.io/projected/24beb51d-3611-47ca-8061-b17dbfc16c80-kube-api-access-k7xpb\") pod \"maas-api-e2e-extmod-b-4d5758-766c8d5467-z6cst\" (UID: \"24beb51d-3611-47ca-8061-b17dbfc16c80\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-4d5758-766c8d5467-z6cst"
Oct 30 20:03:35.239320 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:35.239261    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/24beb51d-3611-47ca-8061-b17dbfc16c80-maas-api-tls\") pod \"maas-api-e2e-extmod-b-4d5758-766c8d5467-z6cst\" (UID: \"24beb51d-3611-47ca-8061-b17dbfc16c80\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-4d5758-766c8d5467-z6cst"
Oct 30 20:03:35.241806 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:35.241781    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/24beb51d-3611-47ca-8061-b17dbfc16c80-maas-api-tls\") pod \"maas-api-e2e-extmod-b-4d5758-766c8d5467-z6cst\" (UID: \"24beb51d-3611-47ca-8061-b17dbfc16c80\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-4d5758-766c8d5467-z6cst"
Oct 30 20:03:35.479385 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:35.479334    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-4d5758-766c8d5467-z6cst"
Oct 30 20:03:35.828809 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:35.828777    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-4d5758-766c8d5467-z6cst"]
Oct 30 20:03:35.830622 ip-10-0-141-42 kubenswrapper[6996]: W1030 20:03:35.830595    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24beb51d_3611_47ca_8061_b17dbfc16c80.slice/crio-ad2a67fef2d7ebc7eef1e98cf9302f9481073ba6973dc2c5c70ac34a8df1d6b6 WatchSource:0}: Error finding container ad2a67fef2d7ebc7eef1e98cf9302f9481073ba6973dc2c5c70ac34a8df1d6b6: Status 404 returned error can't find the container with id ad2a67fef2d7ebc7eef1e98cf9302f9481073ba6973dc2c5c70ac34a8df1d6b6
Oct 30 20:03:35.920399 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:35.920366    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-4d5758-766c8d5467-z6cst" event={"ID":"24beb51d-3611-47ca-8061-b17dbfc16c80","Type":"ContainerStarted","Data":"ad2a67fef2d7ebc7eef1e98cf9302f9481073ba6973dc2c5c70ac34a8df1d6b6"}
Oct 30 20:03:36.928990 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:36.928958    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-4d5758-766c8d5467-z6cst" event={"ID":"24beb51d-3611-47ca-8061-b17dbfc16c80","Type":"ContainerStarted","Data":"be44a5bc7e3d52e65d49e029306791833f63654d70982db4f6eb0624c7f163a2"}
Oct 30 20:03:36.929390 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:36.929026    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-4d5758-766c8d5467-z6cst"
Oct 30 20:03:36.966164 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:36.966113    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-4d5758-766c8d5467-z6cst" podStartSLOduration=2.498389471 podStartE2EDuration="2.966098983s" totalImagesPullingTime="467.709512ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:03:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:03:36.962289659 +0000 UTC m=+1892.290407813" watchObservedRunningTime="2026-10-30 20:03:36.966098983 +0000 UTC m=+1892.294217153"
Oct 30 20:03:39.108917 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:39.108880    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-extmod-b-969878/payload-processing-external-model-e2e-extmod-b-969878-59dbnbrfr"]
Oct 30 20:03:39.115878 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:39.115849    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-969878/payload-processing-external-model-e2e-extmod-b-969878-59dbnbrfr"
Oct 30 20:03:39.130515 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:39.124613    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-969878\"/\"kube-root-ca.crt\""
Oct 30 20:03:39.130515 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:39.125374    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-969878\"/\"openshift-service-ca.crt\""
Oct 30 20:03:39.130515 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:39.125466    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-969878\"/\"routing-overlay\""
Oct 30 20:03:39.130515 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:39.125541    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-969878\"/\"payload-processing-external-model-plugins-e2e-extmod-b-969878\""
Oct 30 20:03:39.130515 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:39.128536    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-969878/payload-processing-external-model-e2e-extmod-b-969878-59dbnbrfr"]
Oct 30 20:03:39.140190 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:39.140164    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-969878\"/\"agc-cred-5fdacea3cbca\""
Oct 30 20:03:39.153575 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:39.147790    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-969878\"/\"payload-processing-external-model-e2e--5b5eb86f-dockercfg-tpqsc\""
Oct 30 20:03:39.206956 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:39.206911    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lw59t\" (UniqueName: \"kubernetes.io/projected/ac917499-56b1-4aab-b9f0-e256dcf04844-kube-api-access-lw59t\") pod \"payload-processing-external-model-e2e-extmod-b-969878-59dbnbrfr\" (UID: \"ac917499-56b1-4aab-b9f0-e256dcf04844\") " pod="ai-tenant-e2e-extmod-b-969878/payload-processing-external-model-e2e-extmod-b-969878-59dbnbrfr"
Oct 30 20:03:39.207163 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:39.206973    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/ac917499-56b1-4aab-b9f0-e256dcf04844-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-969878-59dbnbrfr\" (UID: \"ac917499-56b1-4aab-b9f0-e256dcf04844\") " pod="ai-tenant-e2e-extmod-b-969878/payload-processing-external-model-e2e-extmod-b-969878-59dbnbrfr"
Oct 30 20:03:39.207163 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:39.207135    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac917499-56b1-4aab-b9f0-e256dcf04844-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-969878-59dbnbrfr\" (UID: \"ac917499-56b1-4aab-b9f0-e256dcf04844\") " pod="ai-tenant-e2e-extmod-b-969878/payload-processing-external-model-e2e-extmod-b-969878-59dbnbrfr"
Oct 30 20:03:39.207291 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:39.207217    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/ac917499-56b1-4aab-b9f0-e256dcf04844-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-969878-59dbnbrfr\" (UID: \"ac917499-56b1-4aab-b9f0-e256dcf04844\") " pod="ai-tenant-e2e-extmod-b-969878/payload-processing-external-model-e2e-extmod-b-969878-59dbnbrfr"
Oct 30 20:03:39.308767 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:39.308719    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lw59t\" (UniqueName: \"kubernetes.io/projected/ac917499-56b1-4aab-b9f0-e256dcf04844-kube-api-access-lw59t\") pod \"payload-processing-external-model-e2e-extmod-b-969878-59dbnbrfr\" (UID: \"ac917499-56b1-4aab-b9f0-e256dcf04844\") " pod="ai-tenant-e2e-extmod-b-969878/payload-processing-external-model-e2e-extmod-b-969878-59dbnbrfr"
Oct 30 20:03:39.309059 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:39.309031    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/ac917499-56b1-4aab-b9f0-e256dcf04844-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-969878-59dbnbrfr\" (UID: \"ac917499-56b1-4aab-b9f0-e256dcf04844\") " pod="ai-tenant-e2e-extmod-b-969878/payload-processing-external-model-e2e-extmod-b-969878-59dbnbrfr"
Oct 30 20:03:39.309195 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:39.309116    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac917499-56b1-4aab-b9f0-e256dcf04844-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-969878-59dbnbrfr\" (UID: \"ac917499-56b1-4aab-b9f0-e256dcf04844\") " pod="ai-tenant-e2e-extmod-b-969878/payload-processing-external-model-e2e-extmod-b-969878-59dbnbrfr"
Oct 30 20:03:39.309260 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:39.309222    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/ac917499-56b1-4aab-b9f0-e256dcf04844-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-969878-59dbnbrfr\" (UID: \"ac917499-56b1-4aab-b9f0-e256dcf04844\") " pod="ai-tenant-e2e-extmod-b-969878/payload-processing-external-model-e2e-extmod-b-969878-59dbnbrfr"
Oct 30 20:03:39.310303 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:39.310114    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/ac917499-56b1-4aab-b9f0-e256dcf04844-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-969878-59dbnbrfr\" (UID: \"ac917499-56b1-4aab-b9f0-e256dcf04844\") " pod="ai-tenant-e2e-extmod-b-969878/payload-processing-external-model-e2e-extmod-b-969878-59dbnbrfr"
Oct 30 20:03:39.310303 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:39.310231    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac917499-56b1-4aab-b9f0-e256dcf04844-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-969878-59dbnbrfr\" (UID: \"ac917499-56b1-4aab-b9f0-e256dcf04844\") " pod="ai-tenant-e2e-extmod-b-969878/payload-processing-external-model-e2e-extmod-b-969878-59dbnbrfr"
Oct 30 20:03:39.318020 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:39.317982    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/ac917499-56b1-4aab-b9f0-e256dcf04844-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-969878-59dbnbrfr\" (UID: \"ac917499-56b1-4aab-b9f0-e256dcf04844\") " pod="ai-tenant-e2e-extmod-b-969878/payload-processing-external-model-e2e-extmod-b-969878-59dbnbrfr"
Oct 30 20:03:39.323259 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:39.323222    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lw59t\" (UniqueName: \"kubernetes.io/projected/ac917499-56b1-4aab-b9f0-e256dcf04844-kube-api-access-lw59t\") pod \"payload-processing-external-model-e2e-extmod-b-969878-59dbnbrfr\" (UID: \"ac917499-56b1-4aab-b9f0-e256dcf04844\") " pod="ai-tenant-e2e-extmod-b-969878/payload-processing-external-model-e2e-extmod-b-969878-59dbnbrfr"
Oct 30 20:03:39.454251 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:39.452758    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-969878/payload-processing-external-model-e2e-extmod-b-969878-59dbnbrfr"
Oct 30 20:03:39.749546 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:39.749513    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-969878/payload-processing-external-model-e2e-extmod-b-969878-59dbnbrfr"]
Oct 30 20:03:39.750187 ip-10-0-141-42 kubenswrapper[6996]: W1030 20:03:39.750159    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac917499_56b1_4aab_b9f0_e256dcf04844.slice/crio-fa95f8d8fc38487c821b79a60f1aaefcb24311d5e41e22e865fdb20c29f4fe82 WatchSource:0}: Error finding container fa95f8d8fc38487c821b79a60f1aaefcb24311d5e41e22e865fdb20c29f4fe82: Status 404 returned error can't find the container with id fa95f8d8fc38487c821b79a60f1aaefcb24311d5e41e22e865fdb20c29f4fe82
Oct 30 20:03:39.962132 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:39.962094    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-969878/payload-processing-external-model-e2e-extmod-b-969878-59dbnbrfr" event={"ID":"ac917499-56b1-4aab-b9f0-e256dcf04844","Type":"ContainerStarted","Data":"17df7edff71b2e0055709d4d524f76f063e18ee24472babc621c7e30d14c7d0d"}
Oct 30 20:03:39.962332 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:39.962184    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-969878/payload-processing-external-model-e2e-extmod-b-969878-59dbnbrfr" event={"ID":"ac917499-56b1-4aab-b9f0-e256dcf04844","Type":"ContainerStarted","Data":"fa95f8d8fc38487c821b79a60f1aaefcb24311d5e41e22e865fdb20c29f4fe82"}
Oct 30 20:03:39.962332 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:39.962208    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-extmod-b-969878/payload-processing-external-model-e2e-extmod-b-969878-59dbnbrfr"
Oct 30 20:03:40.019388 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:40.019216    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-extmod-b-969878/payload-processing-external-model-e2e-extmod-b-969878-59dbnbrfr" podStartSLOduration=1.019198519 podStartE2EDuration="1.019198519s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:03:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:03:40.018374544 +0000 UTC m=+1895.346492700" watchObservedRunningTime="2026-10-30 20:03:40.019198519 +0000 UTC m=+1895.347316674"
Oct 30 20:03:40.365000 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:40.364965    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr"]
Oct 30 20:03:40.369885 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:40.369862    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr"
Oct 30 20:03:40.373848 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:40.373824    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-b-969878-openshift-default-dockercfg-4jjpz\""
Oct 30 20:03:40.385690 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:40.385666    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr"]
Oct 30 20:03:40.529244 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:40.529203    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ea6d4d1-f271-426c-862e-a2a43ab41f0a-istio-token\") pod \"e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr\" (UID: \"0ea6d4d1-f271-426c-862e-a2a43ab41f0a\") " pod="openshift-ingress/e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr"
Oct 30 20:03:40.529443 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:40.529264    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ea6d4d1-f271-426c-862e-a2a43ab41f0a-istio-envoy\") pod \"e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr\" (UID: \"0ea6d4d1-f271-426c-862e-a2a43ab41f0a\") " pod="openshift-ingress/e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr"
Oct 30 20:03:40.529443 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:40.529389    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ea6d4d1-f271-426c-862e-a2a43ab41f0a-workload-socket\") pod \"e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr\" (UID: \"0ea6d4d1-f271-426c-862e-a2a43ab41f0a\") " pod="openshift-ingress/e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr"
Oct 30 20:03:40.529443 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:40.529427    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hschv\" (UniqueName: \"kubernetes.io/projected/0ea6d4d1-f271-426c-862e-a2a43ab41f0a-kube-api-access-hschv\") pod \"e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr\" (UID: \"0ea6d4d1-f271-426c-862e-a2a43ab41f0a\") " pod="openshift-ingress/e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr"
Oct 30 20:03:40.529642 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:40.529460    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ea6d4d1-f271-426c-862e-a2a43ab41f0a-istio-data\") pod \"e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr\" (UID: \"0ea6d4d1-f271-426c-862e-a2a43ab41f0a\") " pod="openshift-ingress/e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr"
Oct 30 20:03:40.529642 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:40.529593    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ea6d4d1-f271-426c-862e-a2a43ab41f0a-credential-socket\") pod \"e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr\" (UID: \"0ea6d4d1-f271-426c-862e-a2a43ab41f0a\") " pod="openshift-ingress/e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr"
Oct 30 20:03:40.529753 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:40.529690    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ea6d4d1-f271-426c-862e-a2a43ab41f0a-istiod-ca-cert\") pod \"e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr\" (UID: \"0ea6d4d1-f271-426c-862e-a2a43ab41f0a\") " pod="openshift-ingress/e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr"
Oct 30 20:03:40.529753 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:40.529735    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ea6d4d1-f271-426c-862e-a2a43ab41f0a-workload-certs\") pod \"e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr\" (UID: \"0ea6d4d1-f271-426c-862e-a2a43ab41f0a\") " pod="openshift-ingress/e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr"
Oct 30 20:03:40.529895 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:40.529872    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ea6d4d1-f271-426c-862e-a2a43ab41f0a-istio-podinfo\") pod \"e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr\" (UID: \"0ea6d4d1-f271-426c-862e-a2a43ab41f0a\") " pod="openshift-ingress/e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr"
Oct 30 20:03:40.631825 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:40.631728    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ea6d4d1-f271-426c-862e-a2a43ab41f0a-istio-podinfo\") pod \"e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr\" (UID: \"0ea6d4d1-f271-426c-862e-a2a43ab41f0a\") " pod="openshift-ingress/e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr"
Oct 30 20:03:40.632019 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:40.631826    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ea6d4d1-f271-426c-862e-a2a43ab41f0a-istio-token\") pod \"e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr\" (UID: \"0ea6d4d1-f271-426c-862e-a2a43ab41f0a\") " pod="openshift-ingress/e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr"
Oct 30 20:03:40.632019 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:40.631877    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ea6d4d1-f271-426c-862e-a2a43ab41f0a-istio-envoy\") pod \"e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr\" (UID: \"0ea6d4d1-f271-426c-862e-a2a43ab41f0a\") " pod="openshift-ingress/e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr"
Oct 30 20:03:40.632019 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:40.631984    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ea6d4d1-f271-426c-862e-a2a43ab41f0a-workload-socket\") pod \"e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr\" (UID: \"0ea6d4d1-f271-426c-862e-a2a43ab41f0a\") " pod="openshift-ingress/e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr"
Oct 30 20:03:40.632189 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:40.632032    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hschv\" (UniqueName: \"kubernetes.io/projected/0ea6d4d1-f271-426c-862e-a2a43ab41f0a-kube-api-access-hschv\") pod \"e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr\" (UID: \"0ea6d4d1-f271-426c-862e-a2a43ab41f0a\") " pod="openshift-ingress/e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr"
Oct 30 20:03:40.632189 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:40.632079    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ea6d4d1-f271-426c-862e-a2a43ab41f0a-istio-data\") pod \"e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr\" (UID: \"0ea6d4d1-f271-426c-862e-a2a43ab41f0a\") " pod="openshift-ingress/e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr"
Oct 30 20:03:40.632189 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:40.632106    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ea6d4d1-f271-426c-862e-a2a43ab41f0a-credential-socket\") pod \"e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr\" (UID: \"0ea6d4d1-f271-426c-862e-a2a43ab41f0a\") " pod="openshift-ingress/e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr"
Oct 30 20:03:40.632189 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:40.632165    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ea6d4d1-f271-426c-862e-a2a43ab41f0a-istiod-ca-cert\") pod \"e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr\" (UID: \"0ea6d4d1-f271-426c-862e-a2a43ab41f0a\") " pod="openshift-ingress/e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr"
Oct 30 20:03:40.632390 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:40.632213    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ea6d4d1-f271-426c-862e-a2a43ab41f0a-workload-certs\") pod \"e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr\" (UID: \"0ea6d4d1-f271-426c-862e-a2a43ab41f0a\") " pod="openshift-ingress/e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr"
Oct 30 20:03:40.633996 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:40.632748    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ea6d4d1-f271-426c-862e-a2a43ab41f0a-workload-certs\") pod \"e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr\" (UID: \"0ea6d4d1-f271-426c-862e-a2a43ab41f0a\") " pod="openshift-ingress/e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr"
Oct 30 20:03:40.633996 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:40.632764    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ea6d4d1-f271-426c-862e-a2a43ab41f0a-credential-socket\") pod \"e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr\" (UID: \"0ea6d4d1-f271-426c-862e-a2a43ab41f0a\") " pod="openshift-ingress/e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr"
Oct 30 20:03:40.633996 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:40.633089    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ea6d4d1-f271-426c-862e-a2a43ab41f0a-istiod-ca-cert\") pod \"e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr\" (UID: \"0ea6d4d1-f271-426c-862e-a2a43ab41f0a\") " pod="openshift-ingress/e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr"
Oct 30 20:03:40.633996 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:40.633339    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ea6d4d1-f271-426c-862e-a2a43ab41f0a-istio-data\") pod \"e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr\" (UID: \"0ea6d4d1-f271-426c-862e-a2a43ab41f0a\") " pod="openshift-ingress/e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr"
Oct 30 20:03:40.633996 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:40.633358    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ea6d4d1-f271-426c-862e-a2a43ab41f0a-workload-socket\") pod \"e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr\" (UID: \"0ea6d4d1-f271-426c-862e-a2a43ab41f0a\") " pod="openshift-ingress/e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr"
Oct 30 20:03:40.634888 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:40.634864    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ea6d4d1-f271-426c-862e-a2a43ab41f0a-istio-envoy\") pod \"e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr\" (UID: \"0ea6d4d1-f271-426c-862e-a2a43ab41f0a\") " pod="openshift-ingress/e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr"
Oct 30 20:03:40.644631 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:40.644551    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ea6d4d1-f271-426c-862e-a2a43ab41f0a-istio-podinfo\") pod \"e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr\" (UID: \"0ea6d4d1-f271-426c-862e-a2a43ab41f0a\") " pod="openshift-ingress/e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr"
Oct 30 20:03:40.645119 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:40.645091    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ea6d4d1-f271-426c-862e-a2a43ab41f0a-istio-token\") pod \"e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr\" (UID: \"0ea6d4d1-f271-426c-862e-a2a43ab41f0a\") " pod="openshift-ingress/e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr"
Oct 30 20:03:40.645783 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:40.645756    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hschv\" (UniqueName: \"kubernetes.io/projected/0ea6d4d1-f271-426c-862e-a2a43ab41f0a-kube-api-access-hschv\") pod \"e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr\" (UID: \"0ea6d4d1-f271-426c-862e-a2a43ab41f0a\") " pod="openshift-ingress/e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr"
Oct 30 20:03:40.685939 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:40.685906    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr"
Oct 30 20:03:41.003534 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:41.003501    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr"]
Oct 30 20:03:41.004317 ip-10-0-141-42 kubenswrapper[6996]: W1030 20:03:41.004291    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ea6d4d1_f271_426c_862e_a2a43ab41f0a.slice/crio-7bffc3bd6fee06c174cb7801d85954d941fa478ad0dc724a7a702755b91313c8 WatchSource:0}: Error finding container 7bffc3bd6fee06c174cb7801d85954d941fa478ad0dc724a7a702755b91313c8: Status 404 returned error can't find the container with id 7bffc3bd6fee06c174cb7801d85954d941fa478ad0dc724a7a702755b91313c8
Oct 30 20:03:41.007448 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:41.007408    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 20:03:41.007602 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:41.007536    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 20:03:41.007602 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:41.007588    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 20:03:41.979699 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:41.979641    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr" event={"ID":"0ea6d4d1-f271-426c-862e-a2a43ab41f0a","Type":"ContainerStarted","Data":"79c4dc32987f4d6fe67b5f06a5c695b2fd82909a0901712e09bb023fc681e06e"}
Oct 30 20:03:41.980114 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:41.979718    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr" event={"ID":"0ea6d4d1-f271-426c-862e-a2a43ab41f0a","Type":"ContainerStarted","Data":"7bffc3bd6fee06c174cb7801d85954d941fa478ad0dc724a7a702755b91313c8"}
Oct 30 20:03:42.018083 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:42.018027    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr" podStartSLOduration=2.018012049 podStartE2EDuration="2.018012049s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:03:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:03:42.014858519 +0000 UTC m=+1897.342976674" watchObservedRunningTime="2026-10-30 20:03:42.018012049 +0000 UTC m=+1897.346130202"
Oct 30 20:03:42.686577 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:42.686535    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr"
Oct 30 20:03:42.689046 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:42.689014    6996 patch_prober.go:29] interesting pod/e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.160:15021/healthz/ready\": dial tcp 10.134.0.160:15021: connect: connection refused" start-of-body=
Oct 30 20:03:42.689125 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:42.689078    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr" podUID="0ea6d4d1-f271-426c-862e-a2a43ab41f0a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.160:15021/healthz/ready\": dial tcp 10.134.0.160:15021: connect: connection refused"
Oct 30 20:03:43.008571 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:43.003673    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-4d5758-766c8d5467-z6cst"
Oct 30 20:03:43.686840 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:43.686798    6996 patch_prober.go:29] interesting pod/e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.160:15021/healthz/ready\": dial tcp 10.134.0.160:15021: connect: connection refused" start-of-body=
Oct 30 20:03:43.687031 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:43.686856    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr" podUID="0ea6d4d1-f271-426c-862e-a2a43ab41f0a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.160:15021/healthz/ready\": dial tcp 10.134.0.160:15021: connect: connection refused"
Oct 30 20:03:44.686995 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:44.686956    6996 patch_prober.go:29] interesting pod/e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.160:15021/healthz/ready\": dial tcp 10.134.0.160:15021: connect: connection refused" start-of-body=
Oct 30 20:03:44.687440 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:44.687017    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr" podUID="0ea6d4d1-f271-426c-862e-a2a43ab41f0a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.160:15021/healthz/ready\": dial tcp 10.134.0.160:15021: connect: connection refused"
Oct 30 20:03:45.686297 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:45.686260    6996 patch_prober.go:29] interesting pod/e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.160:15021/healthz/ready\": dial tcp 10.134.0.160:15021: connect: connection refused" start-of-body=
Oct 30 20:03:45.686503 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:45.686316    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr" podUID="0ea6d4d1-f271-426c-862e-a2a43ab41f0a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.160:15021/healthz/ready\": dial tcp 10.134.0.160:15021: connect: connection refused"
Oct 30 20:03:46.687143 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:46.687106    6996 patch_prober.go:29] interesting pod/e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.160:15021/healthz/ready\": dial tcp 10.134.0.160:15021: connect: connection refused" start-of-body=
Oct 30 20:03:46.687619 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:46.687162    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr" podUID="0ea6d4d1-f271-426c-862e-a2a43ab41f0a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.160:15021/healthz/ready\": dial tcp 10.134.0.160:15021: connect: connection refused"
Oct 30 20:03:47.738408 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:47.738357    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr"
Oct 30 20:03:47.738996 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:47.738972    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr"
Oct 30 20:03:47.740535 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:47.740444    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr"
Oct 30 20:03:50.973642 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:03:50.973611    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-extmod-b-969878/payload-processing-external-model-e2e-extmod-b-969878-59dbnbrfr"
Oct 30 20:04:01.211589 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:04:01.211536    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 20:04:06.078911 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:04:06.078868    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 20:04:31.293387 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:04:31.293348    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 20:05:01.191678 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:05:01.191639    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 20:05:03.296200 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:05:03.296161    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 20:05:22.287124 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:05:22.287041    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-969878/payload-processing-external-model-e2e-extmod-b-969878-59dbnbrfr"]
Oct 30 20:05:27.287975 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:05:27.287919    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-extmod-b-969878/payload-processing-external-model-e2e-extmod-b-969878-59dbnbrfr" podUID="ac917499-56b1-4aab-b9f0-e256dcf04844" containerName="payload-processing" containerID="cri-o://17df7edff71b2e0055709d4d524f76f063e18ee24472babc621c7e30d14c7d0d" gracePeriod=25
Oct 30 20:05:27.601443 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:05:27.601410    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-969878/payload-processing-external-model-e2e-extmod-b-969878-59dbnbrfr"
Oct 30 20:05:27.736915 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:05:27.736869    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ac917499-56b1-4aab-b9f0-e256dcf04844-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/ac917499-56b1-4aab-b9f0-e256dcf04844-routing-overlay-volume\") pod \"ac917499-56b1-4aab-b9f0-e256dcf04844\" (UID: \"ac917499-56b1-4aab-b9f0-e256dcf04844\") "
Oct 30 20:05:27.737104 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:05:27.736998    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ac917499-56b1-4aab-b9f0-e256dcf04844-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac917499-56b1-4aab-b9f0-e256dcf04844-plugins-config-volume\") pod \"ac917499-56b1-4aab-b9f0-e256dcf04844\" (UID: \"ac917499-56b1-4aab-b9f0-e256dcf04844\") "
Oct 30 20:05:27.737264 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:05:27.737244    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ac917499-56b1-4aab-b9f0-e256dcf04844-kube-api-access-lw59t\" (UniqueName: \"kubernetes.io/projected/ac917499-56b1-4aab-b9f0-e256dcf04844-kube-api-access-lw59t\") pod \"ac917499-56b1-4aab-b9f0-e256dcf04844\" (UID: \"ac917499-56b1-4aab-b9f0-e256dcf04844\") "
Oct 30 20:05:27.737346 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:05:27.737289    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac917499-56b1-4aab-b9f0-e256dcf04844-routing-overlay-volume" pod "ac917499-56b1-4aab-b9f0-e256dcf04844" (UID: "ac917499-56b1-4aab-b9f0-e256dcf04844"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:05:27.737346 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:05:27.737308    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ac917499-56b1-4aab-b9f0-e256dcf04844-provider-credentials\" (UniqueName: \"kubernetes.io/projected/ac917499-56b1-4aab-b9f0-e256dcf04844-provider-credentials\") pod \"ac917499-56b1-4aab-b9f0-e256dcf04844\" (UID: \"ac917499-56b1-4aab-b9f0-e256dcf04844\") "
Oct 30 20:05:27.737459 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:05:27.737384    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac917499-56b1-4aab-b9f0-e256dcf04844-plugins-config-volume" pod "ac917499-56b1-4aab-b9f0-e256dcf04844" (UID: "ac917499-56b1-4aab-b9f0-e256dcf04844"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:05:27.737946 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:05:27.737924    6996 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/ac917499-56b1-4aab-b9f0-e256dcf04844-routing-overlay-volume\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:05:27.738078 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:05:27.737951    6996 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac917499-56b1-4aab-b9f0-e256dcf04844-plugins-config-volume\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:05:27.739556 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:05:27.739530    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac917499-56b1-4aab-b9f0-e256dcf04844-provider-credentials" pod "ac917499-56b1-4aab-b9f0-e256dcf04844" (UID: "ac917499-56b1-4aab-b9f0-e256dcf04844"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:05:27.739715 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:05:27.739651    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac917499-56b1-4aab-b9f0-e256dcf04844-kube-api-access-lw59t" pod "ac917499-56b1-4aab-b9f0-e256dcf04844" (UID: "ac917499-56b1-4aab-b9f0-e256dcf04844"). InnerVolumeSpecName "kube-api-access-lw59t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:05:27.839237 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:05:27.839199    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lw59t\" (UniqueName: \"kubernetes.io/projected/ac917499-56b1-4aab-b9f0-e256dcf04844-kube-api-access-lw59t\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:05:27.839237 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:05:27.839233    6996 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/ac917499-56b1-4aab-b9f0-e256dcf04844-provider-credentials\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:05:28.264207 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:05:28.264117    6996 generic.go:356] "Generic (PLEG): container finished" podID="ac917499-56b1-4aab-b9f0-e256dcf04844" containerID="17df7edff71b2e0055709d4d524f76f063e18ee24472babc621c7e30d14c7d0d" exitCode=0
Oct 30 20:05:28.264207 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:05:28.264193    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-969878/payload-processing-external-model-e2e-extmod-b-969878-59dbnbrfr"
Oct 30 20:05:28.264444 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:05:28.264204    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-969878/payload-processing-external-model-e2e-extmod-b-969878-59dbnbrfr" event={"ID":"ac917499-56b1-4aab-b9f0-e256dcf04844","Type":"ContainerDied","Data":"17df7edff71b2e0055709d4d524f76f063e18ee24472babc621c7e30d14c7d0d"}
Oct 30 20:05:28.264444 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:05:28.264288    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-969878/payload-processing-external-model-e2e-extmod-b-969878-59dbnbrfr" event={"ID":"ac917499-56b1-4aab-b9f0-e256dcf04844","Type":"ContainerDied","Data":"fa95f8d8fc38487c821b79a60f1aaefcb24311d5e41e22e865fdb20c29f4fe82"}
Oct 30 20:05:28.264444 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:05:28.264311    6996 scope.go:122] "RemoveContainer" containerID="17df7edff71b2e0055709d4d524f76f063e18ee24472babc621c7e30d14c7d0d"
Oct 30 20:05:28.277463 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:05:28.277439    6996 scope.go:122] "RemoveContainer" containerID="17df7edff71b2e0055709d4d524f76f063e18ee24472babc621c7e30d14c7d0d"
Oct 30 20:05:28.277784 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:05:28.277761    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17df7edff71b2e0055709d4d524f76f063e18ee24472babc621c7e30d14c7d0d\": container with ID starting with 17df7edff71b2e0055709d4d524f76f063e18ee24472babc621c7e30d14c7d0d not found: ID does not exist" containerID="17df7edff71b2e0055709d4d524f76f063e18ee24472babc621c7e30d14c7d0d"
Oct 30 20:05:28.277848 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:05:28.277792    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17df7edff71b2e0055709d4d524f76f063e18ee24472babc621c7e30d14c7d0d"} err="failed to get container status \"17df7edff71b2e0055709d4d524f76f063e18ee24472babc621c7e30d14c7d0d\": rpc error: code = NotFound desc = could not find container \"17df7edff71b2e0055709d4d524f76f063e18ee24472babc621c7e30d14c7d0d\": container with ID starting with 17df7edff71b2e0055709d4d524f76f063e18ee24472babc621c7e30d14c7d0d not found: ID does not exist"
Oct 30 20:05:28.289171 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:05:28.289147    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-969878/payload-processing-external-model-e2e-extmod-b-969878-59dbnbrfr"]
Oct 30 20:05:28.294076 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:05:28.294051    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-extmod-b-969878/payload-processing-external-model-e2e-extmod-b-969878-59dbnbrfr"]
Oct 30 20:05:28.792967 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:05:28.792933    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ac917499-56b1-4aab-b9f0-e256dcf04844" path="/var/lib/kubelet/pods/ac917499-56b1-4aab-b9f0-e256dcf04844/volumes"
Oct 30 20:05:31.270419 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:05:31.270374    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 20:05:39.855167 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:05:39.855131    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-5f9b3c-6c5f497c6d-sfvdl"]
Oct 30 20:05:39.856125 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:05:39.856101    6996 cpu_manager.go:414] "containerMap: removing container" podUID="ac917499-56b1-4aab-b9f0-e256dcf04844" containerName="payload-processing"
Oct 30 20:05:39.856242 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:05:39.856126    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ac917499-56b1-4aab-b9f0-e256dcf04844" containerName="payload-processing"
Oct 30 20:05:39.856346 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:05:39.856332    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="ac917499-56b1-4aab-b9f0-e256dcf04844" containerName="payload-processing"
Oct 30 20:05:39.861778 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:05:39.861756    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-5f9b3c-6c5f497c6d-sfvdl"
Oct 30 20:05:39.874871 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:05:39.874843    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-b-5f9b3c-serving-cert\""
Oct 30 20:05:39.881362 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:05:39.881329    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-5f9b3c-6c5f497c6d-sfvdl"]
Oct 30 20:05:40.019303 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:05:40.019236    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91b87b65-f883-485d-abc4-69f270f6b3e9-maas-api-tls\") pod \"maas-api-e2e-extmod-b-5f9b3c-6c5f497c6d-sfvdl\" (UID: \"91b87b65-f883-485d-abc4-69f270f6b3e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-5f9b3c-6c5f497c6d-sfvdl"
Oct 30 20:05:40.019791 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:05:40.019752    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cvn4\" (UniqueName: \"kubernetes.io/projected/91b87b65-f883-485d-abc4-69f270f6b3e9-kube-api-access-4cvn4\") pod \"maas-api-e2e-extmod-b-5f9b3c-6c5f497c6d-sfvdl\" (UID: \"91b87b65-f883-485d-abc4-69f270f6b3e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-5f9b3c-6c5f497c6d-sfvdl"
Oct 30 20:05:40.019913 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:05:40.019856    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91b87b65-f883-485d-abc4-69f270f6b3e9-metrics-certs\") pod \"maas-api-e2e-extmod-b-5f9b3c-6c5f497c6d-sfvdl\" (UID: \"91b87b65-f883-485d-abc4-69f270f6b3e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-5f9b3c-6c5f497c6d-sfvdl"
Oct 30 20:05:40.020052 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:05:40.020032    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91b87b65-f883-485d-abc4-69f270f6b3e9-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-5f9b3c-6c5f497c6d-sfvdl\" (UID: \"91b87b65-f883-485d-abc4-69f270f6b3e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-5f9b3c-6c5f497c6d-sfvdl"
Oct 30 20:05:40.121542 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:05:40.121454    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91b87b65-f883-485d-abc4-69f270f6b3e9-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-5f9b3c-6c5f497c6d-sfvdl\" (UID: \"91b87b65-f883-485d-abc4-69f270f6b3e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-5f9b3c-6c5f497c6d-sfvdl"
Oct 30 20:05:40.121724 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:05:40.121568    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91b87b65-f883-485d-abc4-69f270f6b3e9-maas-api-tls\") pod \"maas-api-e2e-extmod-b-5f9b3c-6c5f497c6d-sfvdl\" (UID: \"91b87b65-f883-485d-abc4-69f270f6b3e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-5f9b3c-6c5f497c6d-sfvdl"
Oct 30 20:05:40.121724 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:05:40.121659    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cvn4\" (UniqueName: \"kubernetes.io/projected/91b87b65-f883-485d-abc4-69f270f6b3e9-kube-api-access-4cvn4\") pod \"maas-api-e2e-extmod-b-5f9b3c-6c5f497c6d-sfvdl\" (UID: \"91b87b65-f883-485d-abc4-69f270f6b3e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-5f9b3c-6c5f497c6d-sfvdl"
Oct 30 20:05:40.121724 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:05:40.121707    6996 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-extmod-b-5f9b3c-serving-cert: secret "maas-api-e2e-extmod-b-5f9b3c-serving-cert" not found
Oct 30 20:05:40.121904 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:05:40.121710    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91b87b65-f883-485d-abc4-69f270f6b3e9-metrics-certs\") pod \"maas-api-e2e-extmod-b-5f9b3c-6c5f497c6d-sfvdl\" (UID: \"91b87b65-f883-485d-abc4-69f270f6b3e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-5f9b3c-6c5f497c6d-sfvdl"
Oct 30 20:05:40.121904 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:05:40.121788    6996 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91b87b65-f883-485d-abc4-69f270f6b3e9-maas-api-tls podName:91b87b65-f883-485d-abc4-69f270f6b3e9 nodeName:}" failed. No retries permitted until 2026-10-30 20:05:40.621766633 +0000 UTC m=+2015.949884785 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/91b87b65-f883-485d-abc4-69f270f6b3e9-maas-api-tls") pod "maas-api-e2e-extmod-b-5f9b3c-6c5f497c6d-sfvdl" (UID: "91b87b65-f883-485d-abc4-69f270f6b3e9") : secret "maas-api-e2e-extmod-b-5f9b3c-serving-cert" not found
Oct 30 20:05:40.122127 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:05:40.122101    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91b87b65-f883-485d-abc4-69f270f6b3e9-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-5f9b3c-6c5f497c6d-sfvdl\" (UID: \"91b87b65-f883-485d-abc4-69f270f6b3e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-5f9b3c-6c5f497c6d-sfvdl"
Oct 30 20:05:40.125679 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:05:40.125655    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91b87b65-f883-485d-abc4-69f270f6b3e9-metrics-certs\") pod \"maas-api-e2e-extmod-b-5f9b3c-6c5f497c6d-sfvdl\" (UID: \"91b87b65-f883-485d-abc4-69f270f6b3e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-5f9b3c-6c5f497c6d-sfvdl"
Oct 30 20:05:40.134628 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:05:40.134601    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cvn4\" (UniqueName: \"kubernetes.io/projected/91b87b65-f883-485d-abc4-69f270f6b3e9-kube-api-access-4cvn4\") pod \"maas-api-e2e-extmod-b-5f9b3c-6c5f497c6d-sfvdl\" (UID: \"91b87b65-f883-485d-abc4-69f270f6b3e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-5f9b3c-6c5f497c6d-sfvdl"
Oct 30 20:05:40.630305 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:05:40.630272    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91b87b65-f883-485d-abc4-69f270f6b3e9-maas-api-tls\") pod \"maas-api-e2e-extmod-b-5f9b3c-6c5f497c6d-sfvdl\" (UID: \"91b87b65-f883-485d-abc4-69f270f6b3e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-5f9b3c-6c5f497c6d-sfvdl"
Oct 30 20:05:40.632958 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:05:40.632935    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91b87b65-f883-485d-abc4-69f270f6b3e9-maas-api-tls\") pod \"maas-api-e2e-extmod-b-5f9b3c-6c5f497c6d-sfvdl\" (UID: \"91b87b65-f883-485d-abc4-69f270f6b3e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-5f9b3c-6c5f497c6d-sfvdl"
Oct 30 20:05:40.795609 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:05:40.795579    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-5f9b3c-6c5f497c6d-sfvdl"
Oct 30 20:05:41.094043 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:05:41.094017    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-5f9b3c-6c5f497c6d-sfvdl"]
Oct 30 20:05:41.095087 ip-10-0-141-42 kubenswrapper[6996]: W1030 20:05:41.095059    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91b87b65_f883_485d_abc4_69f270f6b3e9.slice/crio-717cdc4269002bae7271a3dc6adc7920acae7d3f170162cfb37388c1f89933bc WatchSource:0}: Error finding container 717cdc4269002bae7271a3dc6adc7920acae7d3f170162cfb37388c1f89933bc: Status 404 returned error can't find the container with id 717cdc4269002bae7271a3dc6adc7920acae7d3f170162cfb37388c1f89933bc
Oct 30 20:05:41.405602 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:05:41.405514    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-5f9b3c-6c5f497c6d-sfvdl" event={"ID":"91b87b65-f883-485d-abc4-69f270f6b3e9","Type":"ContainerStarted","Data":"717cdc4269002bae7271a3dc6adc7920acae7d3f170162cfb37388c1f89933bc"}
Oct 30 20:05:42.414044 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:05:42.414002    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-5f9b3c-6c5f497c6d-sfvdl" event={"ID":"91b87b65-f883-485d-abc4-69f270f6b3e9","Type":"ContainerStarted","Data":"bb01fb7c2693485285e671c3d9f424617798bc739a2107922641fd9b74bca4ca"}
Oct 30 20:05:42.414507 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:05:42.414076    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-5f9b3c-6c5f497c6d-sfvdl"
Oct 30 20:05:42.437448 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:05:42.437399    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-5f9b3c-6c5f497c6d-sfvdl" podStartSLOduration=3.009406196 podStartE2EDuration="3.437384944s" totalImagesPullingTime="427.978748ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:05:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:05:42.436167579 +0000 UTC m=+2017.764285734" watchObservedRunningTime="2026-10-30 20:05:42.437384944 +0000 UTC m=+2017.765503139"
Oct 30 20:05:48.427720 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:05:48.427690    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-5f9b3c-6c5f497c6d-sfvdl"
Oct 30 20:06:00.361417 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:00.361371    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 20:06:01.317079 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:06:01.317029    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 20:06:31.191700 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:06:31.191664    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 20:06:31.204583 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:31.204552    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md"]
Oct 30 20:06:31.209295 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:31.209277    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md"
Oct 30 20:06:31.214394 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:31.214369    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-b-d97cc8-openshift-default-dockercfg-2fhzd\""
Oct 30 20:06:31.225275 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:31.225248    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md"]
Oct 30 20:06:31.304760 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:31.304714    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06a16fac-9e8d-4724-a949-a8d31f13002d-credential-socket\") pod \"e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md\" (UID: \"06a16fac-9e8d-4724-a949-a8d31f13002d\") " pod="openshift-ingress/e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md"
Oct 30 20:06:31.305108 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:31.305079    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06a16fac-9e8d-4724-a949-a8d31f13002d-istio-data\") pod \"e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md\" (UID: \"06a16fac-9e8d-4724-a949-a8d31f13002d\") " pod="openshift-ingress/e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md"
Oct 30 20:06:31.305272 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:31.305254    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06a16fac-9e8d-4724-a949-a8d31f13002d-istio-podinfo\") pod \"e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md\" (UID: \"06a16fac-9e8d-4724-a949-a8d31f13002d\") " pod="openshift-ingress/e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md"
Oct 30 20:06:31.305415 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:31.305395    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06a16fac-9e8d-4724-a949-a8d31f13002d-workload-certs\") pod \"e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md\" (UID: \"06a16fac-9e8d-4724-a949-a8d31f13002d\") " pod="openshift-ingress/e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md"
Oct 30 20:06:31.305595 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:31.305575    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06a16fac-9e8d-4724-a949-a8d31f13002d-istio-envoy\") pod \"e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md\" (UID: \"06a16fac-9e8d-4724-a949-a8d31f13002d\") " pod="openshift-ingress/e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md"
Oct 30 20:06:31.305839 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:31.305816    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06a16fac-9e8d-4724-a949-a8d31f13002d-istio-token\") pod \"e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md\" (UID: \"06a16fac-9e8d-4724-a949-a8d31f13002d\") " pod="openshift-ingress/e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md"
Oct 30 20:06:31.306104 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:31.306083    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jtjc\" (UniqueName: \"kubernetes.io/projected/06a16fac-9e8d-4724-a949-a8d31f13002d-kube-api-access-8jtjc\") pod \"e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md\" (UID: \"06a16fac-9e8d-4724-a949-a8d31f13002d\") " pod="openshift-ingress/e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md"
Oct 30 20:06:31.306255 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:31.306231    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06a16fac-9e8d-4724-a949-a8d31f13002d-workload-socket\") pod \"e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md\" (UID: \"06a16fac-9e8d-4724-a949-a8d31f13002d\") " pod="openshift-ingress/e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md"
Oct 30 20:06:31.306361 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:31.306281    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06a16fac-9e8d-4724-a949-a8d31f13002d-istiod-ca-cert\") pod \"e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md\" (UID: \"06a16fac-9e8d-4724-a949-a8d31f13002d\") " pod="openshift-ingress/e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md"
Oct 30 20:06:31.407432 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:31.407397    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06a16fac-9e8d-4724-a949-a8d31f13002d-istiod-ca-cert\") pod \"e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md\" (UID: \"06a16fac-9e8d-4724-a949-a8d31f13002d\") " pod="openshift-ingress/e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md"
Oct 30 20:06:31.407637 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:31.407539    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06a16fac-9e8d-4724-a949-a8d31f13002d-credential-socket\") pod \"e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md\" (UID: \"06a16fac-9e8d-4724-a949-a8d31f13002d\") " pod="openshift-ingress/e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md"
Oct 30 20:06:31.407637 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:31.407616    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06a16fac-9e8d-4724-a949-a8d31f13002d-istio-data\") pod \"e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md\" (UID: \"06a16fac-9e8d-4724-a949-a8d31f13002d\") " pod="openshift-ingress/e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md"
Oct 30 20:06:31.407769 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:31.407656    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06a16fac-9e8d-4724-a949-a8d31f13002d-istio-podinfo\") pod \"e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md\" (UID: \"06a16fac-9e8d-4724-a949-a8d31f13002d\") " pod="openshift-ingress/e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md"
Oct 30 20:06:31.407769 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:31.407706    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06a16fac-9e8d-4724-a949-a8d31f13002d-workload-certs\") pod \"e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md\" (UID: \"06a16fac-9e8d-4724-a949-a8d31f13002d\") " pod="openshift-ingress/e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md"
Oct 30 20:06:31.408033 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:31.408006    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06a16fac-9e8d-4724-a949-a8d31f13002d-credential-socket\") pod \"e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md\" (UID: \"06a16fac-9e8d-4724-a949-a8d31f13002d\") " pod="openshift-ingress/e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md"
Oct 30 20:06:31.408128 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:31.408066    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06a16fac-9e8d-4724-a949-a8d31f13002d-istio-envoy\") pod \"e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md\" (UID: \"06a16fac-9e8d-4724-a949-a8d31f13002d\") " pod="openshift-ingress/e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md"
Oct 30 20:06:31.408128 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:31.408092    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06a16fac-9e8d-4724-a949-a8d31f13002d-workload-certs\") pod \"e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md\" (UID: \"06a16fac-9e8d-4724-a949-a8d31f13002d\") " pod="openshift-ingress/e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md"
Oct 30 20:06:31.408247 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:31.408168    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06a16fac-9e8d-4724-a949-a8d31f13002d-istio-token\") pod \"e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md\" (UID: \"06a16fac-9e8d-4724-a949-a8d31f13002d\") " pod="openshift-ingress/e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md"
Oct 30 20:06:31.408335 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:31.408298    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06a16fac-9e8d-4724-a949-a8d31f13002d-istiod-ca-cert\") pod \"e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md\" (UID: \"06a16fac-9e8d-4724-a949-a8d31f13002d\") " pod="openshift-ingress/e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md"
Oct 30 20:06:31.408464 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:31.408344    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jtjc\" (UniqueName: \"kubernetes.io/projected/06a16fac-9e8d-4724-a949-a8d31f13002d-kube-api-access-8jtjc\") pod \"e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md\" (UID: \"06a16fac-9e8d-4724-a949-a8d31f13002d\") " pod="openshift-ingress/e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md"
Oct 30 20:06:31.408464 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:31.408393    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06a16fac-9e8d-4724-a949-a8d31f13002d-workload-socket\") pod \"e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md\" (UID: \"06a16fac-9e8d-4724-a949-a8d31f13002d\") " pod="openshift-ingress/e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md"
Oct 30 20:06:31.408464 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:31.408429    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06a16fac-9e8d-4724-a949-a8d31f13002d-istio-data\") pod \"e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md\" (UID: \"06a16fac-9e8d-4724-a949-a8d31f13002d\") " pod="openshift-ingress/e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md"
Oct 30 20:06:31.408828 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:31.408797    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06a16fac-9e8d-4724-a949-a8d31f13002d-workload-socket\") pod \"e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md\" (UID: \"06a16fac-9e8d-4724-a949-a8d31f13002d\") " pod="openshift-ingress/e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md"
Oct 30 20:06:31.410293 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:31.410271    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06a16fac-9e8d-4724-a949-a8d31f13002d-istio-envoy\") pod \"e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md\" (UID: \"06a16fac-9e8d-4724-a949-a8d31f13002d\") " pod="openshift-ingress/e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md"
Oct 30 20:06:31.410744 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:31.410720    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06a16fac-9e8d-4724-a949-a8d31f13002d-istio-podinfo\") pod \"e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md\" (UID: \"06a16fac-9e8d-4724-a949-a8d31f13002d\") " pod="openshift-ingress/e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md"
Oct 30 20:06:31.440107 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:31.440084    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06a16fac-9e8d-4724-a949-a8d31f13002d-istio-token\") pod \"e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md\" (UID: \"06a16fac-9e8d-4724-a949-a8d31f13002d\") " pod="openshift-ingress/e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md"
Oct 30 20:06:31.445991 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:31.445933    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jtjc\" (UniqueName: \"kubernetes.io/projected/06a16fac-9e8d-4724-a949-a8d31f13002d-kube-api-access-8jtjc\") pod \"e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md\" (UID: \"06a16fac-9e8d-4724-a949-a8d31f13002d\") " pod="openshift-ingress/e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md"
Oct 30 20:06:31.526512 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:31.526461    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md"
Oct 30 20:06:31.821636 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:31.821607    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md"]
Oct 30 20:06:31.823193 ip-10-0-141-42 kubenswrapper[6996]: W1030 20:06:31.823168    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06a16fac_9e8d_4724_a949_a8d31f13002d.slice/crio-3650f0279d486971dcae4e2663bc0e45fbfc7408b5835126d1b953d35238711f WatchSource:0}: Error finding container 3650f0279d486971dcae4e2663bc0e45fbfc7408b5835126d1b953d35238711f: Status 404 returned error can't find the container with id 3650f0279d486971dcae4e2663bc0e45fbfc7408b5835126d1b953d35238711f
Oct 30 20:06:31.825835 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:31.825800    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 20:06:31.825928 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:31.825868    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 20:06:31.825928 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:31.825905    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 20:06:31.990324 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:31.990287    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md" event={"ID":"06a16fac-9e8d-4724-a949-a8d31f13002d","Type":"ContainerStarted","Data":"59bb9556f598b6daedb4c97d1431d4962734088763306faace05e43981e642dd"}
Oct 30 20:06:31.990519 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:31.990349    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md" event={"ID":"06a16fac-9e8d-4724-a949-a8d31f13002d","Type":"ContainerStarted","Data":"3650f0279d486971dcae4e2663bc0e45fbfc7408b5835126d1b953d35238711f"}
Oct 30 20:06:32.015376 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:32.015315    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md" podStartSLOduration=1.015296233 podStartE2EDuration="1.015296233s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:06:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:06:32.01351999 +0000 UTC m=+2067.341638135" watchObservedRunningTime="2026-10-30 20:06:32.015296233 +0000 UTC m=+2067.343414390"
Oct 30 20:06:32.527670 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:32.527634    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md"
Oct 30 20:06:32.528804 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:32.528779    6996 patch_prober.go:29] interesting pod/e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.162:15021/healthz/ready\": dial tcp 10.134.0.162:15021: connect: connection refused" start-of-body=
Oct 30 20:06:32.528878 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:32.528828    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md" podUID="06a16fac-9e8d-4724-a949-a8d31f13002d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.162:15021/healthz/ready\": dial tcp 10.134.0.162:15021: connect: connection refused"
Oct 30 20:06:33.527831 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:33.527795    6996 patch_prober.go:29] interesting pod/e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.162:15021/healthz/ready\": dial tcp 10.134.0.162:15021: connect: connection refused" start-of-body=
Oct 30 20:06:33.528303 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:33.527848    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md" podUID="06a16fac-9e8d-4724-a949-a8d31f13002d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.162:15021/healthz/ready\": dial tcp 10.134.0.162:15021: connect: connection refused"
Oct 30 20:06:34.526805 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:34.526767    6996 patch_prober.go:29] interesting pod/e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.162:15021/healthz/ready\": dial tcp 10.134.0.162:15021: connect: connection refused" start-of-body=
Oct 30 20:06:34.526986 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:34.526828    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md" podUID="06a16fac-9e8d-4724-a949-a8d31f13002d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.162:15021/healthz/ready\": dial tcp 10.134.0.162:15021: connect: connection refused"
Oct 30 20:06:35.527436 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:35.527399    6996 patch_prober.go:29] interesting pod/e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.162:15021/healthz/ready\": dial tcp 10.134.0.162:15021: connect: connection refused" start-of-body=
Oct 30 20:06:35.527837 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:35.527456    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md" podUID="06a16fac-9e8d-4724-a949-a8d31f13002d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.162:15021/healthz/ready\": dial tcp 10.134.0.162:15021: connect: connection refused"
Oct 30 20:06:36.527038 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:36.526999    6996 patch_prober.go:29] interesting pod/e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.162:15021/healthz/ready\": dial tcp 10.134.0.162:15021: connect: connection refused" start-of-body=
Oct 30 20:06:36.527242 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:36.527062    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md" podUID="06a16fac-9e8d-4724-a949-a8d31f13002d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.162:15021/healthz/ready\": dial tcp 10.134.0.162:15021: connect: connection refused"
Oct 30 20:06:37.581261 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:37.578935    6996 patch_prober.go:29] interesting pod/e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.162:15021/healthz/ready\": dial tcp 10.134.0.162:15021: connect: connection refused" start-of-body=
Oct 30 20:06:37.581261 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:37.578992    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md" podUID="06a16fac-9e8d-4724-a949-a8d31f13002d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.162:15021/healthz/ready\": dial tcp 10.134.0.162:15021: connect: connection refused"
Oct 30 20:06:38.565799 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:38.565761    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md"
Oct 30 20:06:38.566255 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:38.566232    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md"
Oct 30 20:06:38.567104 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:38.567078    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md"
Oct 30 20:06:40.311958 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:40.311921    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw"]
Oct 30 20:06:40.312397 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:40.312122    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw" podUID="3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f" containerName="istio-proxy" containerID="cri-o://ed9b12658f56200202df9922ffeb7c897e533122721cc3f7d9ca7b6ee503e8cc" gracePeriod=30
Oct 30 20:06:45.401029 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:45.400992    6996 patch_prober.go:29] interesting pod/e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body=
Oct 30 20:06:45.401403 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:45.401064    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw" podUID="3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused"
Oct 30 20:06:45.605782 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:45.605716    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw"
Oct 30 20:06:45.644443 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:45.644411    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f-credential-socket\") pod \"3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f\" (UID: \"3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f\") "
Oct 30 20:06:45.644731 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:45.644545    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f-istio-podinfo\") pod \"3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f\" (UID: \"3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f\") "
Oct 30 20:06:45.644731 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:45.644578    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f-istio-envoy\") pod \"3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f\" (UID: \"3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f\") "
Oct 30 20:06:45.644731 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:45.644641    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f-istiod-ca-cert\") pod \"3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f\" (UID: \"3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f\") "
Oct 30 20:06:45.644731 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:45.644731    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f-workload-certs\") pod \"3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f\" (UID: \"3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f\") "
Oct 30 20:06:45.645030 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:45.644774    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f-credential-socket" pod "3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f" (UID: "3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:06:45.645030 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:45.644948    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f-kube-api-access-gzds4\" (UniqueName: \"kubernetes.io/projected/3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f-kube-api-access-gzds4\") pod \"3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f\" (UID: \"3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f\") "
Oct 30 20:06:45.647155 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:45.645318    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f-istiod-ca-cert" pod "3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f" (UID: "3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:06:45.647155 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:45.645597    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f-istio-token\" (UniqueName: \"kubernetes.io/projected/3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f-istio-token\") pod \"3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f\" (UID: \"3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f\") "
Oct 30 20:06:45.647155 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:45.645746    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f-istio-data\") pod \"3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f\" (UID: \"3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f\") "
Oct 30 20:06:45.647155 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:45.645841    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f-workload-socket\") pod \"3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f\" (UID: \"3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f\") "
Oct 30 20:06:45.647155 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:45.646056    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f-workload-certs" pod "3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f" (UID: "3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:06:45.647155 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:45.646263    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f-workload-socket" pod "3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f" (UID: "3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:06:45.647155 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:45.646842    6996 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f-workload-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:06:45.647155 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:45.646867    6996 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f-workload-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:06:45.647155 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:45.646882    6996 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f-credential-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:06:45.647155 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:45.646905    6996 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f-istiod-ca-cert\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:06:45.647155 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:45.646973    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f-istio-data" pod "3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f" (UID: "3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:06:45.647897 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:45.647585    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f-istio-envoy" pod "3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f" (UID: "3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:06:45.649747 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:45.649717    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f-kube-api-access-gzds4" pod "3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f" (UID: "3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f"). InnerVolumeSpecName "kube-api-access-gzds4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:06:45.650108 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:45.650086    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f-istio-token" pod "3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f" (UID: "3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:06:45.665342 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:45.665310    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f-istio-podinfo" pod "3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f" (UID: "3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 20:06:45.750147 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:45.749811    6996 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f-istio-podinfo\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:06:45.750147 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:45.749849    6996 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f-istio-envoy\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:06:45.750147 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:45.749864    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gzds4\" (UniqueName: \"kubernetes.io/projected/3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f-kube-api-access-gzds4\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:06:45.750147 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:45.749880    6996 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f-istio-token\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:06:45.750147 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:45.749897    6996 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f-istio-data\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:06:46.102167 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:46.102133    6996 generic.go:356] "Generic (PLEG): container finished" podID="3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f" containerID="ed9b12658f56200202df9922ffeb7c897e533122721cc3f7d9ca7b6ee503e8cc" exitCode=0
Oct 30 20:06:46.102337 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:46.102176    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw"
Oct 30 20:06:46.102337 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:46.102216    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw" event={"ID":"3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f","Type":"ContainerDied","Data":"ed9b12658f56200202df9922ffeb7c897e533122721cc3f7d9ca7b6ee503e8cc"}
Oct 30 20:06:46.102337 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:46.102273    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw" event={"ID":"3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f","Type":"ContainerDied","Data":"3240e6d1c695c8d25b00ec0269e1f17622b8048fe254028f9282ee2543ed9c86"}
Oct 30 20:06:46.102337 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:46.102297    6996 scope.go:122] "RemoveContainer" containerID="ed9b12658f56200202df9922ffeb7c897e533122721cc3f7d9ca7b6ee503e8cc"
Oct 30 20:06:46.113516 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:46.113496    6996 scope.go:122] "RemoveContainer" containerID="ed9b12658f56200202df9922ffeb7c897e533122721cc3f7d9ca7b6ee503e8cc"
Oct 30 20:06:46.113753 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:06:46.113737    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed9b12658f56200202df9922ffeb7c897e533122721cc3f7d9ca7b6ee503e8cc\": container with ID starting with ed9b12658f56200202df9922ffeb7c897e533122721cc3f7d9ca7b6ee503e8cc not found: ID does not exist" containerID="ed9b12658f56200202df9922ffeb7c897e533122721cc3f7d9ca7b6ee503e8cc"
Oct 30 20:06:46.113792 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:46.113760    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed9b12658f56200202df9922ffeb7c897e533122721cc3f7d9ca7b6ee503e8cc"} err="failed to get container status \"ed9b12658f56200202df9922ffeb7c897e533122721cc3f7d9ca7b6ee503e8cc\": rpc error: code = NotFound desc = could not find container \"ed9b12658f56200202df9922ffeb7c897e533122721cc3f7d9ca7b6ee503e8cc\": container with ID starting with ed9b12658f56200202df9922ffeb7c897e533122721cc3f7d9ca7b6ee503e8cc not found: ID does not exist"
Oct 30 20:06:46.141155 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:46.141127    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw"]
Oct 30 20:06:46.154872 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:46.154843    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-qtslw"]
Oct 30 20:06:46.794637 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:46.794596    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f" path="/var/lib/kubelet/pods/3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f/volumes"
Oct 30 20:06:50.541727 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:50.541686    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-extmod-b-969878/payload-processing-external-model-e2e-extmod-b-969878-59dbh9hjs"]
Oct 30 20:06:50.543731 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:50.543699    6996 cpu_manager.go:414] "containerMap: removing container" podUID="3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f" containerName="istio-proxy"
Oct 30 20:06:50.543893 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:50.543878    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f" containerName="istio-proxy"
Oct 30 20:06:50.544359 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:50.544180    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="3e2461ca-03d1-49f8-9c16-a97d1bcc4b4f" containerName="istio-proxy"
Oct 30 20:06:50.551875 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:50.551855    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-969878/payload-processing-external-model-e2e-extmod-b-969878-59dbh9hjs"
Oct 30 20:06:50.560176 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:50.560151    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-969878\"/\"kube-root-ca.crt\""
Oct 30 20:06:50.560586 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:50.560556    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-969878\"/\"agc-cred-5fdacea3cbca\""
Oct 30 20:06:50.562135 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:50.562106    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-969878\"/\"openshift-service-ca.crt\""
Oct 30 20:06:50.562868 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:50.562190    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-969878\"/\"routing-overlay\""
Oct 30 20:06:50.562868 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:50.562227    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-969878\"/\"payload-processing-external-model-plugins-e2e-extmod-b-969878\""
Oct 30 20:06:50.564322 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:50.564295    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-969878/payload-processing-external-model-e2e-extmod-b-969878-59dbh9hjs"]
Oct 30 20:06:50.567752 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:50.566728    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-969878\"/\"payload-processing-external-model-e2e--5b5eb86f-dockercfg-tpqsc\""
Oct 30 20:06:50.578866 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:50.578832    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/3d743539-78fc-4adc-9cf0-6d83b1da1032-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-969878-59dbh9hjs\" (UID: \"3d743539-78fc-4adc-9cf0-6d83b1da1032\") " pod="ai-tenant-e2e-extmod-b-969878/payload-processing-external-model-e2e-extmod-b-969878-59dbh9hjs"
Oct 30 20:06:50.578994 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:50.578960    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3d743539-78fc-4adc-9cf0-6d83b1da1032-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-969878-59dbh9hjs\" (UID: \"3d743539-78fc-4adc-9cf0-6d83b1da1032\") " pod="ai-tenant-e2e-extmod-b-969878/payload-processing-external-model-e2e-extmod-b-969878-59dbh9hjs"
Oct 30 20:06:50.579081 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:50.579052    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7smbg\" (UniqueName: \"kubernetes.io/projected/3d743539-78fc-4adc-9cf0-6d83b1da1032-kube-api-access-7smbg\") pod \"payload-processing-external-model-e2e-extmod-b-969878-59dbh9hjs\" (UID: \"3d743539-78fc-4adc-9cf0-6d83b1da1032\") " pod="ai-tenant-e2e-extmod-b-969878/payload-processing-external-model-e2e-extmod-b-969878-59dbh9hjs"
Oct 30 20:06:50.579143 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:50.579105    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/3d743539-78fc-4adc-9cf0-6d83b1da1032-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-969878-59dbh9hjs\" (UID: \"3d743539-78fc-4adc-9cf0-6d83b1da1032\") " pod="ai-tenant-e2e-extmod-b-969878/payload-processing-external-model-e2e-extmod-b-969878-59dbh9hjs"
Oct 30 20:06:50.680572 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:50.680540    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7smbg\" (UniqueName: \"kubernetes.io/projected/3d743539-78fc-4adc-9cf0-6d83b1da1032-kube-api-access-7smbg\") pod \"payload-processing-external-model-e2e-extmod-b-969878-59dbh9hjs\" (UID: \"3d743539-78fc-4adc-9cf0-6d83b1da1032\") " pod="ai-tenant-e2e-extmod-b-969878/payload-processing-external-model-e2e-extmod-b-969878-59dbh9hjs"
Oct 30 20:06:50.680738 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:50.680596    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/3d743539-78fc-4adc-9cf0-6d83b1da1032-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-969878-59dbh9hjs\" (UID: \"3d743539-78fc-4adc-9cf0-6d83b1da1032\") " pod="ai-tenant-e2e-extmod-b-969878/payload-processing-external-model-e2e-extmod-b-969878-59dbh9hjs"
Oct 30 20:06:50.680786 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:50.680735    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/3d743539-78fc-4adc-9cf0-6d83b1da1032-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-969878-59dbh9hjs\" (UID: \"3d743539-78fc-4adc-9cf0-6d83b1da1032\") " pod="ai-tenant-e2e-extmod-b-969878/payload-processing-external-model-e2e-extmod-b-969878-59dbh9hjs"
Oct 30 20:06:50.680836 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:50.680827    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3d743539-78fc-4adc-9cf0-6d83b1da1032-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-969878-59dbh9hjs\" (UID: \"3d743539-78fc-4adc-9cf0-6d83b1da1032\") " pod="ai-tenant-e2e-extmod-b-969878/payload-processing-external-model-e2e-extmod-b-969878-59dbh9hjs"
Oct 30 20:06:50.681303 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:50.681276    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/3d743539-78fc-4adc-9cf0-6d83b1da1032-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-969878-59dbh9hjs\" (UID: \"3d743539-78fc-4adc-9cf0-6d83b1da1032\") " pod="ai-tenant-e2e-extmod-b-969878/payload-processing-external-model-e2e-extmod-b-969878-59dbh9hjs"
Oct 30 20:06:50.681568 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:50.681545    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3d743539-78fc-4adc-9cf0-6d83b1da1032-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-969878-59dbh9hjs\" (UID: \"3d743539-78fc-4adc-9cf0-6d83b1da1032\") " pod="ai-tenant-e2e-extmod-b-969878/payload-processing-external-model-e2e-extmod-b-969878-59dbh9hjs"
Oct 30 20:06:50.683196 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:50.683174    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/3d743539-78fc-4adc-9cf0-6d83b1da1032-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-969878-59dbh9hjs\" (UID: \"3d743539-78fc-4adc-9cf0-6d83b1da1032\") " pod="ai-tenant-e2e-extmod-b-969878/payload-processing-external-model-e2e-extmod-b-969878-59dbh9hjs"
Oct 30 20:06:50.701620 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:50.701596    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7smbg\" (UniqueName: \"kubernetes.io/projected/3d743539-78fc-4adc-9cf0-6d83b1da1032-kube-api-access-7smbg\") pod \"payload-processing-external-model-e2e-extmod-b-969878-59dbh9hjs\" (UID: \"3d743539-78fc-4adc-9cf0-6d83b1da1032\") " pod="ai-tenant-e2e-extmod-b-969878/payload-processing-external-model-e2e-extmod-b-969878-59dbh9hjs"
Oct 30 20:06:50.886921 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:50.886873    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-969878/payload-processing-external-model-e2e-extmod-b-969878-59dbh9hjs"
Oct 30 20:06:51.223099 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:51.223063    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-969878/payload-processing-external-model-e2e-extmod-b-969878-59dbh9hjs"]
Oct 30 20:06:51.223768 ip-10-0-141-42 kubenswrapper[6996]: W1030 20:06:51.223740    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d743539_78fc_4adc_9cf0_6d83b1da1032.slice/crio-a59988f403838e1914c2203c56461ab968eb96e23330bc84307245aab69bbf6d WatchSource:0}: Error finding container a59988f403838e1914c2203c56461ab968eb96e23330bc84307245aab69bbf6d: Status 404 returned error can't find the container with id a59988f403838e1914c2203c56461ab968eb96e23330bc84307245aab69bbf6d
Oct 30 20:06:52.150274 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:52.150236    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-969878/payload-processing-external-model-e2e-extmod-b-969878-59dbh9hjs" event={"ID":"3d743539-78fc-4adc-9cf0-6d83b1da1032","Type":"ContainerStarted","Data":"fc3155ed61940e11cf2d711d9829bcf711484b1e76c83253a8e77e17dc398631"}
Oct 30 20:06:52.150719 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:52.150298    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-969878/payload-processing-external-model-e2e-extmod-b-969878-59dbh9hjs" event={"ID":"3d743539-78fc-4adc-9cf0-6d83b1da1032","Type":"ContainerStarted","Data":"a59988f403838e1914c2203c56461ab968eb96e23330bc84307245aab69bbf6d"}
Oct 30 20:06:52.150719 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:52.150414    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-extmod-b-969878/payload-processing-external-model-e2e-extmod-b-969878-59dbh9hjs"
Oct 30 20:06:52.184925 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:52.184876    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-extmod-b-969878/payload-processing-external-model-e2e-extmod-b-969878-59dbh9hjs" podStartSLOduration=2.184862381 podStartE2EDuration="2.184862381s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:06:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:06:52.182036123 +0000 UTC m=+2087.510154276" watchObservedRunningTime="2026-10-30 20:06:52.184862381 +0000 UTC m=+2087.512980533"
Oct 30 20:06:59.802683 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:06:59.802640    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 20:07:01.226656 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:07:01.226621    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 20:07:03.160406 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:03.160375    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-extmod-b-969878/payload-processing-external-model-e2e-extmod-b-969878-59dbh9hjs"
Oct 30 20:07:19.681012 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:19.680974    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v"]
Oct 30 20:07:19.682047 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:19.681987    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v" podUID="b99ab818-3be9-406e-be18-c0ed26c43bea" containerName="istio-proxy" containerID="cri-o://0e38bd02fc5f548539f1c3a269077b42a10645acd3329bd78a14d359aaebe30f" gracePeriod=30
Oct 30 20:07:23.176292 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:23.175852    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-969878/payload-processing-external-model-e2e-extmod-b-969878-59dbh9hjs"]
Oct 30 20:07:24.147591 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:24.147541    6996 patch_prober.go:29] interesting pod/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 20:07:24.147787 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:24.147618    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v" podUID="b99ab818-3be9-406e-be18-c0ed26c43bea" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 20:07:25.165933 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:25.165905    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v"
Oct 30 20:07:25.209717 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:25.209669    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b99ab818-3be9-406e-be18-c0ed26c43bea-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b99ab818-3be9-406e-be18-c0ed26c43bea-istio-podinfo\") pod \"b99ab818-3be9-406e-be18-c0ed26c43bea\" (UID: \"b99ab818-3be9-406e-be18-c0ed26c43bea\") "
Oct 30 20:07:25.209908 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:25.209732    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b99ab818-3be9-406e-be18-c0ed26c43bea-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b99ab818-3be9-406e-be18-c0ed26c43bea-workload-socket\") pod \"b99ab818-3be9-406e-be18-c0ed26c43bea\" (UID: \"b99ab818-3be9-406e-be18-c0ed26c43bea\") "
Oct 30 20:07:25.209908 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:25.209819    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b99ab818-3be9-406e-be18-c0ed26c43bea-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b99ab818-3be9-406e-be18-c0ed26c43bea-workload-certs\") pod \"b99ab818-3be9-406e-be18-c0ed26c43bea\" (UID: \"b99ab818-3be9-406e-be18-c0ed26c43bea\") "
Oct 30 20:07:25.209908 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:25.209850    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b99ab818-3be9-406e-be18-c0ed26c43bea-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b99ab818-3be9-406e-be18-c0ed26c43bea-istio-data\") pod \"b99ab818-3be9-406e-be18-c0ed26c43bea\" (UID: \"b99ab818-3be9-406e-be18-c0ed26c43bea\") "
Oct 30 20:07:25.210169 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:25.210018    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b99ab818-3be9-406e-be18-c0ed26c43bea-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b99ab818-3be9-406e-be18-c0ed26c43bea-istiod-ca-cert\") pod \"b99ab818-3be9-406e-be18-c0ed26c43bea\" (UID: \"b99ab818-3be9-406e-be18-c0ed26c43bea\") "
Oct 30 20:07:25.210169 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:25.210055    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b99ab818-3be9-406e-be18-c0ed26c43bea-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b99ab818-3be9-406e-be18-c0ed26c43bea-istio-envoy\") pod \"b99ab818-3be9-406e-be18-c0ed26c43bea\" (UID: \"b99ab818-3be9-406e-be18-c0ed26c43bea\") "
Oct 30 20:07:25.210169 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:25.210087    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b99ab818-3be9-406e-be18-c0ed26c43bea-kube-api-access-lr5rb\" (UniqueName: \"kubernetes.io/projected/b99ab818-3be9-406e-be18-c0ed26c43bea-kube-api-access-lr5rb\") pod \"b99ab818-3be9-406e-be18-c0ed26c43bea\" (UID: \"b99ab818-3be9-406e-be18-c0ed26c43bea\") "
Oct 30 20:07:25.210169 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:25.210106    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b99ab818-3be9-406e-be18-c0ed26c43bea-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b99ab818-3be9-406e-be18-c0ed26c43bea-credential-socket\") pod \"b99ab818-3be9-406e-be18-c0ed26c43bea\" (UID: \"b99ab818-3be9-406e-be18-c0ed26c43bea\") "
Oct 30 20:07:25.210347 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:25.210179    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b99ab818-3be9-406e-be18-c0ed26c43bea-istio-token\" (UniqueName: \"kubernetes.io/projected/b99ab818-3be9-406e-be18-c0ed26c43bea-istio-token\") pod \"b99ab818-3be9-406e-be18-c0ed26c43bea\" (UID: \"b99ab818-3be9-406e-be18-c0ed26c43bea\") "
Oct 30 20:07:25.210842 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:25.210806    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b99ab818-3be9-406e-be18-c0ed26c43bea-workload-socket" pod "b99ab818-3be9-406e-be18-c0ed26c43bea" (UID: "b99ab818-3be9-406e-be18-c0ed26c43bea"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:07:25.211437 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:25.211183    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b99ab818-3be9-406e-be18-c0ed26c43bea-workload-certs" pod "b99ab818-3be9-406e-be18-c0ed26c43bea" (UID: "b99ab818-3be9-406e-be18-c0ed26c43bea"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:07:25.211661 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:25.211643    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b99ab818-3be9-406e-be18-c0ed26c43bea-credential-socket" pod "b99ab818-3be9-406e-be18-c0ed26c43bea" (UID: "b99ab818-3be9-406e-be18-c0ed26c43bea"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:07:25.211921 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:25.211876    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b99ab818-3be9-406e-be18-c0ed26c43bea-istiod-ca-cert" pod "b99ab818-3be9-406e-be18-c0ed26c43bea" (UID: "b99ab818-3be9-406e-be18-c0ed26c43bea"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:07:25.212008 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:25.211934    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b99ab818-3be9-406e-be18-c0ed26c43bea-istio-data" pod "b99ab818-3be9-406e-be18-c0ed26c43bea" (UID: "b99ab818-3be9-406e-be18-c0ed26c43bea"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:07:25.217871 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:25.217844    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b99ab818-3be9-406e-be18-c0ed26c43bea-istio-envoy" pod "b99ab818-3be9-406e-be18-c0ed26c43bea" (UID: "b99ab818-3be9-406e-be18-c0ed26c43bea"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:07:25.222646 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:25.222616    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b99ab818-3be9-406e-be18-c0ed26c43bea-istio-podinfo" pod "b99ab818-3be9-406e-be18-c0ed26c43bea" (UID: "b99ab818-3be9-406e-be18-c0ed26c43bea"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 20:07:25.222747 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:25.222640    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b99ab818-3be9-406e-be18-c0ed26c43bea-kube-api-access-lr5rb" pod "b99ab818-3be9-406e-be18-c0ed26c43bea" (UID: "b99ab818-3be9-406e-be18-c0ed26c43bea"). InnerVolumeSpecName "kube-api-access-lr5rb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:07:25.222747 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:25.222680    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b99ab818-3be9-406e-be18-c0ed26c43bea-istio-token" pod "b99ab818-3be9-406e-be18-c0ed26c43bea" (UID: "b99ab818-3be9-406e-be18-c0ed26c43bea"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:07:25.312984 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:25.312945    6996 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b99ab818-3be9-406e-be18-c0ed26c43bea-istio-token\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:07:25.312984 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:25.312981    6996 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b99ab818-3be9-406e-be18-c0ed26c43bea-istio-podinfo\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:07:25.312984 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:25.312991    6996 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b99ab818-3be9-406e-be18-c0ed26c43bea-workload-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:07:25.313252 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:25.313000    6996 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b99ab818-3be9-406e-be18-c0ed26c43bea-workload-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:07:25.313252 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:25.313009    6996 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b99ab818-3be9-406e-be18-c0ed26c43bea-istio-data\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:07:25.313252 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:25.313017    6996 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b99ab818-3be9-406e-be18-c0ed26c43bea-istiod-ca-cert\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:07:25.313252 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:25.313025    6996 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b99ab818-3be9-406e-be18-c0ed26c43bea-istio-envoy\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:07:25.313252 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:25.313034    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lr5rb\" (UniqueName: \"kubernetes.io/projected/b99ab818-3be9-406e-be18-c0ed26c43bea-kube-api-access-lr5rb\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:07:25.313252 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:25.313045    6996 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b99ab818-3be9-406e-be18-c0ed26c43bea-credential-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:07:25.498282 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:25.497185    6996 generic.go:356] "Generic (PLEG): container finished" podID="b99ab818-3be9-406e-be18-c0ed26c43bea" containerID="0e38bd02fc5f548539f1c3a269077b42a10645acd3329bd78a14d359aaebe30f" exitCode=0
Oct 30 20:07:25.498282 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:25.497401    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v" event={"ID":"b99ab818-3be9-406e-be18-c0ed26c43bea","Type":"ContainerDied","Data":"0e38bd02fc5f548539f1c3a269077b42a10645acd3329bd78a14d359aaebe30f"}
Oct 30 20:07:25.498282 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:25.497456    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v" event={"ID":"b99ab818-3be9-406e-be18-c0ed26c43bea","Type":"ContainerDied","Data":"98879e1b59c007b8a0972675b2be82eb6dedeea506170f15d35fb69b53b1f298"}
Oct 30 20:07:25.498282 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:25.497508    6996 scope.go:122] "RemoveContainer" containerID="0e38bd02fc5f548539f1c3a269077b42a10645acd3329bd78a14d359aaebe30f"
Oct 30 20:07:25.498282 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:25.497690    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v"
Oct 30 20:07:25.515826 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:25.515800    6996 scope.go:122] "RemoveContainer" containerID="0e38bd02fc5f548539f1c3a269077b42a10645acd3329bd78a14d359aaebe30f"
Oct 30 20:07:25.516184 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:07:25.516162    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e38bd02fc5f548539f1c3a269077b42a10645acd3329bd78a14d359aaebe30f\": container with ID starting with 0e38bd02fc5f548539f1c3a269077b42a10645acd3329bd78a14d359aaebe30f not found: ID does not exist" containerID="0e38bd02fc5f548539f1c3a269077b42a10645acd3329bd78a14d359aaebe30f"
Oct 30 20:07:25.516255 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:25.516194    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e38bd02fc5f548539f1c3a269077b42a10645acd3329bd78a14d359aaebe30f"} err="failed to get container status \"0e38bd02fc5f548539f1c3a269077b42a10645acd3329bd78a14d359aaebe30f\": rpc error: code = NotFound desc = could not find container \"0e38bd02fc5f548539f1c3a269077b42a10645acd3329bd78a14d359aaebe30f\": container with ID starting with 0e38bd02fc5f548539f1c3a269077b42a10645acd3329bd78a14d359aaebe30f not found: ID does not exist"
Oct 30 20:07:25.534214 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:25.534170    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v"]
Oct 30 20:07:25.545694 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:25.545662    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hm4v"]
Oct 30 20:07:26.793661 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:26.793625    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b99ab818-3be9-406e-be18-c0ed26c43bea" path="/var/lib/kubelet/pods/b99ab818-3be9-406e-be18-c0ed26c43bea/volumes"
Oct 30 20:07:28.178078 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:28.178032    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-extmod-b-969878/payload-processing-external-model-e2e-extmod-b-969878-59dbh9hjs" podUID="3d743539-78fc-4adc-9cf0-6d83b1da1032" containerName="payload-processing" containerID="cri-o://fc3155ed61940e11cf2d711d9829bcf711484b1e76c83253a8e77e17dc398631" gracePeriod=25
Oct 30 20:07:28.517706 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:28.517675    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-969878/payload-processing-external-model-e2e-extmod-b-969878-59dbh9hjs"
Oct 30 20:07:28.528681 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:28.528646    6996 generic.go:356] "Generic (PLEG): container finished" podID="3d743539-78fc-4adc-9cf0-6d83b1da1032" containerID="fc3155ed61940e11cf2d711d9829bcf711484b1e76c83253a8e77e17dc398631" exitCode=0
Oct 30 20:07:28.528863 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:28.528690    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-969878/payload-processing-external-model-e2e-extmod-b-969878-59dbh9hjs"
Oct 30 20:07:28.528863 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:28.528727    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-969878/payload-processing-external-model-e2e-extmod-b-969878-59dbh9hjs" event={"ID":"3d743539-78fc-4adc-9cf0-6d83b1da1032","Type":"ContainerDied","Data":"fc3155ed61940e11cf2d711d9829bcf711484b1e76c83253a8e77e17dc398631"}
Oct 30 20:07:28.528863 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:28.528787    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-969878/payload-processing-external-model-e2e-extmod-b-969878-59dbh9hjs" event={"ID":"3d743539-78fc-4adc-9cf0-6d83b1da1032","Type":"ContainerDied","Data":"a59988f403838e1914c2203c56461ab968eb96e23330bc84307245aab69bbf6d"}
Oct 30 20:07:28.528863 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:28.528815    6996 scope.go:122] "RemoveContainer" containerID="fc3155ed61940e11cf2d711d9829bcf711484b1e76c83253a8e77e17dc398631"
Oct 30 20:07:28.539066 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:28.539049    6996 scope.go:122] "RemoveContainer" containerID="fc3155ed61940e11cf2d711d9829bcf711484b1e76c83253a8e77e17dc398631"
Oct 30 20:07:28.539371 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:07:28.539347    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc3155ed61940e11cf2d711d9829bcf711484b1e76c83253a8e77e17dc398631\": container with ID starting with fc3155ed61940e11cf2d711d9829bcf711484b1e76c83253a8e77e17dc398631 not found: ID does not exist" containerID="fc3155ed61940e11cf2d711d9829bcf711484b1e76c83253a8e77e17dc398631"
Oct 30 20:07:28.539435 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:28.539382    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc3155ed61940e11cf2d711d9829bcf711484b1e76c83253a8e77e17dc398631"} err="failed to get container status \"fc3155ed61940e11cf2d711d9829bcf711484b1e76c83253a8e77e17dc398631\": rpc error: code = NotFound desc = could not find container \"fc3155ed61940e11cf2d711d9829bcf711484b1e76c83253a8e77e17dc398631\": container with ID starting with fc3155ed61940e11cf2d711d9829bcf711484b1e76c83253a8e77e17dc398631 not found: ID does not exist"
Oct 30 20:07:28.556729 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:28.556691    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3d743539-78fc-4adc-9cf0-6d83b1da1032-kube-api-access-7smbg\" (UniqueName: \"kubernetes.io/projected/3d743539-78fc-4adc-9cf0-6d83b1da1032-kube-api-access-7smbg\") pod \"3d743539-78fc-4adc-9cf0-6d83b1da1032\" (UID: \"3d743539-78fc-4adc-9cf0-6d83b1da1032\") "
Oct 30 20:07:28.556969 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:28.556943    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3d743539-78fc-4adc-9cf0-6d83b1da1032-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/3d743539-78fc-4adc-9cf0-6d83b1da1032-routing-overlay-volume\") pod \"3d743539-78fc-4adc-9cf0-6d83b1da1032\" (UID: \"3d743539-78fc-4adc-9cf0-6d83b1da1032\") "
Oct 30 20:07:28.557114 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:28.557084    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3d743539-78fc-4adc-9cf0-6d83b1da1032-provider-credentials\" (UniqueName: \"kubernetes.io/projected/3d743539-78fc-4adc-9cf0-6d83b1da1032-provider-credentials\") pod \"3d743539-78fc-4adc-9cf0-6d83b1da1032\" (UID: \"3d743539-78fc-4adc-9cf0-6d83b1da1032\") "
Oct 30 20:07:28.557188 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:28.557164    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3d743539-78fc-4adc-9cf0-6d83b1da1032-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3d743539-78fc-4adc-9cf0-6d83b1da1032-plugins-config-volume\") pod \"3d743539-78fc-4adc-9cf0-6d83b1da1032\" (UID: \"3d743539-78fc-4adc-9cf0-6d83b1da1032\") "
Oct 30 20:07:28.557534 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:28.557470    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d743539-78fc-4adc-9cf0-6d83b1da1032-routing-overlay-volume" pod "3d743539-78fc-4adc-9cf0-6d83b1da1032" (UID: "3d743539-78fc-4adc-9cf0-6d83b1da1032"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:07:28.557788 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:28.557760    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d743539-78fc-4adc-9cf0-6d83b1da1032-plugins-config-volume" pod "3d743539-78fc-4adc-9cf0-6d83b1da1032" (UID: "3d743539-78fc-4adc-9cf0-6d83b1da1032"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:07:28.558079 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:28.558057    6996 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/3d743539-78fc-4adc-9cf0-6d83b1da1032-routing-overlay-volume\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:07:28.558188 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:28.558104    6996 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3d743539-78fc-4adc-9cf0-6d83b1da1032-plugins-config-volume\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:07:28.559201 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:28.559174    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d743539-78fc-4adc-9cf0-6d83b1da1032-kube-api-access-7smbg" pod "3d743539-78fc-4adc-9cf0-6d83b1da1032" (UID: "3d743539-78fc-4adc-9cf0-6d83b1da1032"). InnerVolumeSpecName "kube-api-access-7smbg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:07:28.559667 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:28.559643    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d743539-78fc-4adc-9cf0-6d83b1da1032-provider-credentials" pod "3d743539-78fc-4adc-9cf0-6d83b1da1032" (UID: "3d743539-78fc-4adc-9cf0-6d83b1da1032"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:07:28.659777 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:28.659737    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7smbg\" (UniqueName: \"kubernetes.io/projected/3d743539-78fc-4adc-9cf0-6d83b1da1032-kube-api-access-7smbg\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:07:28.659777 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:28.659769    6996 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/3d743539-78fc-4adc-9cf0-6d83b1da1032-provider-credentials\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:07:28.850438 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:28.850402    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-969878/payload-processing-external-model-e2e-extmod-b-969878-59dbh9hjs"]
Oct 30 20:07:28.855543 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:28.855514    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-extmod-b-969878/payload-processing-external-model-e2e-extmod-b-969878-59dbh9hjs"]
Oct 30 20:07:30.798358 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:30.798319    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3d743539-78fc-4adc-9cf0-6d83b1da1032" path="/var/lib/kubelet/pods/3d743539-78fc-4adc-9cf0-6d83b1da1032/volumes"
Oct 30 20:07:31.207076 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:07:31.207033    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 20:07:40.313626 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:07:40.313560    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 20:08:01.194245 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:08:01.194205    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 20:08:20.869025 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:20.868823    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-5jtlw"]
Oct 30 20:08:20.869552 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:20.869113    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-5jtlw" podUID="abe1772a-c94e-4dd9-b023-176bab5081d3" containerName="istio-proxy" containerID="cri-o://8e7ebb1ebab993527d3113ca0c69ae01fa3bd47da507481ea9012243fdc43dc8" gracePeriod=30
Oct 30 20:08:26.054163 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:26.054095    6996 generic.go:356] "Generic (PLEG): container finished" podID="abe1772a-c94e-4dd9-b023-176bab5081d3" containerID="8e7ebb1ebab993527d3113ca0c69ae01fa3bd47da507481ea9012243fdc43dc8" exitCode=0
Oct 30 20:08:26.054603 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:26.054454    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-5jtlw" event={"ID":"abe1772a-c94e-4dd9-b023-176bab5081d3","Type":"ContainerDied","Data":"8e7ebb1ebab993527d3113ca0c69ae01fa3bd47da507481ea9012243fdc43dc8"}
Oct 30 20:08:26.178592 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:26.178564    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-5jtlw"
Oct 30 20:08:26.233737 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:26.233699    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/abe1772a-c94e-4dd9-b023-176bab5081d3-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/abe1772a-c94e-4dd9-b023-176bab5081d3-istio-data\") pod \"abe1772a-c94e-4dd9-b023-176bab5081d3\" (UID: \"abe1772a-c94e-4dd9-b023-176bab5081d3\") "
Oct 30 20:08:26.233917 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:26.233808    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/abe1772a-c94e-4dd9-b023-176bab5081d3-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/abe1772a-c94e-4dd9-b023-176bab5081d3-istio-envoy\") pod \"abe1772a-c94e-4dd9-b023-176bab5081d3\" (UID: \"abe1772a-c94e-4dd9-b023-176bab5081d3\") "
Oct 30 20:08:26.233993 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:26.233943    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/abe1772a-c94e-4dd9-b023-176bab5081d3-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/abe1772a-c94e-4dd9-b023-176bab5081d3-workload-certs\") pod \"abe1772a-c94e-4dd9-b023-176bab5081d3\" (UID: \"abe1772a-c94e-4dd9-b023-176bab5081d3\") "
Oct 30 20:08:26.233993 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:26.233962    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abe1772a-c94e-4dd9-b023-176bab5081d3-istio-data" pod "abe1772a-c94e-4dd9-b023-176bab5081d3" (UID: "abe1772a-c94e-4dd9-b023-176bab5081d3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:08:26.234106 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:26.233995    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/abe1772a-c94e-4dd9-b023-176bab5081d3-kube-api-access-7z7tc\" (UniqueName: \"kubernetes.io/projected/abe1772a-c94e-4dd9-b023-176bab5081d3-kube-api-access-7z7tc\") pod \"abe1772a-c94e-4dd9-b023-176bab5081d3\" (UID: \"abe1772a-c94e-4dd9-b023-176bab5081d3\") "
Oct 30 20:08:26.234106 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:26.234099    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/abe1772a-c94e-4dd9-b023-176bab5081d3-istio-token\" (UniqueName: \"kubernetes.io/projected/abe1772a-c94e-4dd9-b023-176bab5081d3-istio-token\") pod \"abe1772a-c94e-4dd9-b023-176bab5081d3\" (UID: \"abe1772a-c94e-4dd9-b023-176bab5081d3\") "
Oct 30 20:08:26.234218 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:26.234177    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/abe1772a-c94e-4dd9-b023-176bab5081d3-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/abe1772a-c94e-4dd9-b023-176bab5081d3-workload-socket\") pod \"abe1772a-c94e-4dd9-b023-176bab5081d3\" (UID: \"abe1772a-c94e-4dd9-b023-176bab5081d3\") "
Oct 30 20:08:26.234277 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:26.234220    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/abe1772a-c94e-4dd9-b023-176bab5081d3-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/abe1772a-c94e-4dd9-b023-176bab5081d3-istiod-ca-cert\") pod \"abe1772a-c94e-4dd9-b023-176bab5081d3\" (UID: \"abe1772a-c94e-4dd9-b023-176bab5081d3\") "
Oct 30 20:08:26.234277 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:26.234228    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abe1772a-c94e-4dd9-b023-176bab5081d3-workload-certs" pod "abe1772a-c94e-4dd9-b023-176bab5081d3" (UID: "abe1772a-c94e-4dd9-b023-176bab5081d3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:08:26.234277 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:26.234262    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/abe1772a-c94e-4dd9-b023-176bab5081d3-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/abe1772a-c94e-4dd9-b023-176bab5081d3-credential-socket\") pod \"abe1772a-c94e-4dd9-b023-176bab5081d3\" (UID: \"abe1772a-c94e-4dd9-b023-176bab5081d3\") "
Oct 30 20:08:26.234445 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:26.234392    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/abe1772a-c94e-4dd9-b023-176bab5081d3-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/abe1772a-c94e-4dd9-b023-176bab5081d3-istio-podinfo\") pod \"abe1772a-c94e-4dd9-b023-176bab5081d3\" (UID: \"abe1772a-c94e-4dd9-b023-176bab5081d3\") "
Oct 30 20:08:26.234445 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:26.234433    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abe1772a-c94e-4dd9-b023-176bab5081d3-workload-socket" pod "abe1772a-c94e-4dd9-b023-176bab5081d3" (UID: "abe1772a-c94e-4dd9-b023-176bab5081d3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:08:26.234629 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:26.234610    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abe1772a-c94e-4dd9-b023-176bab5081d3-credential-socket" pod "abe1772a-c94e-4dd9-b023-176bab5081d3" (UID: "abe1772a-c94e-4dd9-b023-176bab5081d3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:08:26.234955 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:26.234930    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/abe1772a-c94e-4dd9-b023-176bab5081d3-istiod-ca-cert" pod "abe1772a-c94e-4dd9-b023-176bab5081d3" (UID: "abe1772a-c94e-4dd9-b023-176bab5081d3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:08:26.235145 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:26.235119    6996 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/abe1772a-c94e-4dd9-b023-176bab5081d3-istio-data\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:08:26.235145 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:26.235143    6996 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/abe1772a-c94e-4dd9-b023-176bab5081d3-workload-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:08:26.235341 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:26.235158    6996 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/abe1772a-c94e-4dd9-b023-176bab5081d3-workload-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:08:26.235341 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:26.235173    6996 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/abe1772a-c94e-4dd9-b023-176bab5081d3-istiod-ca-cert\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:08:26.235341 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:26.235186    6996 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/abe1772a-c94e-4dd9-b023-176bab5081d3-credential-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:08:26.237573 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:26.236965    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abe1772a-c94e-4dd9-b023-176bab5081d3-istio-token" pod "abe1772a-c94e-4dd9-b023-176bab5081d3" (UID: "abe1772a-c94e-4dd9-b023-176bab5081d3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:08:26.237573 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:26.237188    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abe1772a-c94e-4dd9-b023-176bab5081d3-kube-api-access-7z7tc" pod "abe1772a-c94e-4dd9-b023-176bab5081d3" (UID: "abe1772a-c94e-4dd9-b023-176bab5081d3"). InnerVolumeSpecName "kube-api-access-7z7tc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:08:26.237751 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:26.237600    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/abe1772a-c94e-4dd9-b023-176bab5081d3-istio-podinfo" pod "abe1772a-c94e-4dd9-b023-176bab5081d3" (UID: "abe1772a-c94e-4dd9-b023-176bab5081d3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 20:08:26.240873 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:26.240843    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abe1772a-c94e-4dd9-b023-176bab5081d3-istio-envoy" pod "abe1772a-c94e-4dd9-b023-176bab5081d3" (UID: "abe1772a-c94e-4dd9-b023-176bab5081d3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:08:26.337081 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:26.337048    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7z7tc\" (UniqueName: \"kubernetes.io/projected/abe1772a-c94e-4dd9-b023-176bab5081d3-kube-api-access-7z7tc\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:08:26.337081 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:26.337078    6996 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/abe1772a-c94e-4dd9-b023-176bab5081d3-istio-token\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:08:26.337081 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:26.337090    6996 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/abe1772a-c94e-4dd9-b023-176bab5081d3-istio-podinfo\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:08:26.337329 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:26.337100    6996 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/abe1772a-c94e-4dd9-b023-176bab5081d3-istio-envoy\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:08:27.064701 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:27.064620    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-5jtlw" event={"ID":"abe1772a-c94e-4dd9-b023-176bab5081d3","Type":"ContainerDied","Data":"b07634d1a46807bdfd512009ed98fb60ef229b99b3c45e41f431633566d11d49"}
Oct 30 20:08:27.064701 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:27.064646    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-5jtlw"
Oct 30 20:08:27.065182 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:27.064712    6996 scope.go:122] "RemoveContainer" containerID="8e7ebb1ebab993527d3113ca0c69ae01fa3bd47da507481ea9012243fdc43dc8"
Oct 30 20:08:27.095591 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:27.095560    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-5jtlw"]
Oct 30 20:08:27.102228 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:27.102199    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-5jtlw"]
Oct 30 20:08:28.818385 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:28.818342    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="abe1772a-c94e-4dd9-b023-176bab5081d3" path="/var/lib/kubelet/pods/abe1772a-c94e-4dd9-b023-176bab5081d3/volumes"
Oct 30 20:08:30.491024 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:30.490988    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 20:08:31.193813 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:08:31.193770    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 20:08:33.010255 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:33.010212    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-extmod-a-68685b/payload-processing-external-model-e2e-extmod-a-68685b-8484pz9cs"]
Oct 30 20:08:33.012766 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:33.010975    6996 cpu_manager.go:414] "containerMap: removing container" podUID="b99ab818-3be9-406e-be18-c0ed26c43bea" containerName="istio-proxy"
Oct 30 20:08:33.012766 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:33.010991    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b99ab818-3be9-406e-be18-c0ed26c43bea" containerName="istio-proxy"
Oct 30 20:08:33.012766 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:33.011015    6996 cpu_manager.go:414] "containerMap: removing container" podUID="3d743539-78fc-4adc-9cf0-6d83b1da1032" containerName="payload-processing"
Oct 30 20:08:33.012766 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:33.011022    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3d743539-78fc-4adc-9cf0-6d83b1da1032" containerName="payload-processing"
Oct 30 20:08:33.012766 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:33.011048    6996 cpu_manager.go:414] "containerMap: removing container" podUID="abe1772a-c94e-4dd9-b023-176bab5081d3" containerName="istio-proxy"
Oct 30 20:08:33.012766 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:33.011054    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="abe1772a-c94e-4dd9-b023-176bab5081d3" containerName="istio-proxy"
Oct 30 20:08:33.012766 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:33.011145    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="3d743539-78fc-4adc-9cf0-6d83b1da1032" containerName="payload-processing"
Oct 30 20:08:33.012766 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:33.011156    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="abe1772a-c94e-4dd9-b023-176bab5081d3" containerName="istio-proxy"
Oct 30 20:08:33.012766 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:33.011167    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="b99ab818-3be9-406e-be18-c0ed26c43bea" containerName="istio-proxy"
Oct 30 20:08:33.014787 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:33.014765    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-68685b/payload-processing-external-model-e2e-extmod-a-68685b-8484pz9cs"
Oct 30 20:08:33.019276 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:33.019253    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-a-68685b\"/\"e2e-cfl-19316d-provider-cred\""
Oct 30 20:08:33.019688 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:33.019664    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-a-68685b\"/\"kube-root-ca.crt\""
Oct 30 20:08:33.019805 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:33.019782    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-a-68685b\"/\"routing-overlay\""
Oct 30 20:08:33.022373 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:33.022348    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-a-68685b\"/\"openshift-service-ca.crt\""
Oct 30 20:08:33.022517 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:33.022458    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-a-68685b\"/\"payload-processing-external-model-plugins-e2e-extmod-a-68685b\""
Oct 30 20:08:33.022641 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:33.022566    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-a-68685b\"/\"payload-processing-external-model-e2e--babb2686-dockercfg-tc6p4\""
Oct 30 20:08:33.046579 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:33.046548    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-a-68685b/payload-processing-external-model-e2e-extmod-a-68685b-8484pz9cs"]
Oct 30 20:08:33.157503 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:33.152983    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a04c80cf-438f-4b49-933b-bf281cef07ad-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-a-68685b-8484pz9cs\" (UID: \"a04c80cf-438f-4b49-933b-bf281cef07ad\") " pod="ai-tenant-e2e-extmod-a-68685b/payload-processing-external-model-e2e-extmod-a-68685b-8484pz9cs"
Oct 30 20:08:33.157503 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:33.153030    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/a04c80cf-438f-4b49-933b-bf281cef07ad-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-a-68685b-8484pz9cs\" (UID: \"a04c80cf-438f-4b49-933b-bf281cef07ad\") " pod="ai-tenant-e2e-extmod-a-68685b/payload-processing-external-model-e2e-extmod-a-68685b-8484pz9cs"
Oct 30 20:08:33.157503 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:33.153121    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/a04c80cf-438f-4b49-933b-bf281cef07ad-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-a-68685b-8484pz9cs\" (UID: \"a04c80cf-438f-4b49-933b-bf281cef07ad\") " pod="ai-tenant-e2e-extmod-a-68685b/payload-processing-external-model-e2e-extmod-a-68685b-8484pz9cs"
Oct 30 20:08:33.157503 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:33.153345    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7f4r\" (UniqueName: \"kubernetes.io/projected/a04c80cf-438f-4b49-933b-bf281cef07ad-kube-api-access-f7f4r\") pod \"payload-processing-external-model-e2e-extmod-a-68685b-8484pz9cs\" (UID: \"a04c80cf-438f-4b49-933b-bf281cef07ad\") " pod="ai-tenant-e2e-extmod-a-68685b/payload-processing-external-model-e2e-extmod-a-68685b-8484pz9cs"
Oct 30 20:08:33.254352 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:33.254321    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7f4r\" (UniqueName: \"kubernetes.io/projected/a04c80cf-438f-4b49-933b-bf281cef07ad-kube-api-access-f7f4r\") pod \"payload-processing-external-model-e2e-extmod-a-68685b-8484pz9cs\" (UID: \"a04c80cf-438f-4b49-933b-bf281cef07ad\") " pod="ai-tenant-e2e-extmod-a-68685b/payload-processing-external-model-e2e-extmod-a-68685b-8484pz9cs"
Oct 30 20:08:33.254559 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:33.254431    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a04c80cf-438f-4b49-933b-bf281cef07ad-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-a-68685b-8484pz9cs\" (UID: \"a04c80cf-438f-4b49-933b-bf281cef07ad\") " pod="ai-tenant-e2e-extmod-a-68685b/payload-processing-external-model-e2e-extmod-a-68685b-8484pz9cs"
Oct 30 20:08:33.254559 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:33.254454    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/a04c80cf-438f-4b49-933b-bf281cef07ad-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-a-68685b-8484pz9cs\" (UID: \"a04c80cf-438f-4b49-933b-bf281cef07ad\") " pod="ai-tenant-e2e-extmod-a-68685b/payload-processing-external-model-e2e-extmod-a-68685b-8484pz9cs"
Oct 30 20:08:33.254677 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:33.254566    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/a04c80cf-438f-4b49-933b-bf281cef07ad-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-a-68685b-8484pz9cs\" (UID: \"a04c80cf-438f-4b49-933b-bf281cef07ad\") " pod="ai-tenant-e2e-extmod-a-68685b/payload-processing-external-model-e2e-extmod-a-68685b-8484pz9cs"
Oct 30 20:08:33.255287 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:33.255247    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a04c80cf-438f-4b49-933b-bf281cef07ad-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-a-68685b-8484pz9cs\" (UID: \"a04c80cf-438f-4b49-933b-bf281cef07ad\") " pod="ai-tenant-e2e-extmod-a-68685b/payload-processing-external-model-e2e-extmod-a-68685b-8484pz9cs"
Oct 30 20:08:33.255404 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:33.255286    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/a04c80cf-438f-4b49-933b-bf281cef07ad-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-a-68685b-8484pz9cs\" (UID: \"a04c80cf-438f-4b49-933b-bf281cef07ad\") " pod="ai-tenant-e2e-extmod-a-68685b/payload-processing-external-model-e2e-extmod-a-68685b-8484pz9cs"
Oct 30 20:08:33.257241 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:33.257214    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/a04c80cf-438f-4b49-933b-bf281cef07ad-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-a-68685b-8484pz9cs\" (UID: \"a04c80cf-438f-4b49-933b-bf281cef07ad\") " pod="ai-tenant-e2e-extmod-a-68685b/payload-processing-external-model-e2e-extmod-a-68685b-8484pz9cs"
Oct 30 20:08:33.268154 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:33.268081    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7f4r\" (UniqueName: \"kubernetes.io/projected/a04c80cf-438f-4b49-933b-bf281cef07ad-kube-api-access-f7f4r\") pod \"payload-processing-external-model-e2e-extmod-a-68685b-8484pz9cs\" (UID: \"a04c80cf-438f-4b49-933b-bf281cef07ad\") " pod="ai-tenant-e2e-extmod-a-68685b/payload-processing-external-model-e2e-extmod-a-68685b-8484pz9cs"
Oct 30 20:08:33.349962 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:33.349923    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-68685b/payload-processing-external-model-e2e-extmod-a-68685b-8484pz9cs"
Oct 30 20:08:33.668335 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:33.668235    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-a-68685b/payload-processing-external-model-e2e-extmod-a-68685b-8484pz9cs"]
Oct 30 20:08:33.671962 ip-10-0-141-42 kubenswrapper[6996]: W1030 20:08:33.671929    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda04c80cf_438f_4b49_933b_bf281cef07ad.slice/crio-240623b8057461c6145f102694ae990da218915a23ce233ed6d68ead6466c238 WatchSource:0}: Error finding container 240623b8057461c6145f102694ae990da218915a23ce233ed6d68ead6466c238: Status 404 returned error can't find the container with id 240623b8057461c6145f102694ae990da218915a23ce233ed6d68ead6466c238
Oct 30 20:08:34.166371 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:34.166332    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-68685b/payload-processing-external-model-e2e-extmod-a-68685b-8484pz9cs" event={"ID":"a04c80cf-438f-4b49-933b-bf281cef07ad","Type":"ContainerStarted","Data":"0b9e13c1a6475af6ca28f196e97ac244ee402c88348a1868d6af9e47d1d62602"}
Oct 30 20:08:34.166931 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:34.166408    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-68685b/payload-processing-external-model-e2e-extmod-a-68685b-8484pz9cs" event={"ID":"a04c80cf-438f-4b49-933b-bf281cef07ad","Type":"ContainerStarted","Data":"240623b8057461c6145f102694ae990da218915a23ce233ed6d68ead6466c238"}
Oct 30 20:08:34.166931 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:34.166635    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-extmod-a-68685b/payload-processing-external-model-e2e-extmod-a-68685b-8484pz9cs"
Oct 30 20:08:34.195755 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:34.195703    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-extmod-a-68685b/payload-processing-external-model-e2e-extmod-a-68685b-8484pz9cs" podStartSLOduration=2.19568865 podStartE2EDuration="2.19568865s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:08:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:08:34.195474118 +0000 UTC m=+2189.523592273" watchObservedRunningTime="2026-10-30 20:08:34.19568865 +0000 UTC m=+2189.523806805"
Oct 30 20:08:44.382994 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:44.382951    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-a-68685b/payload-processing-external-model-e2e-extmod-a-68685b-8484pz9cs"]
Oct 30 20:08:44.387918 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:44.387892    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-extmod-a-68685b/payload-processing-external-model-e2e-extmod-a-68685b-8484pz9cs"
Oct 30 20:08:49.385631 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:49.385573    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-extmod-a-68685b/payload-processing-external-model-e2e-extmod-a-68685b-8484pz9cs" podUID="a04c80cf-438f-4b49-933b-bf281cef07ad" containerName="payload-processing" containerID="cri-o://0b9e13c1a6475af6ca28f196e97ac244ee402c88348a1868d6af9e47d1d62602" gracePeriod=25
Oct 30 20:08:49.667874 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:49.667848    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-68685b/payload-processing-external-model-e2e-extmod-a-68685b-8484pz9cs"
Oct 30 20:08:49.784463 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:49.784427    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a04c80cf-438f-4b49-933b-bf281cef07ad-kube-api-access-f7f4r\" (UniqueName: \"kubernetes.io/projected/a04c80cf-438f-4b49-933b-bf281cef07ad-kube-api-access-f7f4r\") pod \"a04c80cf-438f-4b49-933b-bf281cef07ad\" (UID: \"a04c80cf-438f-4b49-933b-bf281cef07ad\") "
Oct 30 20:08:49.784648 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:49.784572    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a04c80cf-438f-4b49-933b-bf281cef07ad-provider-credentials\" (UniqueName: \"kubernetes.io/projected/a04c80cf-438f-4b49-933b-bf281cef07ad-provider-credentials\") pod \"a04c80cf-438f-4b49-933b-bf281cef07ad\" (UID: \"a04c80cf-438f-4b49-933b-bf281cef07ad\") "
Oct 30 20:08:49.784648 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:49.784621    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a04c80cf-438f-4b49-933b-bf281cef07ad-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a04c80cf-438f-4b49-933b-bf281cef07ad-plugins-config-volume\") pod \"a04c80cf-438f-4b49-933b-bf281cef07ad\" (UID: \"a04c80cf-438f-4b49-933b-bf281cef07ad\") "
Oct 30 20:08:49.784754 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:49.784655    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a04c80cf-438f-4b49-933b-bf281cef07ad-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/a04c80cf-438f-4b49-933b-bf281cef07ad-routing-overlay-volume\") pod \"a04c80cf-438f-4b49-933b-bf281cef07ad\" (UID: \"a04c80cf-438f-4b49-933b-bf281cef07ad\") "
Oct 30 20:08:49.785056 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:49.785029    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a04c80cf-438f-4b49-933b-bf281cef07ad-plugins-config-volume" pod "a04c80cf-438f-4b49-933b-bf281cef07ad" (UID: "a04c80cf-438f-4b49-933b-bf281cef07ad"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:08:49.785145 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:49.785084    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a04c80cf-438f-4b49-933b-bf281cef07ad-routing-overlay-volume" pod "a04c80cf-438f-4b49-933b-bf281cef07ad" (UID: "a04c80cf-438f-4b49-933b-bf281cef07ad"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:08:49.786806 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:49.786786    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a04c80cf-438f-4b49-933b-bf281cef07ad-provider-credentials" pod "a04c80cf-438f-4b49-933b-bf281cef07ad" (UID: "a04c80cf-438f-4b49-933b-bf281cef07ad"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:08:49.786904 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:49.786807    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a04c80cf-438f-4b49-933b-bf281cef07ad-kube-api-access-f7f4r" pod "a04c80cf-438f-4b49-933b-bf281cef07ad" (UID: "a04c80cf-438f-4b49-933b-bf281cef07ad"). InnerVolumeSpecName "kube-api-access-f7f4r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:08:49.887499 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:49.886754    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f7f4r\" (UniqueName: \"kubernetes.io/projected/a04c80cf-438f-4b49-933b-bf281cef07ad-kube-api-access-f7f4r\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:08:49.887499 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:49.886792    6996 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/a04c80cf-438f-4b49-933b-bf281cef07ad-provider-credentials\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:08:49.887499 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:49.886807    6996 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a04c80cf-438f-4b49-933b-bf281cef07ad-plugins-config-volume\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:08:49.887499 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:49.886822    6996 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/a04c80cf-438f-4b49-933b-bf281cef07ad-routing-overlay-volume\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:08:50.309580 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:50.309541    6996 generic.go:356] "Generic (PLEG): container finished" podID="a04c80cf-438f-4b49-933b-bf281cef07ad" containerID="0b9e13c1a6475af6ca28f196e97ac244ee402c88348a1868d6af9e47d1d62602" exitCode=0
Oct 30 20:08:50.309770 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:50.309585    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-68685b/payload-processing-external-model-e2e-extmod-a-68685b-8484pz9cs"
Oct 30 20:08:50.309770 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:50.309621    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-68685b/payload-processing-external-model-e2e-extmod-a-68685b-8484pz9cs" event={"ID":"a04c80cf-438f-4b49-933b-bf281cef07ad","Type":"ContainerDied","Data":"0b9e13c1a6475af6ca28f196e97ac244ee402c88348a1868d6af9e47d1d62602"}
Oct 30 20:08:50.309770 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:50.309676    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-68685b/payload-processing-external-model-e2e-extmod-a-68685b-8484pz9cs" event={"ID":"a04c80cf-438f-4b49-933b-bf281cef07ad","Type":"ContainerDied","Data":"240623b8057461c6145f102694ae990da218915a23ce233ed6d68ead6466c238"}
Oct 30 20:08:50.309770 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:50.309702    6996 scope.go:122] "RemoveContainer" containerID="0b9e13c1a6475af6ca28f196e97ac244ee402c88348a1868d6af9e47d1d62602"
Oct 30 20:08:50.321410 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:50.321393    6996 scope.go:122] "RemoveContainer" containerID="0b9e13c1a6475af6ca28f196e97ac244ee402c88348a1868d6af9e47d1d62602"
Oct 30 20:08:50.321701 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:08:50.321679    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b9e13c1a6475af6ca28f196e97ac244ee402c88348a1868d6af9e47d1d62602\": container with ID starting with 0b9e13c1a6475af6ca28f196e97ac244ee402c88348a1868d6af9e47d1d62602 not found: ID does not exist" containerID="0b9e13c1a6475af6ca28f196e97ac244ee402c88348a1868d6af9e47d1d62602"
Oct 30 20:08:50.321791 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:50.321708    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b9e13c1a6475af6ca28f196e97ac244ee402c88348a1868d6af9e47d1d62602"} err="failed to get container status \"0b9e13c1a6475af6ca28f196e97ac244ee402c88348a1868d6af9e47d1d62602\": rpc error: code = NotFound desc = could not find container \"0b9e13c1a6475af6ca28f196e97ac244ee402c88348a1868d6af9e47d1d62602\": container with ID starting with 0b9e13c1a6475af6ca28f196e97ac244ee402c88348a1868d6af9e47d1d62602 not found: ID does not exist"
Oct 30 20:08:50.336861 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:50.336838    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-a-68685b/payload-processing-external-model-e2e-extmod-a-68685b-8484pz9cs"]
Oct 30 20:08:50.340636 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:50.340614    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-extmod-a-68685b/payload-processing-external-model-e2e-extmod-a-68685b-8484pz9cs"]
Oct 30 20:08:50.795579 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:08:50.795542    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a04c80cf-438f-4b49-933b-bf281cef07ad" path="/var/lib/kubelet/pods/a04c80cf-438f-4b49-933b-bf281cef07ad/volumes"
Oct 30 20:09:01.193641 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:09:01.193598    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 20:09:02.558027 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:02.557990    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["models-as-a-service/payload-processing-external-model-55d5448f77-cg4z5"]
Oct 30 20:09:02.558739 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:02.558724    6996 cpu_manager.go:414] "containerMap: removing container" podUID="a04c80cf-438f-4b49-933b-bf281cef07ad" containerName="payload-processing"
Oct 30 20:09:02.558812 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:02.558740    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a04c80cf-438f-4b49-933b-bf281cef07ad" containerName="payload-processing"
Oct 30 20:09:02.558871 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:02.558862    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="a04c80cf-438f-4b49-933b-bf281cef07ad" containerName="payload-processing"
Oct 30 20:09:02.562741 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:02.562719    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-55d5448f77-cg4z5"
Oct 30 20:09:02.574590 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:02.574553    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"payload-processing-external-model-plugins\""
Oct 30 20:09:02.574590 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:02.574558    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"agc-cred-08c2badb5904\""
Oct 30 20:09:02.574866 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:02.574565    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"routing-overlay\""
Oct 30 20:09:02.574866 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:02.574763    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"openshift-service-ca.crt\""
Oct 30 20:09:02.574866 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:02.574818    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"payload-processing-external-model-dockercfg-mqtgc\""
Oct 30 20:09:02.580961 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:02.580941    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"kube-root-ca.crt\""
Oct 30 20:09:02.599172 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:02.599142    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-55d5448f77-cg4z5"]
Oct 30 20:09:02.704139 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:02.704076    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/27830dfc-9a91-4dda-8aa4-d0fda52f977c-routing-overlay-volume\") pod \"payload-processing-external-model-55d5448f77-cg4z5\" (UID: \"27830dfc-9a91-4dda-8aa4-d0fda52f977c\") " pod="models-as-a-service/payload-processing-external-model-55d5448f77-cg4z5"
Oct 30 20:09:02.704680 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:02.704586    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/27830dfc-9a91-4dda-8aa4-d0fda52f977c-provider-credentials\") pod \"payload-processing-external-model-55d5448f77-cg4z5\" (UID: \"27830dfc-9a91-4dda-8aa4-d0fda52f977c\") " pod="models-as-a-service/payload-processing-external-model-55d5448f77-cg4z5"
Oct 30 20:09:02.704872 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:02.704843    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbwtk\" (UniqueName: \"kubernetes.io/projected/27830dfc-9a91-4dda-8aa4-d0fda52f977c-kube-api-access-bbwtk\") pod \"payload-processing-external-model-55d5448f77-cg4z5\" (UID: \"27830dfc-9a91-4dda-8aa4-d0fda52f977c\") " pod="models-as-a-service/payload-processing-external-model-55d5448f77-cg4z5"
Oct 30 20:09:02.705132 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:02.705105    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/27830dfc-9a91-4dda-8aa4-d0fda52f977c-plugins-config-volume\") pod \"payload-processing-external-model-55d5448f77-cg4z5\" (UID: \"27830dfc-9a91-4dda-8aa4-d0fda52f977c\") " pod="models-as-a-service/payload-processing-external-model-55d5448f77-cg4z5"
Oct 30 20:09:02.812548 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:02.807957    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/27830dfc-9a91-4dda-8aa4-d0fda52f977c-provider-credentials\") pod \"payload-processing-external-model-55d5448f77-cg4z5\" (UID: \"27830dfc-9a91-4dda-8aa4-d0fda52f977c\") " pod="models-as-a-service/payload-processing-external-model-55d5448f77-cg4z5"
Oct 30 20:09:02.812548 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:02.808015    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbwtk\" (UniqueName: \"kubernetes.io/projected/27830dfc-9a91-4dda-8aa4-d0fda52f977c-kube-api-access-bbwtk\") pod \"payload-processing-external-model-55d5448f77-cg4z5\" (UID: \"27830dfc-9a91-4dda-8aa4-d0fda52f977c\") " pod="models-as-a-service/payload-processing-external-model-55d5448f77-cg4z5"
Oct 30 20:09:02.812548 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:02.808183    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/27830dfc-9a91-4dda-8aa4-d0fda52f977c-plugins-config-volume\") pod \"payload-processing-external-model-55d5448f77-cg4z5\" (UID: \"27830dfc-9a91-4dda-8aa4-d0fda52f977c\") " pod="models-as-a-service/payload-processing-external-model-55d5448f77-cg4z5"
Oct 30 20:09:02.812548 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:02.808349    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/27830dfc-9a91-4dda-8aa4-d0fda52f977c-routing-overlay-volume\") pod \"payload-processing-external-model-55d5448f77-cg4z5\" (UID: \"27830dfc-9a91-4dda-8aa4-d0fda52f977c\") " pod="models-as-a-service/payload-processing-external-model-55d5448f77-cg4z5"
Oct 30 20:09:02.812548 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:02.809364    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/27830dfc-9a91-4dda-8aa4-d0fda52f977c-routing-overlay-volume\") pod \"payload-processing-external-model-55d5448f77-cg4z5\" (UID: \"27830dfc-9a91-4dda-8aa4-d0fda52f977c\") " pod="models-as-a-service/payload-processing-external-model-55d5448f77-cg4z5"
Oct 30 20:09:02.812548 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:02.810715    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/27830dfc-9a91-4dda-8aa4-d0fda52f977c-plugins-config-volume\") pod \"payload-processing-external-model-55d5448f77-cg4z5\" (UID: \"27830dfc-9a91-4dda-8aa4-d0fda52f977c\") " pod="models-as-a-service/payload-processing-external-model-55d5448f77-cg4z5"
Oct 30 20:09:02.822211 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:02.814138    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/27830dfc-9a91-4dda-8aa4-d0fda52f977c-provider-credentials\") pod \"payload-processing-external-model-55d5448f77-cg4z5\" (UID: \"27830dfc-9a91-4dda-8aa4-d0fda52f977c\") " pod="models-as-a-service/payload-processing-external-model-55d5448f77-cg4z5"
Oct 30 20:09:02.853929 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:02.853893    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbwtk\" (UniqueName: \"kubernetes.io/projected/27830dfc-9a91-4dda-8aa4-d0fda52f977c-kube-api-access-bbwtk\") pod \"payload-processing-external-model-55d5448f77-cg4z5\" (UID: \"27830dfc-9a91-4dda-8aa4-d0fda52f977c\") " pod="models-as-a-service/payload-processing-external-model-55d5448f77-cg4z5"
Oct 30 20:09:02.878915 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:02.878877    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-55d5448f77-cg4z5"
Oct 30 20:09:03.252967 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:03.252939    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-55d5448f77-cg4z5"]
Oct 30 20:09:03.254287 ip-10-0-141-42 kubenswrapper[6996]: W1030 20:09:03.254251    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27830dfc_9a91_4dda_8aa4_d0fda52f977c.slice/crio-72f9dd1dca7ef2d4e93ddfd2dbf9b44d0f2375724016e9f3c13061c4391b58c1 WatchSource:0}: Error finding container 72f9dd1dca7ef2d4e93ddfd2dbf9b44d0f2375724016e9f3c13061c4391b58c1: Status 404 returned error can't find the container with id 72f9dd1dca7ef2d4e93ddfd2dbf9b44d0f2375724016e9f3c13061c4391b58c1
Oct 30 20:09:03.449551 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:03.449514    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-55d5448f77-cg4z5" event={"ID":"27830dfc-9a91-4dda-8aa4-d0fda52f977c","Type":"ContainerStarted","Data":"a7ce900fb054b46cf6ef4755a2207ef0bcfae18eaeef4defd2e6523554b860c9"}
Oct 30 20:09:03.449743 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:03.449568    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-55d5448f77-cg4z5" event={"ID":"27830dfc-9a91-4dda-8aa4-d0fda52f977c","Type":"ContainerStarted","Data":"72f9dd1dca7ef2d4e93ddfd2dbf9b44d0f2375724016e9f3c13061c4391b58c1"}
Oct 30 20:09:03.482708 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:03.482626    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="models-as-a-service/payload-processing-external-model-55d5448f77-cg4z5" podStartSLOduration=1.482604639 podStartE2EDuration="1.482604639s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:09:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:09:03.47499289 +0000 UTC m=+2218.803111041" watchObservedRunningTime="2026-10-30 20:09:03.482604639 +0000 UTC m=+2218.810722793"
Oct 30 20:09:04.456379 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:04.456346    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="models-as-a-service/payload-processing-external-model-55d5448f77-cg4z5"
Oct 30 20:09:10.370768 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:10.370725    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-55d5448f77-cg4z5"]
Oct 30 20:09:10.374701 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:10.374669    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="models-as-a-service/payload-processing-external-model-55d5448f77-cg4z5"
Oct 30 20:09:15.371919 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:15.371849    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="models-as-a-service/payload-processing-external-model-55d5448f77-cg4z5" podUID="27830dfc-9a91-4dda-8aa4-d0fda52f977c" containerName="payload-processing" containerID="cri-o://a7ce900fb054b46cf6ef4755a2207ef0bcfae18eaeef4defd2e6523554b860c9" gracePeriod=25
Oct 30 20:09:15.588521 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:15.586971    6996 generic.go:356] "Generic (PLEG): container finished" podID="27830dfc-9a91-4dda-8aa4-d0fda52f977c" containerID="a7ce900fb054b46cf6ef4755a2207ef0bcfae18eaeef4defd2e6523554b860c9" exitCode=0
Oct 30 20:09:15.588521 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:15.587051    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-55d5448f77-cg4z5" event={"ID":"27830dfc-9a91-4dda-8aa4-d0fda52f977c","Type":"ContainerDied","Data":"a7ce900fb054b46cf6ef4755a2207ef0bcfae18eaeef4defd2e6523554b860c9"}
Oct 30 20:09:15.688471 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:15.688438    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-55d5448f77-cg4z5"
Oct 30 20:09:15.807122 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:15.807083    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/27830dfc-9a91-4dda-8aa4-d0fda52f977c-kube-api-access-bbwtk\" (UniqueName: \"kubernetes.io/projected/27830dfc-9a91-4dda-8aa4-d0fda52f977c-kube-api-access-bbwtk\") pod \"27830dfc-9a91-4dda-8aa4-d0fda52f977c\" (UID: \"27830dfc-9a91-4dda-8aa4-d0fda52f977c\") "
Oct 30 20:09:15.807322 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:15.807178    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/27830dfc-9a91-4dda-8aa4-d0fda52f977c-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/27830dfc-9a91-4dda-8aa4-d0fda52f977c-plugins-config-volume\") pod \"27830dfc-9a91-4dda-8aa4-d0fda52f977c\" (UID: \"27830dfc-9a91-4dda-8aa4-d0fda52f977c\") "
Oct 30 20:09:15.807322 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:15.807251    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/27830dfc-9a91-4dda-8aa4-d0fda52f977c-provider-credentials\" (UniqueName: \"kubernetes.io/projected/27830dfc-9a91-4dda-8aa4-d0fda52f977c-provider-credentials\") pod \"27830dfc-9a91-4dda-8aa4-d0fda52f977c\" (UID: \"27830dfc-9a91-4dda-8aa4-d0fda52f977c\") "
Oct 30 20:09:15.807441 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:15.807363    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/27830dfc-9a91-4dda-8aa4-d0fda52f977c-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/27830dfc-9a91-4dda-8aa4-d0fda52f977c-routing-overlay-volume\") pod \"27830dfc-9a91-4dda-8aa4-d0fda52f977c\" (UID: \"27830dfc-9a91-4dda-8aa4-d0fda52f977c\") "
Oct 30 20:09:15.807738 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:15.807664    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27830dfc-9a91-4dda-8aa4-d0fda52f977c-plugins-config-volume" pod "27830dfc-9a91-4dda-8aa4-d0fda52f977c" (UID: "27830dfc-9a91-4dda-8aa4-d0fda52f977c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:09:15.808137 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:15.808110    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27830dfc-9a91-4dda-8aa4-d0fda52f977c-routing-overlay-volume" pod "27830dfc-9a91-4dda-8aa4-d0fda52f977c" (UID: "27830dfc-9a91-4dda-8aa4-d0fda52f977c"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:09:15.808236 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:15.808136    6996 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/27830dfc-9a91-4dda-8aa4-d0fda52f977c-plugins-config-volume\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:09:15.809678 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:15.809651    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27830dfc-9a91-4dda-8aa4-d0fda52f977c-provider-credentials" pod "27830dfc-9a91-4dda-8aa4-d0fda52f977c" (UID: "27830dfc-9a91-4dda-8aa4-d0fda52f977c"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:09:15.809850 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:15.809829    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27830dfc-9a91-4dda-8aa4-d0fda52f977c-kube-api-access-bbwtk" pod "27830dfc-9a91-4dda-8aa4-d0fda52f977c" (UID: "27830dfc-9a91-4dda-8aa4-d0fda52f977c"). InnerVolumeSpecName "kube-api-access-bbwtk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:09:15.909424 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:15.909327    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bbwtk\" (UniqueName: \"kubernetes.io/projected/27830dfc-9a91-4dda-8aa4-d0fda52f977c-kube-api-access-bbwtk\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:09:15.909424 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:15.909361    6996 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/27830dfc-9a91-4dda-8aa4-d0fda52f977c-provider-credentials\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:09:15.909424 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:15.909375    6996 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/27830dfc-9a91-4dda-8aa4-d0fda52f977c-routing-overlay-volume\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:09:16.614232 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:16.614200    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-55d5448f77-cg4z5"
Oct 30 20:09:16.614853 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:16.614368    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-55d5448f77-cg4z5" event={"ID":"27830dfc-9a91-4dda-8aa4-d0fda52f977c","Type":"ContainerDied","Data":"72f9dd1dca7ef2d4e93ddfd2dbf9b44d0f2375724016e9f3c13061c4391b58c1"}
Oct 30 20:09:16.614853 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:16.614444    6996 scope.go:122] "RemoveContainer" containerID="a7ce900fb054b46cf6ef4755a2207ef0bcfae18eaeef4defd2e6523554b860c9"
Oct 30 20:09:16.656794 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:16.656758    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-55d5448f77-cg4z5"]
Oct 30 20:09:16.661549 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:16.661514    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["models-as-a-service/payload-processing-external-model-55d5448f77-cg4z5"]
Oct 30 20:09:16.793848 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:16.793814    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="27830dfc-9a91-4dda-8aa4-d0fda52f977c" path="/var/lib/kubelet/pods/27830dfc-9a91-4dda-8aa4-d0fda52f977c/volumes"
Oct 30 20:09:25.509321 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:25.509280    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 20:09:31.211839 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:09:31.211782    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 20:09:36.097623 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:36.097583    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-c2lhh"]
Oct 30 20:09:36.098361 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:36.098342    6996 cpu_manager.go:414] "containerMap: removing container" podUID="27830dfc-9a91-4dda-8aa4-d0fda52f977c" containerName="payload-processing"
Oct 30 20:09:36.098361 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:36.098360    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="27830dfc-9a91-4dda-8aa4-d0fda52f977c" containerName="payload-processing"
Oct 30 20:09:36.098535 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:36.098489    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="27830dfc-9a91-4dda-8aa4-d0fda52f977c" containerName="payload-processing"
Oct 30 20:09:36.102532 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:36.102507    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-c2lhh"
Oct 30 20:09:36.121994 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:36.121952    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-c2lhh"]
Oct 30 20:09:36.155843 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:36.155808    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac5ca902-49d8-4354-8c3d-57f0db27912b-istio-data\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-c2lhh\" (UID: \"ac5ca902-49d8-4354-8c3d-57f0db27912b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-c2lhh"
Oct 30 20:09:36.155994 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:36.155937    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac5ca902-49d8-4354-8c3d-57f0db27912b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-c2lhh\" (UID: \"ac5ca902-49d8-4354-8c3d-57f0db27912b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-c2lhh"
Oct 30 20:09:36.156070 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:36.156046    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac5ca902-49d8-4354-8c3d-57f0db27912b-workload-socket\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-c2lhh\" (UID: \"ac5ca902-49d8-4354-8c3d-57f0db27912b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-c2lhh"
Oct 30 20:09:36.156125 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:36.156109    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac5ca902-49d8-4354-8c3d-57f0db27912b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-c2lhh\" (UID: \"ac5ca902-49d8-4354-8c3d-57f0db27912b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-c2lhh"
Oct 30 20:09:36.156296 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:36.156276    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac5ca902-49d8-4354-8c3d-57f0db27912b-istio-token\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-c2lhh\" (UID: \"ac5ca902-49d8-4354-8c3d-57f0db27912b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-c2lhh"
Oct 30 20:09:36.156384 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:36.156328    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac5ca902-49d8-4354-8c3d-57f0db27912b-workload-certs\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-c2lhh\" (UID: \"ac5ca902-49d8-4354-8c3d-57f0db27912b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-c2lhh"
Oct 30 20:09:36.156460 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:36.156438    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac5ca902-49d8-4354-8c3d-57f0db27912b-credential-socket\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-c2lhh\" (UID: \"ac5ca902-49d8-4354-8c3d-57f0db27912b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-c2lhh"
Oct 30 20:09:36.156548 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:36.156518    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2zjb\" (UniqueName: \"kubernetes.io/projected/ac5ca902-49d8-4354-8c3d-57f0db27912b-kube-api-access-d2zjb\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-c2lhh\" (UID: \"ac5ca902-49d8-4354-8c3d-57f0db27912b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-c2lhh"
Oct 30 20:09:36.156612 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:36.156593    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac5ca902-49d8-4354-8c3d-57f0db27912b-istio-envoy\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-c2lhh\" (UID: \"ac5ca902-49d8-4354-8c3d-57f0db27912b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-c2lhh"
Oct 30 20:09:36.257676 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:36.257642    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac5ca902-49d8-4354-8c3d-57f0db27912b-istio-data\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-c2lhh\" (UID: \"ac5ca902-49d8-4354-8c3d-57f0db27912b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-c2lhh"
Oct 30 20:09:36.257876 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:36.257711    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac5ca902-49d8-4354-8c3d-57f0db27912b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-c2lhh\" (UID: \"ac5ca902-49d8-4354-8c3d-57f0db27912b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-c2lhh"
Oct 30 20:09:36.257876 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:36.257765    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac5ca902-49d8-4354-8c3d-57f0db27912b-workload-socket\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-c2lhh\" (UID: \"ac5ca902-49d8-4354-8c3d-57f0db27912b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-c2lhh"
Oct 30 20:09:36.257876 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:36.257808    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac5ca902-49d8-4354-8c3d-57f0db27912b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-c2lhh\" (UID: \"ac5ca902-49d8-4354-8c3d-57f0db27912b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-c2lhh"
Oct 30 20:09:36.258071 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:36.257963    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac5ca902-49d8-4354-8c3d-57f0db27912b-istio-token\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-c2lhh\" (UID: \"ac5ca902-49d8-4354-8c3d-57f0db27912b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-c2lhh"
Oct 30 20:09:36.258071 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:36.258016    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac5ca902-49d8-4354-8c3d-57f0db27912b-workload-certs\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-c2lhh\" (UID: \"ac5ca902-49d8-4354-8c3d-57f0db27912b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-c2lhh"
Oct 30 20:09:36.258226 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:36.258071    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac5ca902-49d8-4354-8c3d-57f0db27912b-credential-socket\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-c2lhh\" (UID: \"ac5ca902-49d8-4354-8c3d-57f0db27912b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-c2lhh"
Oct 30 20:09:36.258226 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:36.258121    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac5ca902-49d8-4354-8c3d-57f0db27912b-istio-data\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-c2lhh\" (UID: \"ac5ca902-49d8-4354-8c3d-57f0db27912b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-c2lhh"
Oct 30 20:09:36.258226 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:36.258124    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2zjb\" (UniqueName: \"kubernetes.io/projected/ac5ca902-49d8-4354-8c3d-57f0db27912b-kube-api-access-d2zjb\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-c2lhh\" (UID: \"ac5ca902-49d8-4354-8c3d-57f0db27912b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-c2lhh"
Oct 30 20:09:36.258226 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:36.258162    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac5ca902-49d8-4354-8c3d-57f0db27912b-workload-socket\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-c2lhh\" (UID: \"ac5ca902-49d8-4354-8c3d-57f0db27912b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-c2lhh"
Oct 30 20:09:36.258618 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:36.258258    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac5ca902-49d8-4354-8c3d-57f0db27912b-istio-envoy\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-c2lhh\" (UID: \"ac5ca902-49d8-4354-8c3d-57f0db27912b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-c2lhh"
Oct 30 20:09:36.258618 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:36.258341    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac5ca902-49d8-4354-8c3d-57f0db27912b-workload-certs\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-c2lhh\" (UID: \"ac5ca902-49d8-4354-8c3d-57f0db27912b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-c2lhh"
Oct 30 20:09:36.258618 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:36.258424    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac5ca902-49d8-4354-8c3d-57f0db27912b-credential-socket\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-c2lhh\" (UID: \"ac5ca902-49d8-4354-8c3d-57f0db27912b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-c2lhh"
Oct 30 20:09:36.258728 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:36.258646    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac5ca902-49d8-4354-8c3d-57f0db27912b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-c2lhh\" (UID: \"ac5ca902-49d8-4354-8c3d-57f0db27912b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-c2lhh"
Oct 30 20:09:36.260616 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:36.260595    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac5ca902-49d8-4354-8c3d-57f0db27912b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-c2lhh\" (UID: \"ac5ca902-49d8-4354-8c3d-57f0db27912b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-c2lhh"
Oct 30 20:09:36.260707 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:36.260665    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac5ca902-49d8-4354-8c3d-57f0db27912b-istio-envoy\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-c2lhh\" (UID: \"ac5ca902-49d8-4354-8c3d-57f0db27912b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-c2lhh"
Oct 30 20:09:36.271754 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:36.271732    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac5ca902-49d8-4354-8c3d-57f0db27912b-istio-token\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-c2lhh\" (UID: \"ac5ca902-49d8-4354-8c3d-57f0db27912b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-c2lhh"
Oct 30 20:09:36.278839 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:36.278815    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2zjb\" (UniqueName: \"kubernetes.io/projected/ac5ca902-49d8-4354-8c3d-57f0db27912b-kube-api-access-d2zjb\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-c2lhh\" (UID: \"ac5ca902-49d8-4354-8c3d-57f0db27912b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-c2lhh"
Oct 30 20:09:36.424097 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:36.424007    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-c2lhh"
Oct 30 20:09:36.751328 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:36.751298    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-c2lhh"]
Oct 30 20:09:36.752800 ip-10-0-141-42 kubenswrapper[6996]: W1030 20:09:36.752776    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac5ca902_49d8_4354_8c3d_57f0db27912b.slice/crio-91a2edda0a17b577ba73534468c473c9d22aa8c379f5ec8ad4ba6007b89d739d WatchSource:0}: Error finding container 91a2edda0a17b577ba73534468c473c9d22aa8c379f5ec8ad4ba6007b89d739d: Status 404 returned error can't find the container with id 91a2edda0a17b577ba73534468c473c9d22aa8c379f5ec8ad4ba6007b89d739d
Oct 30 20:09:36.755699 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:36.755665    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 20:09:36.755789 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:36.755731    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 20:09:36.755789 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:36.755762    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 20:09:36.780889 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:36.780859    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-c2lhh" event={"ID":"ac5ca902-49d8-4354-8c3d-57f0db27912b","Type":"ContainerStarted","Data":"91a2edda0a17b577ba73534468c473c9d22aa8c379f5ec8ad4ba6007b89d739d"}
Oct 30 20:09:37.789534 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:37.789496    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-c2lhh" event={"ID":"ac5ca902-49d8-4354-8c3d-57f0db27912b","Type":"ContainerStarted","Data":"7211057bf473da110c92369cd6e527336fb7578e835453c0cf2a31bd4c43fb3e"}
Oct 30 20:09:37.817759 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:37.817700    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-c2lhh" podStartSLOduration=1.817686692 podStartE2EDuration="1.817686692s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:09:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:09:37.815929742 +0000 UTC m=+2253.144047895" watchObservedRunningTime="2026-10-30 20:09:37.817686692 +0000 UTC m=+2253.145804846"
Oct 30 20:09:38.424519 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:38.424441    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-c2lhh"
Oct 30 20:09:38.426210 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:38.426179    6996 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5db77fbd9f-c2lhh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.166:15021/healthz/ready\": dial tcp 10.134.0.166:15021: connect: connection refused" start-of-body=
Oct 30 20:09:38.426342 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:38.426245    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-c2lhh" podUID="ac5ca902-49d8-4354-8c3d-57f0db27912b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.166:15021/healthz/ready\": dial tcp 10.134.0.166:15021: connect: connection refused"
Oct 30 20:09:39.424397 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:39.424359    6996 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5db77fbd9f-c2lhh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.166:15021/healthz/ready\": dial tcp 10.134.0.166:15021: connect: connection refused" start-of-body=
Oct 30 20:09:39.424809 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:39.424427    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-c2lhh" podUID="ac5ca902-49d8-4354-8c3d-57f0db27912b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.166:15021/healthz/ready\": dial tcp 10.134.0.166:15021: connect: connection refused"
Oct 30 20:09:40.429556 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:40.429518    6996 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5db77fbd9f-c2lhh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.166:15021/healthz/ready\": dial tcp 10.134.0.166:15021: connect: connection refused" start-of-body=
Oct 30 20:09:40.430219 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:40.430186    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-c2lhh" podUID="ac5ca902-49d8-4354-8c3d-57f0db27912b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.166:15021/healthz/ready\": dial tcp 10.134.0.166:15021: connect: connection refused"
Oct 30 20:09:41.425286 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:41.425252    6996 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5db77fbd9f-c2lhh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.166:15021/healthz/ready\": dial tcp 10.134.0.166:15021: connect: connection refused" start-of-body=
Oct 30 20:09:41.425554 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:41.425307    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-c2lhh" podUID="ac5ca902-49d8-4354-8c3d-57f0db27912b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.166:15021/healthz/ready\": dial tcp 10.134.0.166:15021: connect: connection refused"
Oct 30 20:09:42.424367 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:42.424331    6996 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5db77fbd9f-c2lhh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.166:15021/healthz/ready\": dial tcp 10.134.0.166:15021: connect: connection refused" start-of-body=
Oct 30 20:09:42.424841 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:42.424386    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-c2lhh" podUID="ac5ca902-49d8-4354-8c3d-57f0db27912b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.166:15021/healthz/ready\": dial tcp 10.134.0.166:15021: connect: connection refused"
Oct 30 20:09:43.424678 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:43.424624    6996 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5db77fbd9f-c2lhh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.166:15021/healthz/ready\": dial tcp 10.134.0.166:15021: connect: connection refused" start-of-body=
Oct 30 20:09:43.425194 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:43.424689    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-c2lhh" podUID="ac5ca902-49d8-4354-8c3d-57f0db27912b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.166:15021/healthz/ready\": dial tcp 10.134.0.166:15021: connect: connection refused"
Oct 30 20:09:44.424734 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:44.424698    6996 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5db77fbd9f-c2lhh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.166:15021/healthz/ready\": dial tcp 10.134.0.166:15021: connect: connection refused" start-of-body=
Oct 30 20:09:44.425127 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:44.424761    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-c2lhh" podUID="ac5ca902-49d8-4354-8c3d-57f0db27912b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.166:15021/healthz/ready\": dial tcp 10.134.0.166:15021: connect: connection refused"
Oct 30 20:09:45.704087 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:45.704043    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-c2lhh"
Oct 30 20:09:45.704528 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:45.704471    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-c2lhh"
Oct 30 20:09:45.705366 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:45.705344    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-c2lhh"
Oct 30 20:09:51.136654 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:51.136612    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-rbwvb"]
Oct 30 20:09:51.142521 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:51.142490    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-rbwvb"
Oct 30 20:09:51.178547 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:51.177660    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-rbwvb"]
Oct 30 20:09:51.214742 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:51.214703    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a76e670-c0a4-49ee-9b5f-7fabddaef0e9-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-rbwvb\" (UID: \"9a76e670-c0a4-49ee-9b5f-7fabddaef0e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-rbwvb"
Oct 30 20:09:51.214943 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:51.214788    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9a76e670-c0a4-49ee-9b5f-7fabddaef0e9-istio-token\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-rbwvb\" (UID: \"9a76e670-c0a4-49ee-9b5f-7fabddaef0e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-rbwvb"
Oct 30 20:09:51.214943 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:51.214861    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a76e670-c0a4-49ee-9b5f-7fabddaef0e9-istio-data\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-rbwvb\" (UID: \"9a76e670-c0a4-49ee-9b5f-7fabddaef0e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-rbwvb"
Oct 30 20:09:51.214943 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:51.214914    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kg5q2\" (UniqueName: \"kubernetes.io/projected/9a76e670-c0a4-49ee-9b5f-7fabddaef0e9-kube-api-access-kg5q2\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-rbwvb\" (UID: \"9a76e670-c0a4-49ee-9b5f-7fabddaef0e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-rbwvb"
Oct 30 20:09:51.215141 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:51.214971    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a76e670-c0a4-49ee-9b5f-7fabddaef0e9-workload-certs\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-rbwvb\" (UID: \"9a76e670-c0a4-49ee-9b5f-7fabddaef0e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-rbwvb"
Oct 30 20:09:51.215141 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:51.215012    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a76e670-c0a4-49ee-9b5f-7fabddaef0e9-istio-envoy\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-rbwvb\" (UID: \"9a76e670-c0a4-49ee-9b5f-7fabddaef0e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-rbwvb"
Oct 30 20:09:51.215141 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:51.215070    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a76e670-c0a4-49ee-9b5f-7fabddaef0e9-credential-socket\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-rbwvb\" (UID: \"9a76e670-c0a4-49ee-9b5f-7fabddaef0e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-rbwvb"
Oct 30 20:09:51.215141 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:51.215092    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a76e670-c0a4-49ee-9b5f-7fabddaef0e9-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-rbwvb\" (UID: \"9a76e670-c0a4-49ee-9b5f-7fabddaef0e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-rbwvb"
Oct 30 20:09:51.215357 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:51.215242    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a76e670-c0a4-49ee-9b5f-7fabddaef0e9-workload-socket\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-rbwvb\" (UID: \"9a76e670-c0a4-49ee-9b5f-7fabddaef0e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-rbwvb"
Oct 30 20:09:51.316732 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:51.316702    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a76e670-c0a4-49ee-9b5f-7fabddaef0e9-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-rbwvb\" (UID: \"9a76e670-c0a4-49ee-9b5f-7fabddaef0e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-rbwvb"
Oct 30 20:09:51.316891 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:51.316749    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9a76e670-c0a4-49ee-9b5f-7fabddaef0e9-istio-token\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-rbwvb\" (UID: \"9a76e670-c0a4-49ee-9b5f-7fabddaef0e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-rbwvb"
Oct 30 20:09:51.316891 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:51.316807    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a76e670-c0a4-49ee-9b5f-7fabddaef0e9-istio-data\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-rbwvb\" (UID: \"9a76e670-c0a4-49ee-9b5f-7fabddaef0e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-rbwvb"
Oct 30 20:09:51.316891 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:51.316834    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kg5q2\" (UniqueName: \"kubernetes.io/projected/9a76e670-c0a4-49ee-9b5f-7fabddaef0e9-kube-api-access-kg5q2\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-rbwvb\" (UID: \"9a76e670-c0a4-49ee-9b5f-7fabddaef0e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-rbwvb"
Oct 30 20:09:51.316891 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:51.316875    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a76e670-c0a4-49ee-9b5f-7fabddaef0e9-workload-certs\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-rbwvb\" (UID: \"9a76e670-c0a4-49ee-9b5f-7fabddaef0e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-rbwvb"
Oct 30 20:09:51.317076 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:51.316906    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a76e670-c0a4-49ee-9b5f-7fabddaef0e9-istio-envoy\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-rbwvb\" (UID: \"9a76e670-c0a4-49ee-9b5f-7fabddaef0e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-rbwvb"
Oct 30 20:09:51.317076 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:51.316958    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a76e670-c0a4-49ee-9b5f-7fabddaef0e9-credential-socket\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-rbwvb\" (UID: \"9a76e670-c0a4-49ee-9b5f-7fabddaef0e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-rbwvb"
Oct 30 20:09:51.317076 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:51.316980    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a76e670-c0a4-49ee-9b5f-7fabddaef0e9-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-rbwvb\" (UID: \"9a76e670-c0a4-49ee-9b5f-7fabddaef0e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-rbwvb"
Oct 30 20:09:51.317236 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:51.317101    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a76e670-c0a4-49ee-9b5f-7fabddaef0e9-workload-socket\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-rbwvb\" (UID: \"9a76e670-c0a4-49ee-9b5f-7fabddaef0e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-rbwvb"
Oct 30 20:09:51.317319 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:51.317293    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a76e670-c0a4-49ee-9b5f-7fabddaef0e9-workload-certs\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-rbwvb\" (UID: \"9a76e670-c0a4-49ee-9b5f-7fabddaef0e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-rbwvb"
Oct 30 20:09:51.317385 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:51.317339    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a76e670-c0a4-49ee-9b5f-7fabddaef0e9-istio-data\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-rbwvb\" (UID: \"9a76e670-c0a4-49ee-9b5f-7fabddaef0e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-rbwvb"
Oct 30 20:09:51.317614 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:51.317584    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a76e670-c0a4-49ee-9b5f-7fabddaef0e9-credential-socket\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-rbwvb\" (UID: \"9a76e670-c0a4-49ee-9b5f-7fabddaef0e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-rbwvb"
Oct 30 20:09:51.317614 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:51.317603    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a76e670-c0a4-49ee-9b5f-7fabddaef0e9-workload-socket\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-rbwvb\" (UID: \"9a76e670-c0a4-49ee-9b5f-7fabddaef0e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-rbwvb"
Oct 30 20:09:51.317857 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:51.317836    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a76e670-c0a4-49ee-9b5f-7fabddaef0e9-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-rbwvb\" (UID: \"9a76e670-c0a4-49ee-9b5f-7fabddaef0e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-rbwvb"
Oct 30 20:09:51.319301 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:51.319282    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a76e670-c0a4-49ee-9b5f-7fabddaef0e9-istio-envoy\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-rbwvb\" (UID: \"9a76e670-c0a4-49ee-9b5f-7fabddaef0e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-rbwvb"
Oct 30 20:09:51.319388 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:51.319333    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a76e670-c0a4-49ee-9b5f-7fabddaef0e9-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-rbwvb\" (UID: \"9a76e670-c0a4-49ee-9b5f-7fabddaef0e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-rbwvb"
Oct 30 20:09:51.331133 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:51.331111    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9a76e670-c0a4-49ee-9b5f-7fabddaef0e9-istio-token\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-rbwvb\" (UID: \"9a76e670-c0a4-49ee-9b5f-7fabddaef0e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-rbwvb"
Oct 30 20:09:51.334145 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:51.334119    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kg5q2\" (UniqueName: \"kubernetes.io/projected/9a76e670-c0a4-49ee-9b5f-7fabddaef0e9-kube-api-access-kg5q2\") pod \"maas-default-gateway-openshift-default-5db77fbd9f-rbwvb\" (UID: \"9a76e670-c0a4-49ee-9b5f-7fabddaef0e9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-rbwvb"
Oct 30 20:09:51.463358 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:51.463283    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-rbwvb"
Oct 30 20:09:51.761136 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:51.761104    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-rbwvb"]
Oct 30 20:09:51.762381 ip-10-0-141-42 kubenswrapper[6996]: W1030 20:09:51.762349    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a76e670_c0a4_49ee_9b5f_7fabddaef0e9.slice/crio-a9b2d1bdfc00960c1277ac3675fdbc8ea83aba580fd61d49df5a38405ce8264c WatchSource:0}: Error finding container a9b2d1bdfc00960c1277ac3675fdbc8ea83aba580fd61d49df5a38405ce8264c: Status 404 returned error can't find the container with id a9b2d1bdfc00960c1277ac3675fdbc8ea83aba580fd61d49df5a38405ce8264c
Oct 30 20:09:51.765464 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:51.765427    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 20:09:51.765576 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:51.765538    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 20:09:51.765619 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:51.765574    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 20:09:51.903557 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:51.903519    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-rbwvb" event={"ID":"9a76e670-c0a4-49ee-9b5f-7fabddaef0e9","Type":"ContainerStarted","Data":"995b0f40241869bb396be27ca6069f8a7f394bd692c1436edb5d7c8dbb9f342b"}
Oct 30 20:09:51.903779 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:51.903664    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-rbwvb" event={"ID":"9a76e670-c0a4-49ee-9b5f-7fabddaef0e9","Type":"ContainerStarted","Data":"a9b2d1bdfc00960c1277ac3675fdbc8ea83aba580fd61d49df5a38405ce8264c"}
Oct 30 20:09:51.933088 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:51.933020    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-rbwvb" podStartSLOduration=0.933001114 podStartE2EDuration="933.001114ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:09:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:09:51.930022481 +0000 UTC m=+2267.258140636" watchObservedRunningTime="2026-10-30 20:09:51.933001114 +0000 UTC m=+2267.261119269"
Oct 30 20:09:52.463443 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:52.463409    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-rbwvb"
Oct 30 20:09:52.465251 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:52.465224    6996 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5db77fbd9f-rbwvb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.167:15021/healthz/ready\": dial tcp 10.134.0.167:15021: connect: connection refused" start-of-body=
Oct 30 20:09:52.465375 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:52.465288    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-rbwvb" podUID="9a76e670-c0a4-49ee-9b5f-7fabddaef0e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.167:15021/healthz/ready\": dial tcp 10.134.0.167:15021: connect: connection refused"
Oct 30 20:09:53.464133 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:53.464093    6996 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5db77fbd9f-rbwvb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.167:15021/healthz/ready\": dial tcp 10.134.0.167:15021: connect: connection refused" start-of-body=
Oct 30 20:09:53.464586 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:53.464172    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-rbwvb" podUID="9a76e670-c0a4-49ee-9b5f-7fabddaef0e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.167:15021/healthz/ready\": dial tcp 10.134.0.167:15021: connect: connection refused"
Oct 30 20:09:54.467502 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:54.466989    6996 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5db77fbd9f-rbwvb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.167:15021/healthz/ready\": dial tcp 10.134.0.167:15021: connect: connection refused" start-of-body=
Oct 30 20:09:54.467502 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:54.467075    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-rbwvb" podUID="9a76e670-c0a4-49ee-9b5f-7fabddaef0e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.167:15021/healthz/ready\": dial tcp 10.134.0.167:15021: connect: connection refused"
Oct 30 20:09:55.463996 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:55.463954    6996 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5db77fbd9f-rbwvb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.167:15021/healthz/ready\": dial tcp 10.134.0.167:15021: connect: connection refused" start-of-body=
Oct 30 20:09:55.464175 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:55.464024    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-rbwvb" podUID="9a76e670-c0a4-49ee-9b5f-7fabddaef0e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.167:15021/healthz/ready\": dial tcp 10.134.0.167:15021: connect: connection refused"
Oct 30 20:09:56.463999 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:56.463961    6996 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5db77fbd9f-rbwvb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.167:15021/healthz/ready\": dial tcp 10.134.0.167:15021: connect: connection refused" start-of-body=
Oct 30 20:09:56.464445 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:56.464054    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-rbwvb" podUID="9a76e670-c0a4-49ee-9b5f-7fabddaef0e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.167:15021/healthz/ready\": dial tcp 10.134.0.167:15021: connect: connection refused"
Oct 30 20:09:57.465739 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:57.465702    6996 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5db77fbd9f-rbwvb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.167:15021/healthz/ready\": dial tcp 10.134.0.167:15021: connect: connection refused" start-of-body=
Oct 30 20:09:57.466398 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:57.466353    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-rbwvb" podUID="9a76e670-c0a4-49ee-9b5f-7fabddaef0e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.167:15021/healthz/ready\": dial tcp 10.134.0.167:15021: connect: connection refused"
Oct 30 20:09:58.463763 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:58.463719    6996 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5db77fbd9f-rbwvb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.167:15021/healthz/ready\": dial tcp 10.134.0.167:15021: connect: connection refused" start-of-body=
Oct 30 20:09:58.463971 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:58.463803    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-rbwvb" podUID="9a76e670-c0a4-49ee-9b5f-7fabddaef0e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.167:15021/healthz/ready\": dial tcp 10.134.0.167:15021: connect: connection refused"
Oct 30 20:09:59.464457 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:59.464412    6996 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5db77fbd9f-rbwvb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.167:15021/healthz/ready\": dial tcp 10.134.0.167:15021: connect: connection refused" start-of-body=
Oct 30 20:09:59.464937 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:09:59.464508    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-rbwvb" podUID="9a76e670-c0a4-49ee-9b5f-7fabddaef0e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.167:15021/healthz/ready\": dial tcp 10.134.0.167:15021: connect: connection refused"
Oct 30 20:10:01.017140 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:10:01.017106    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-rbwvb"
Oct 30 20:10:01.018245 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:10:01.018067    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-rbwvb"
Oct 30 20:10:01.019180 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:10:01.019159    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-rbwvb"
Oct 30 20:10:01.194758 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:10:01.194718    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 20:10:24.084725 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:10:24.084690    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 20:10:31.192305 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:10:31.192268    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 20:10:50.890988 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:10:50.890954    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-extmod-b-d97cc8/payload-processing-external-model-e2e-extmod-b-d97cc8-7f949lwfc"]
Oct 30 20:10:50.895647 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:10:50.895616    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-d97cc8/payload-processing-external-model-e2e-extmod-b-d97cc8-7f949lwfc"
Oct 30 20:10:50.905802 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:10:50.905769    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-d97cc8\"/\"agc-cred-6e85945c923c\""
Oct 30 20:10:50.907139 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:10:50.907117    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-d97cc8\"/\"openshift-service-ca.crt\""
Oct 30 20:10:50.907327 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:10:50.907303    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-d97cc8\"/\"routing-overlay\""
Oct 30 20:10:50.907458 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:10:50.907376    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-d97cc8\"/\"payload-processing-external-model-plugins-e2e-extmod-b-d97cc8\""
Oct 30 20:10:50.907458 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:10:50.907451    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-d97cc8\"/\"kube-root-ca.crt\""
Oct 30 20:10:50.909930 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:10:50.909901    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-d97cc8/payload-processing-external-model-e2e-extmod-b-d97cc8-7f949lwfc"]
Oct 30 20:10:51.002262 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:10:51.002224    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/799847f8-0366-4fba-a5a6-3e90bb4bba55-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-d97cc8-7f949lwfc\" (UID: \"799847f8-0366-4fba-a5a6-3e90bb4bba55\") " pod="ai-tenant-e2e-extmod-b-d97cc8/payload-processing-external-model-e2e-extmod-b-d97cc8-7f949lwfc"
Oct 30 20:10:51.002459 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:10:51.002300    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzfxg\" (UniqueName: \"kubernetes.io/projected/799847f8-0366-4fba-a5a6-3e90bb4bba55-kube-api-access-bzfxg\") pod \"payload-processing-external-model-e2e-extmod-b-d97cc8-7f949lwfc\" (UID: \"799847f8-0366-4fba-a5a6-3e90bb4bba55\") " pod="ai-tenant-e2e-extmod-b-d97cc8/payload-processing-external-model-e2e-extmod-b-d97cc8-7f949lwfc"
Oct 30 20:10:51.002459 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:10:51.002443    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/799847f8-0366-4fba-a5a6-3e90bb4bba55-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-d97cc8-7f949lwfc\" (UID: \"799847f8-0366-4fba-a5a6-3e90bb4bba55\") " pod="ai-tenant-e2e-extmod-b-d97cc8/payload-processing-external-model-e2e-extmod-b-d97cc8-7f949lwfc"
Oct 30 20:10:51.002653 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:10:51.002529    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/799847f8-0366-4fba-a5a6-3e90bb4bba55-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-d97cc8-7f949lwfc\" (UID: \"799847f8-0366-4fba-a5a6-3e90bb4bba55\") " pod="ai-tenant-e2e-extmod-b-d97cc8/payload-processing-external-model-e2e-extmod-b-d97cc8-7f949lwfc"
Oct 30 20:10:51.111001 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:10:51.107457    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzfxg\" (UniqueName: \"kubernetes.io/projected/799847f8-0366-4fba-a5a6-3e90bb4bba55-kube-api-access-bzfxg\") pod \"payload-processing-external-model-e2e-extmod-b-d97cc8-7f949lwfc\" (UID: \"799847f8-0366-4fba-a5a6-3e90bb4bba55\") " pod="ai-tenant-e2e-extmod-b-d97cc8/payload-processing-external-model-e2e-extmod-b-d97cc8-7f949lwfc"
Oct 30 20:10:51.111001 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:10:51.107661    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/799847f8-0366-4fba-a5a6-3e90bb4bba55-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-d97cc8-7f949lwfc\" (UID: \"799847f8-0366-4fba-a5a6-3e90bb4bba55\") " pod="ai-tenant-e2e-extmod-b-d97cc8/payload-processing-external-model-e2e-extmod-b-d97cc8-7f949lwfc"
Oct 30 20:10:51.111001 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:10:51.107760    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/799847f8-0366-4fba-a5a6-3e90bb4bba55-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-d97cc8-7f949lwfc\" (UID: \"799847f8-0366-4fba-a5a6-3e90bb4bba55\") " pod="ai-tenant-e2e-extmod-b-d97cc8/payload-processing-external-model-e2e-extmod-b-d97cc8-7f949lwfc"
Oct 30 20:10:51.111001 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:10:51.107904    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/799847f8-0366-4fba-a5a6-3e90bb4bba55-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-d97cc8-7f949lwfc\" (UID: \"799847f8-0366-4fba-a5a6-3e90bb4bba55\") " pod="ai-tenant-e2e-extmod-b-d97cc8/payload-processing-external-model-e2e-extmod-b-d97cc8-7f949lwfc"
Oct 30 20:10:51.111001 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:10:51.108965    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/799847f8-0366-4fba-a5a6-3e90bb4bba55-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-d97cc8-7f949lwfc\" (UID: \"799847f8-0366-4fba-a5a6-3e90bb4bba55\") " pod="ai-tenant-e2e-extmod-b-d97cc8/payload-processing-external-model-e2e-extmod-b-d97cc8-7f949lwfc"
Oct 30 20:10:51.125511 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:10:51.111977    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/799847f8-0366-4fba-a5a6-3e90bb4bba55-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-d97cc8-7f949lwfc\" (UID: \"799847f8-0366-4fba-a5a6-3e90bb4bba55\") " pod="ai-tenant-e2e-extmod-b-d97cc8/payload-processing-external-model-e2e-extmod-b-d97cc8-7f949lwfc"
Oct 30 20:10:51.156541 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:10:51.153058    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzfxg\" (UniqueName: \"kubernetes.io/projected/799847f8-0366-4fba-a5a6-3e90bb4bba55-kube-api-access-bzfxg\") pod \"payload-processing-external-model-e2e-extmod-b-d97cc8-7f949lwfc\" (UID: \"799847f8-0366-4fba-a5a6-3e90bb4bba55\") " pod="ai-tenant-e2e-extmod-b-d97cc8/payload-processing-external-model-e2e-extmod-b-d97cc8-7f949lwfc"
Oct 30 20:10:51.188505 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:10:51.177433    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/799847f8-0366-4fba-a5a6-3e90bb4bba55-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-d97cc8-7f949lwfc\" (UID: \"799847f8-0366-4fba-a5a6-3e90bb4bba55\") " pod="ai-tenant-e2e-extmod-b-d97cc8/payload-processing-external-model-e2e-extmod-b-d97cc8-7f949lwfc"
Oct 30 20:10:51.220315 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:10:51.219748    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-d97cc8/payload-processing-external-model-e2e-extmod-b-d97cc8-7f949lwfc"
Oct 30 20:10:51.574285 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:10:51.574254    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-d97cc8/payload-processing-external-model-e2e-extmod-b-d97cc8-7f949lwfc"]
Oct 30 20:10:51.575318 ip-10-0-141-42 kubenswrapper[6996]: W1030 20:10:51.575293    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod799847f8_0366_4fba_a5a6_3e90bb4bba55.slice/crio-c861e40e16da7687703ff9b4566b515c6fcc1e9ef7b5eb309dc172706c49541f WatchSource:0}: Error finding container c861e40e16da7687703ff9b4566b515c6fcc1e9ef7b5eb309dc172706c49541f: Status 404 returned error can't find the container with id c861e40e16da7687703ff9b4566b515c6fcc1e9ef7b5eb309dc172706c49541f
Oct 30 20:10:52.544620 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:10:52.544576    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-d97cc8/payload-processing-external-model-e2e-extmod-b-d97cc8-7f949lwfc" event={"ID":"799847f8-0366-4fba-a5a6-3e90bb4bba55","Type":"ContainerStarted","Data":"4758e863ff72bd301eb0cef23b11a6a32d08660760123276fc5491891e39d18c"}
Oct 30 20:10:52.545021 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:10:52.544646    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-d97cc8/payload-processing-external-model-e2e-extmod-b-d97cc8-7f949lwfc" event={"ID":"799847f8-0366-4fba-a5a6-3e90bb4bba55","Type":"ContainerStarted","Data":"c861e40e16da7687703ff9b4566b515c6fcc1e9ef7b5eb309dc172706c49541f"}
Oct 30 20:10:52.545021 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:10:52.544673    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-extmod-b-d97cc8/payload-processing-external-model-e2e-extmod-b-d97cc8-7f949lwfc"
Oct 30 20:10:52.563553 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:10:52.563442    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-extmod-b-d97cc8/payload-processing-external-model-e2e-extmod-b-d97cc8-7f949lwfc" podStartSLOduration=2.563424681 podStartE2EDuration="2.563424681s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:10:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:10:52.562580705 +0000 UTC m=+2327.890698872" watchObservedRunningTime="2026-10-30 20:10:52.563424681 +0000 UTC m=+2327.891542872"
Oct 30 20:11:01.193775 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:11:01.193736    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 20:11:03.556652 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:11:03.556616    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-extmod-b-d97cc8/payload-processing-external-model-e2e-extmod-b-d97cc8-7f949lwfc"
Oct 30 20:11:05.485162 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:11:05.485122    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 20:11:31.196856 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:11:31.196817    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 20:11:53.860004 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:11:53.859966    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 20:12:01.192572 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:12:01.192525    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 20:12:08.492242 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:12:08.492195    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-d97cc8/payload-processing-external-model-e2e-extmod-b-d97cc8-7f949lwfc"]
Oct 30 20:12:13.493107 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:12:13.493056    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-extmod-b-d97cc8/payload-processing-external-model-e2e-extmod-b-d97cc8-7f949lwfc" podUID="799847f8-0366-4fba-a5a6-3e90bb4bba55" containerName="payload-processing" containerID="cri-o://4758e863ff72bd301eb0cef23b11a6a32d08660760123276fc5491891e39d18c" gracePeriod=25
Oct 30 20:12:13.554214 ip-10-0-141-42 kubenswrapper[6996]: W1030 20:12:13.554182    6996 logging.go:55] [core] [Channel #4247 SubChannel #4248]grpc: addrConn.createTransport failed to connect to {Addr: "10.134.0.168:50052", ServerName: "10.134.0.168:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.134.0.168:50052: connect: connection refused"
Oct 30 20:12:13.799334 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:12:13.799298    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-d97cc8/payload-processing-external-model-e2e-extmod-b-d97cc8-7f949lwfc"
Oct 30 20:12:13.930813 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:12:13.930764    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/799847f8-0366-4fba-a5a6-3e90bb4bba55-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/799847f8-0366-4fba-a5a6-3e90bb4bba55-routing-overlay-volume\") pod \"799847f8-0366-4fba-a5a6-3e90bb4bba55\" (UID: \"799847f8-0366-4fba-a5a6-3e90bb4bba55\") "
Oct 30 20:12:13.931039 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:12:13.931026    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/799847f8-0366-4fba-a5a6-3e90bb4bba55-provider-credentials\" (UniqueName: \"kubernetes.io/projected/799847f8-0366-4fba-a5a6-3e90bb4bba55-provider-credentials\") pod \"799847f8-0366-4fba-a5a6-3e90bb4bba55\" (UID: \"799847f8-0366-4fba-a5a6-3e90bb4bba55\") "
Oct 30 20:12:13.931087 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:12:13.931052    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/799847f8-0366-4fba-a5a6-3e90bb4bba55-kube-api-access-bzfxg\" (UniqueName: \"kubernetes.io/projected/799847f8-0366-4fba-a5a6-3e90bb4bba55-kube-api-access-bzfxg\") pod \"799847f8-0366-4fba-a5a6-3e90bb4bba55\" (UID: \"799847f8-0366-4fba-a5a6-3e90bb4bba55\") "
Oct 30 20:12:13.931169 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:12:13.931158    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/799847f8-0366-4fba-a5a6-3e90bb4bba55-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/799847f8-0366-4fba-a5a6-3e90bb4bba55-plugins-config-volume\") pod \"799847f8-0366-4fba-a5a6-3e90bb4bba55\" (UID: \"799847f8-0366-4fba-a5a6-3e90bb4bba55\") "
Oct 30 20:12:13.931226 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:12:13.931191    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/799847f8-0366-4fba-a5a6-3e90bb4bba55-routing-overlay-volume" pod "799847f8-0366-4fba-a5a6-3e90bb4bba55" (UID: "799847f8-0366-4fba-a5a6-3e90bb4bba55"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:12:13.931668 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:12:13.931640    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/799847f8-0366-4fba-a5a6-3e90bb4bba55-plugins-config-volume" pod "799847f8-0366-4fba-a5a6-3e90bb4bba55" (UID: "799847f8-0366-4fba-a5a6-3e90bb4bba55"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:12:13.931796 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:12:13.931744    6996 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/799847f8-0366-4fba-a5a6-3e90bb4bba55-routing-overlay-volume\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:12:13.931796 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:12:13.931758    6996 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/799847f8-0366-4fba-a5a6-3e90bb4bba55-plugins-config-volume\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:12:13.933403 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:12:13.933385    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/799847f8-0366-4fba-a5a6-3e90bb4bba55-provider-credentials" pod "799847f8-0366-4fba-a5a6-3e90bb4bba55" (UID: "799847f8-0366-4fba-a5a6-3e90bb4bba55"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:12:13.933508 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:12:13.933409    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/799847f8-0366-4fba-a5a6-3e90bb4bba55-kube-api-access-bzfxg" pod "799847f8-0366-4fba-a5a6-3e90bb4bba55" (UID: "799847f8-0366-4fba-a5a6-3e90bb4bba55"). InnerVolumeSpecName "kube-api-access-bzfxg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:12:14.032967 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:12:14.032876    6996 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/799847f8-0366-4fba-a5a6-3e90bb4bba55-provider-credentials\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:12:14.032967 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:12:14.032909    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bzfxg\" (UniqueName: \"kubernetes.io/projected/799847f8-0366-4fba-a5a6-3e90bb4bba55-kube-api-access-bzfxg\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:12:14.253707 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:12:14.253669    6996 generic.go:356] "Generic (PLEG): container finished" podID="799847f8-0366-4fba-a5a6-3e90bb4bba55" containerID="4758e863ff72bd301eb0cef23b11a6a32d08660760123276fc5491891e39d18c" exitCode=0
Oct 30 20:12:14.253899 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:12:14.253726    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-d97cc8/payload-processing-external-model-e2e-extmod-b-d97cc8-7f949lwfc"
Oct 30 20:12:14.253899 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:12:14.253761    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-d97cc8/payload-processing-external-model-e2e-extmod-b-d97cc8-7f949lwfc" event={"ID":"799847f8-0366-4fba-a5a6-3e90bb4bba55","Type":"ContainerDied","Data":"4758e863ff72bd301eb0cef23b11a6a32d08660760123276fc5491891e39d18c"}
Oct 30 20:12:14.253899 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:12:14.253820    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-d97cc8/payload-processing-external-model-e2e-extmod-b-d97cc8-7f949lwfc" event={"ID":"799847f8-0366-4fba-a5a6-3e90bb4bba55","Type":"ContainerDied","Data":"c861e40e16da7687703ff9b4566b515c6fcc1e9ef7b5eb309dc172706c49541f"}
Oct 30 20:12:14.253899 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:12:14.253842    6996 scope.go:122] "RemoveContainer" containerID="4758e863ff72bd301eb0cef23b11a6a32d08660760123276fc5491891e39d18c"
Oct 30 20:12:14.266429 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:12:14.266400    6996 scope.go:122] "RemoveContainer" containerID="4758e863ff72bd301eb0cef23b11a6a32d08660760123276fc5491891e39d18c"
Oct 30 20:12:14.266803 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:12:14.266781    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4758e863ff72bd301eb0cef23b11a6a32d08660760123276fc5491891e39d18c\": container with ID starting with 4758e863ff72bd301eb0cef23b11a6a32d08660760123276fc5491891e39d18c not found: ID does not exist" containerID="4758e863ff72bd301eb0cef23b11a6a32d08660760123276fc5491891e39d18c"
Oct 30 20:12:14.266858 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:12:14.266818    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4758e863ff72bd301eb0cef23b11a6a32d08660760123276fc5491891e39d18c"} err="failed to get container status \"4758e863ff72bd301eb0cef23b11a6a32d08660760123276fc5491891e39d18c\": rpc error: code = NotFound desc = could not find container \"4758e863ff72bd301eb0cef23b11a6a32d08660760123276fc5491891e39d18c\": container with ID starting with 4758e863ff72bd301eb0cef23b11a6a32d08660760123276fc5491891e39d18c not found: ID does not exist"
Oct 30 20:12:14.281565 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:12:14.281526    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-d97cc8/payload-processing-external-model-e2e-extmod-b-d97cc8-7f949lwfc"]
Oct 30 20:12:14.285300 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:12:14.285242    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-extmod-b-d97cc8/payload-processing-external-model-e2e-extmod-b-d97cc8-7f949lwfc"]
Oct 30 20:12:14.554450 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:12:14.554335    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-extmod-b-d97cc8/payload-processing-external-model-e2e-extmod-b-d97cc8-7f949lwfc" podUID="799847f8-0366-4fba-a5a6-3e90bb4bba55" containerName="payload-processing" probeResult="failure" output="timeout: failed to connect service \"10.134.0.168:50052\" within 1s: context deadline exceeded"
Oct 30 20:12:14.793876 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:12:14.793829    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="799847f8-0366-4fba-a5a6-3e90bb4bba55" path="/var/lib/kubelet/pods/799847f8-0366-4fba-a5a6-3e90bb4bba55/volumes"
Oct 30 20:12:15.266759 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:12:15.266722    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr"]
Oct 30 20:12:15.266972 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:12:15.266943    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr" podUID="bae90963-7525-42cb-9cdc-fdb32da947e7" containerName="istio-proxy" containerID="cri-o://c1a35430b4f4cd4dde26d39e3d10c40bb8be21277769637a7e0e5d051a39ff28" gracePeriod=30
Oct 30 20:12:20.592813 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:12:20.592785    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr"
Oct 30 20:12:20.761500 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:12:20.761443    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bae90963-7525-42cb-9cdc-fdb32da947e7-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bae90963-7525-42cb-9cdc-fdb32da947e7-workload-socket\") pod \"bae90963-7525-42cb-9cdc-fdb32da947e7\" (UID: \"bae90963-7525-42cb-9cdc-fdb32da947e7\") "
Oct 30 20:12:20.762107 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:12:20.762080    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bae90963-7525-42cb-9cdc-fdb32da947e7-kube-api-access-5cpsl\" (UniqueName: \"kubernetes.io/projected/bae90963-7525-42cb-9cdc-fdb32da947e7-kube-api-access-5cpsl\") pod \"bae90963-7525-42cb-9cdc-fdb32da947e7\" (UID: \"bae90963-7525-42cb-9cdc-fdb32da947e7\") "
Oct 30 20:12:20.762309 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:12:20.762293    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bae90963-7525-42cb-9cdc-fdb32da947e7-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bae90963-7525-42cb-9cdc-fdb32da947e7-credential-socket\") pod \"bae90963-7525-42cb-9cdc-fdb32da947e7\" (UID: \"bae90963-7525-42cb-9cdc-fdb32da947e7\") "
Oct 30 20:12:20.762541 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:12:20.762524    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bae90963-7525-42cb-9cdc-fdb32da947e7-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bae90963-7525-42cb-9cdc-fdb32da947e7-istio-data\") pod \"bae90963-7525-42cb-9cdc-fdb32da947e7\" (UID: \"bae90963-7525-42cb-9cdc-fdb32da947e7\") "
Oct 30 20:12:20.762674 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:12:20.762659    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bae90963-7525-42cb-9cdc-fdb32da947e7-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bae90963-7525-42cb-9cdc-fdb32da947e7-istio-envoy\") pod \"bae90963-7525-42cb-9cdc-fdb32da947e7\" (UID: \"bae90963-7525-42cb-9cdc-fdb32da947e7\") "
Oct 30 20:12:20.762789 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:12:20.762776    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bae90963-7525-42cb-9cdc-fdb32da947e7-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bae90963-7525-42cb-9cdc-fdb32da947e7-istiod-ca-cert\") pod \"bae90963-7525-42cb-9cdc-fdb32da947e7\" (UID: \"bae90963-7525-42cb-9cdc-fdb32da947e7\") "
Oct 30 20:12:20.762904 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:12:20.762881    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/bae90963-7525-42cb-9cdc-fdb32da947e7-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bae90963-7525-42cb-9cdc-fdb32da947e7-istio-podinfo\") pod \"bae90963-7525-42cb-9cdc-fdb32da947e7\" (UID: \"bae90963-7525-42cb-9cdc-fdb32da947e7\") "
Oct 30 20:12:20.763140 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:12:20.763123    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bae90963-7525-42cb-9cdc-fdb32da947e7-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bae90963-7525-42cb-9cdc-fdb32da947e7-workload-certs\") pod \"bae90963-7525-42cb-9cdc-fdb32da947e7\" (UID: \"bae90963-7525-42cb-9cdc-fdb32da947e7\") "
Oct 30 20:12:20.763262 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:12:20.763245    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bae90963-7525-42cb-9cdc-fdb32da947e7-istio-token\" (UniqueName: \"kubernetes.io/projected/bae90963-7525-42cb-9cdc-fdb32da947e7-istio-token\") pod \"bae90963-7525-42cb-9cdc-fdb32da947e7\" (UID: \"bae90963-7525-42cb-9cdc-fdb32da947e7\") "
Oct 30 20:12:20.764873 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:12:20.761968    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bae90963-7525-42cb-9cdc-fdb32da947e7-workload-socket" pod "bae90963-7525-42cb-9cdc-fdb32da947e7" (UID: "bae90963-7525-42cb-9cdc-fdb32da947e7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:12:20.764873 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:12:20.764572    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bae90963-7525-42cb-9cdc-fdb32da947e7-istio-data" pod "bae90963-7525-42cb-9cdc-fdb32da947e7" (UID: "bae90963-7525-42cb-9cdc-fdb32da947e7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:12:20.779875 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:12:20.779766    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bae90963-7525-42cb-9cdc-fdb32da947e7-workload-certs" pod "bae90963-7525-42cb-9cdc-fdb32da947e7" (UID: "bae90963-7525-42cb-9cdc-fdb32da947e7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:12:20.779875 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:12:20.779850    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bae90963-7525-42cb-9cdc-fdb32da947e7-credential-socket" pod "bae90963-7525-42cb-9cdc-fdb32da947e7" (UID: "bae90963-7525-42cb-9cdc-fdb32da947e7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:12:20.780284 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:12:20.780248    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bae90963-7525-42cb-9cdc-fdb32da947e7-istiod-ca-cert" pod "bae90963-7525-42cb-9cdc-fdb32da947e7" (UID: "bae90963-7525-42cb-9cdc-fdb32da947e7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:12:20.827627 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:12:20.793816    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bae90963-7525-42cb-9cdc-fdb32da947e7-kube-api-access-5cpsl" pod "bae90963-7525-42cb-9cdc-fdb32da947e7" (UID: "bae90963-7525-42cb-9cdc-fdb32da947e7"). InnerVolumeSpecName "kube-api-access-5cpsl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:12:20.827627 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:12:20.794355    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bae90963-7525-42cb-9cdc-fdb32da947e7-istio-token" pod "bae90963-7525-42cb-9cdc-fdb32da947e7" (UID: "bae90963-7525-42cb-9cdc-fdb32da947e7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:12:20.827627 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:12:20.799460    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bae90963-7525-42cb-9cdc-fdb32da947e7-istio-podinfo" pod "bae90963-7525-42cb-9cdc-fdb32da947e7" (UID: "bae90963-7525-42cb-9cdc-fdb32da947e7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 20:12:20.827627 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:12:20.805536    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bae90963-7525-42cb-9cdc-fdb32da947e7-istio-envoy" pod "bae90963-7525-42cb-9cdc-fdb32da947e7" (UID: "bae90963-7525-42cb-9cdc-fdb32da947e7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:12:20.869989 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:12:20.868788    6996 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bae90963-7525-42cb-9cdc-fdb32da947e7-workload-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:12:20.869989 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:12:20.868820    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5cpsl\" (UniqueName: \"kubernetes.io/projected/bae90963-7525-42cb-9cdc-fdb32da947e7-kube-api-access-5cpsl\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:12:20.869989 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:12:20.868837    6996 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bae90963-7525-42cb-9cdc-fdb32da947e7-credential-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:12:20.869989 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:12:20.868851    6996 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bae90963-7525-42cb-9cdc-fdb32da947e7-istio-data\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:12:20.869989 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:12:20.868865    6996 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bae90963-7525-42cb-9cdc-fdb32da947e7-istio-envoy\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:12:20.869989 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:12:20.868878    6996 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bae90963-7525-42cb-9cdc-fdb32da947e7-istiod-ca-cert\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:12:20.869989 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:12:20.868891    6996 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bae90963-7525-42cb-9cdc-fdb32da947e7-istio-podinfo\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:12:20.869989 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:12:20.868904    6996 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bae90963-7525-42cb-9cdc-fdb32da947e7-workload-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:12:20.869989 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:12:20.868917    6996 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bae90963-7525-42cb-9cdc-fdb32da947e7-istio-token\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:12:21.327347 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:12:21.327077    6996 generic.go:356] "Generic (PLEG): container finished" podID="bae90963-7525-42cb-9cdc-fdb32da947e7" containerID="c1a35430b4f4cd4dde26d39e3d10c40bb8be21277769637a7e0e5d051a39ff28" exitCode=0
Oct 30 20:12:21.327347 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:12:21.327118    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr" event={"ID":"bae90963-7525-42cb-9cdc-fdb32da947e7","Type":"ContainerDied","Data":"c1a35430b4f4cd4dde26d39e3d10c40bb8be21277769637a7e0e5d051a39ff28"}
Oct 30 20:12:21.327347 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:12:21.327201    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr" event={"ID":"bae90963-7525-42cb-9cdc-fdb32da947e7","Type":"ContainerDied","Data":"3f2441b9cc308c9278d7868cd6fca2d1c73cb64b61b706519320b83fae024770"}
Oct 30 20:12:21.327347 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:12:21.327233    6996 scope.go:122] "RemoveContainer" containerID="c1a35430b4f4cd4dde26d39e3d10c40bb8be21277769637a7e0e5d051a39ff28"
Oct 30 20:12:21.327753 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:12:21.327468    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr"
Oct 30 20:12:21.342193 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:12:21.342170    6996 scope.go:122] "RemoveContainer" containerID="c1a35430b4f4cd4dde26d39e3d10c40bb8be21277769637a7e0e5d051a39ff28"
Oct 30 20:12:21.342520 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:12:21.342502    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1a35430b4f4cd4dde26d39e3d10c40bb8be21277769637a7e0e5d051a39ff28\": container with ID starting with c1a35430b4f4cd4dde26d39e3d10c40bb8be21277769637a7e0e5d051a39ff28 not found: ID does not exist" containerID="c1a35430b4f4cd4dde26d39e3d10c40bb8be21277769637a7e0e5d051a39ff28"
Oct 30 20:12:21.342582 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:12:21.342529    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1a35430b4f4cd4dde26d39e3d10c40bb8be21277769637a7e0e5d051a39ff28"} err="failed to get container status \"c1a35430b4f4cd4dde26d39e3d10c40bb8be21277769637a7e0e5d051a39ff28\": rpc error: code = NotFound desc = could not find container \"c1a35430b4f4cd4dde26d39e3d10c40bb8be21277769637a7e0e5d051a39ff28\": container with ID starting with c1a35430b4f4cd4dde26d39e3d10c40bb8be21277769637a7e0e5d051a39ff28 not found: ID does not exist"
Oct 30 20:12:21.356498 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:12:21.356445    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr"]
Oct 30 20:12:21.360221 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:12:21.360192    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-d7rkr"]
Oct 30 20:12:22.793612 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:12:22.793579    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bae90963-7525-42cb-9cdc-fdb32da947e7" path="/var/lib/kubelet/pods/bae90963-7525-42cb-9cdc-fdb32da947e7/volumes"
Oct 30 20:12:31.215712 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:12:31.215671    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 20:12:33.995165 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:12:33.995115    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 20:13:01.191691 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:13:01.191609    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 20:13:24.083204 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:13:24.083159    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 20:13:31.192822 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:13:31.192782    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 20:14:01.192646 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:14:01.192609    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 20:14:01.548439 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:01.548349    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 20:14:06.982599 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:06.982556    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs"]
Oct 30 20:14:06.983191 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:06.982828    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs" podUID="6d58ffd8-ffb2-4751-a622-11b584616c7d" containerName="istio-proxy" containerID="cri-o://417c6f89ccf751106408d11b66bea18d9e8cb16687401c3ba4e55f1386ae5eb5" gracePeriod=30
Oct 30 20:14:11.385714 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:11.385680    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr"]
Oct 30 20:14:11.386138 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:11.385896    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr" podUID="0ea6d4d1-f271-426c-862e-a2a43ab41f0a" containerName="istio-proxy" containerID="cri-o://79c4dc32987f4d6fe67b5f06a5c695b2fd82909a0901712e09bb023fc681e06e" gracePeriod=30
Oct 30 20:14:12.297687 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:12.297656    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs"
Oct 30 20:14:12.299945 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:12.299916    6996 generic.go:356] "Generic (PLEG): container finished" podID="6d58ffd8-ffb2-4751-a622-11b584616c7d" containerID="417c6f89ccf751106408d11b66bea18d9e8cb16687401c3ba4e55f1386ae5eb5" exitCode=0
Oct 30 20:14:12.300096 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:12.299961    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs"
Oct 30 20:14:12.300096 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:12.300002    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs" event={"ID":"6d58ffd8-ffb2-4751-a622-11b584616c7d","Type":"ContainerDied","Data":"417c6f89ccf751106408d11b66bea18d9e8cb16687401c3ba4e55f1386ae5eb5"}
Oct 30 20:14:12.300096 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:12.300050    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs" event={"ID":"6d58ffd8-ffb2-4751-a622-11b584616c7d","Type":"ContainerDied","Data":"5b4c72b8383048a488d9cb7d01a92724fc355c8a00443b89b65cab530e487f4a"}
Oct 30 20:14:12.300096 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:12.300073    6996 scope.go:122] "RemoveContainer" containerID="417c6f89ccf751106408d11b66bea18d9e8cb16687401c3ba4e55f1386ae5eb5"
Oct 30 20:14:12.313116 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:12.313086    6996 scope.go:122] "RemoveContainer" containerID="417c6f89ccf751106408d11b66bea18d9e8cb16687401c3ba4e55f1386ae5eb5"
Oct 30 20:14:12.313611 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:14:12.313580    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"417c6f89ccf751106408d11b66bea18d9e8cb16687401c3ba4e55f1386ae5eb5\": container with ID starting with 417c6f89ccf751106408d11b66bea18d9e8cb16687401c3ba4e55f1386ae5eb5 not found: ID does not exist" containerID="417c6f89ccf751106408d11b66bea18d9e8cb16687401c3ba4e55f1386ae5eb5"
Oct 30 20:14:12.313758 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:12.313613    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"417c6f89ccf751106408d11b66bea18d9e8cb16687401c3ba4e55f1386ae5eb5"} err="failed to get container status \"417c6f89ccf751106408d11b66bea18d9e8cb16687401c3ba4e55f1386ae5eb5\": rpc error: code = NotFound desc = could not find container \"417c6f89ccf751106408d11b66bea18d9e8cb16687401c3ba4e55f1386ae5eb5\": container with ID starting with 417c6f89ccf751106408d11b66bea18d9e8cb16687401c3ba4e55f1386ae5eb5 not found: ID does not exist"
Oct 30 20:14:12.382582 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:12.382542    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6d58ffd8-ffb2-4751-a622-11b584616c7d-kube-api-access-tjntn\" (UniqueName: \"kubernetes.io/projected/6d58ffd8-ffb2-4751-a622-11b584616c7d-kube-api-access-tjntn\") pod \"6d58ffd8-ffb2-4751-a622-11b584616c7d\" (UID: \"6d58ffd8-ffb2-4751-a622-11b584616c7d\") "
Oct 30 20:14:12.382748 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:12.382691    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6d58ffd8-ffb2-4751-a622-11b584616c7d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d58ffd8-ffb2-4751-a622-11b584616c7d-workload-socket\") pod \"6d58ffd8-ffb2-4751-a622-11b584616c7d\" (UID: \"6d58ffd8-ffb2-4751-a622-11b584616c7d\") "
Oct 30 20:14:12.382748 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:12.382736    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6d58ffd8-ffb2-4751-a622-11b584616c7d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d58ffd8-ffb2-4751-a622-11b584616c7d-workload-certs\") pod \"6d58ffd8-ffb2-4751-a622-11b584616c7d\" (UID: \"6d58ffd8-ffb2-4751-a622-11b584616c7d\") "
Oct 30 20:14:12.382836 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:12.382818    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6d58ffd8-ffb2-4751-a622-11b584616c7d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d58ffd8-ffb2-4751-a622-11b584616c7d-istio-data\") pod \"6d58ffd8-ffb2-4751-a622-11b584616c7d\" (UID: \"6d58ffd8-ffb2-4751-a622-11b584616c7d\") "
Oct 30 20:14:12.382927 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:12.382908    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6d58ffd8-ffb2-4751-a622-11b584616c7d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d58ffd8-ffb2-4751-a622-11b584616c7d-istio-envoy\") pod \"6d58ffd8-ffb2-4751-a622-11b584616c7d\" (UID: \"6d58ffd8-ffb2-4751-a622-11b584616c7d\") "
Oct 30 20:14:12.383014 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:12.382945    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6d58ffd8-ffb2-4751-a622-11b584616c7d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d58ffd8-ffb2-4751-a622-11b584616c7d-istiod-ca-cert\") pod \"6d58ffd8-ffb2-4751-a622-11b584616c7d\" (UID: \"6d58ffd8-ffb2-4751-a622-11b584616c7d\") "
Oct 30 20:14:12.383077 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:12.383030    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6d58ffd8-ffb2-4751-a622-11b584616c7d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d58ffd8-ffb2-4751-a622-11b584616c7d-istio-podinfo\") pod \"6d58ffd8-ffb2-4751-a622-11b584616c7d\" (UID: \"6d58ffd8-ffb2-4751-a622-11b584616c7d\") "
Oct 30 20:14:12.383077 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:12.383049    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d58ffd8-ffb2-4751-a622-11b584616c7d-workload-socket" pod "6d58ffd8-ffb2-4751-a622-11b584616c7d" (UID: "6d58ffd8-ffb2-4751-a622-11b584616c7d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:14:12.383189 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:12.383089    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d58ffd8-ffb2-4751-a622-11b584616c7d-istio-data" pod "6d58ffd8-ffb2-4751-a622-11b584616c7d" (UID: "6d58ffd8-ffb2-4751-a622-11b584616c7d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:14:12.383189 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:12.383144    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d58ffd8-ffb2-4751-a622-11b584616c7d-workload-certs" pod "6d58ffd8-ffb2-4751-a622-11b584616c7d" (UID: "6d58ffd8-ffb2-4751-a622-11b584616c7d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:14:12.383305 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:12.383206    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6d58ffd8-ffb2-4751-a622-11b584616c7d-istio-token\" (UniqueName: \"kubernetes.io/projected/6d58ffd8-ffb2-4751-a622-11b584616c7d-istio-token\") pod \"6d58ffd8-ffb2-4751-a622-11b584616c7d\" (UID: \"6d58ffd8-ffb2-4751-a622-11b584616c7d\") "
Oct 30 20:14:12.383305 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:12.383280    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6d58ffd8-ffb2-4751-a622-11b584616c7d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d58ffd8-ffb2-4751-a622-11b584616c7d-credential-socket\") pod \"6d58ffd8-ffb2-4751-a622-11b584616c7d\" (UID: \"6d58ffd8-ffb2-4751-a622-11b584616c7d\") "
Oct 30 20:14:12.383436 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:12.383410    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d58ffd8-ffb2-4751-a622-11b584616c7d-istiod-ca-cert" pod "6d58ffd8-ffb2-4751-a622-11b584616c7d" (UID: "6d58ffd8-ffb2-4751-a622-11b584616c7d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:14:12.383747 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:12.383717    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d58ffd8-ffb2-4751-a622-11b584616c7d-credential-socket" pod "6d58ffd8-ffb2-4751-a622-11b584616c7d" (UID: "6d58ffd8-ffb2-4751-a622-11b584616c7d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:14:12.384177 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:12.384148    6996 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d58ffd8-ffb2-4751-a622-11b584616c7d-istiod-ca-cert\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:14:12.384276 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:12.384182    6996 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d58ffd8-ffb2-4751-a622-11b584616c7d-credential-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:14:12.384276 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:12.384198    6996 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d58ffd8-ffb2-4751-a622-11b584616c7d-workload-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:14:12.384276 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:12.384212    6996 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d58ffd8-ffb2-4751-a622-11b584616c7d-workload-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:14:12.384276 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:12.384228    6996 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d58ffd8-ffb2-4751-a622-11b584616c7d-istio-data\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:14:12.385323 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:12.385297    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d58ffd8-ffb2-4751-a622-11b584616c7d-istio-envoy" pod "6d58ffd8-ffb2-4751-a622-11b584616c7d" (UID: "6d58ffd8-ffb2-4751-a622-11b584616c7d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:14:12.385519 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:12.385466    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6d58ffd8-ffb2-4751-a622-11b584616c7d-istio-podinfo" pod "6d58ffd8-ffb2-4751-a622-11b584616c7d" (UID: "6d58ffd8-ffb2-4751-a622-11b584616c7d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 20:14:12.385577 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:12.385522    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d58ffd8-ffb2-4751-a622-11b584616c7d-kube-api-access-tjntn" pod "6d58ffd8-ffb2-4751-a622-11b584616c7d" (UID: "6d58ffd8-ffb2-4751-a622-11b584616c7d"). InnerVolumeSpecName "kube-api-access-tjntn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:14:12.386579 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:12.386553    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d58ffd8-ffb2-4751-a622-11b584616c7d-istio-token" pod "6d58ffd8-ffb2-4751-a622-11b584616c7d" (UID: "6d58ffd8-ffb2-4751-a622-11b584616c7d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:14:12.486099 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:12.486062    6996 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d58ffd8-ffb2-4751-a622-11b584616c7d-istio-envoy\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:14:12.486099 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:12.486093    6996 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d58ffd8-ffb2-4751-a622-11b584616c7d-istio-podinfo\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:14:12.486099 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:12.486103    6996 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d58ffd8-ffb2-4751-a622-11b584616c7d-istio-token\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:14:12.486359 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:12.486113    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tjntn\" (UniqueName: \"kubernetes.io/projected/6d58ffd8-ffb2-4751-a622-11b584616c7d-kube-api-access-tjntn\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:14:12.628149 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:12.628110    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs"]
Oct 30 20:14:12.641759 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:12.641725    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs"]
Oct 30 20:14:12.792749 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:12.792713    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6d58ffd8-ffb2-4751-a622-11b584616c7d" path="/var/lib/kubelet/pods/6d58ffd8-ffb2-4751-a622-11b584616c7d/volumes"
Oct 30 20:14:13.280663 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:13.280627    6996 patch_prober.go:29] interesting pod/e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.145:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 20:14:13.280857 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:13.280688    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w3-779fa3-openshift-default-b556b9b45-swqgs" podUID="6d58ffd8-ffb2-4751-a622-11b584616c7d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.145:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:14:15.457170 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:15.457131    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw"]
Oct 30 20:14:15.457706 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:15.457385    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw" podUID="ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8" containerName="istio-proxy" containerID="cri-o://3794d879be1a019eab9cfaff47c6d5bb19470a96503b33abed996c7ab8f6c536" gracePeriod=30
Oct 30 20:14:16.674276 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:16.673863    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr"
Oct 30 20:14:16.756135 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:16.756093    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ea6d4d1-f271-426c-862e-a2a43ab41f0a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ea6d4d1-f271-426c-862e-a2a43ab41f0a-credential-socket\") pod \"0ea6d4d1-f271-426c-862e-a2a43ab41f0a\" (UID: \"0ea6d4d1-f271-426c-862e-a2a43ab41f0a\") "
Oct 30 20:14:16.756388 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:16.756252    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ea6d4d1-f271-426c-862e-a2a43ab41f0a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ea6d4d1-f271-426c-862e-a2a43ab41f0a-istio-data\") pod \"0ea6d4d1-f271-426c-862e-a2a43ab41f0a\" (UID: \"0ea6d4d1-f271-426c-862e-a2a43ab41f0a\") "
Oct 30 20:14:16.756388 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:16.756292    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ea6d4d1-f271-426c-862e-a2a43ab41f0a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ea6d4d1-f271-426c-862e-a2a43ab41f0a-workload-certs\") pod \"0ea6d4d1-f271-426c-862e-a2a43ab41f0a\" (UID: \"0ea6d4d1-f271-426c-862e-a2a43ab41f0a\") "
Oct 30 20:14:16.756388 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:16.756373    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ea6d4d1-f271-426c-862e-a2a43ab41f0a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ea6d4d1-f271-426c-862e-a2a43ab41f0a-istio-envoy\") pod \"0ea6d4d1-f271-426c-862e-a2a43ab41f0a\" (UID: \"0ea6d4d1-f271-426c-862e-a2a43ab41f0a\") "
Oct 30 20:14:16.756582 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:16.756429    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0ea6d4d1-f271-426c-862e-a2a43ab41f0a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ea6d4d1-f271-426c-862e-a2a43ab41f0a-istiod-ca-cert\") pod \"0ea6d4d1-f271-426c-862e-a2a43ab41f0a\" (UID: \"0ea6d4d1-f271-426c-862e-a2a43ab41f0a\") "
Oct 30 20:14:16.756582 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:16.756466    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ea6d4d1-f271-426c-862e-a2a43ab41f0a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ea6d4d1-f271-426c-862e-a2a43ab41f0a-workload-socket\") pod \"0ea6d4d1-f271-426c-862e-a2a43ab41f0a\" (UID: \"0ea6d4d1-f271-426c-862e-a2a43ab41f0a\") "
Oct 30 20:14:16.756582 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:16.756519    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0ea6d4d1-f271-426c-862e-a2a43ab41f0a-istio-token\" (UniqueName: \"kubernetes.io/projected/0ea6d4d1-f271-426c-862e-a2a43ab41f0a-istio-token\") pod \"0ea6d4d1-f271-426c-862e-a2a43ab41f0a\" (UID: \"0ea6d4d1-f271-426c-862e-a2a43ab41f0a\") "
Oct 30 20:14:16.756697 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:16.756602    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0ea6d4d1-f271-426c-862e-a2a43ab41f0a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ea6d4d1-f271-426c-862e-a2a43ab41f0a-istio-podinfo\") pod \"0ea6d4d1-f271-426c-862e-a2a43ab41f0a\" (UID: \"0ea6d4d1-f271-426c-862e-a2a43ab41f0a\") "
Oct 30 20:14:16.756697 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:16.756633    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0ea6d4d1-f271-426c-862e-a2a43ab41f0a-kube-api-access-hschv\" (UniqueName: \"kubernetes.io/projected/0ea6d4d1-f271-426c-862e-a2a43ab41f0a-kube-api-access-hschv\") pod \"0ea6d4d1-f271-426c-862e-a2a43ab41f0a\" (UID: \"0ea6d4d1-f271-426c-862e-a2a43ab41f0a\") "
Oct 30 20:14:16.757624 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:16.757258    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ea6d4d1-f271-426c-862e-a2a43ab41f0a-workload-socket" pod "0ea6d4d1-f271-426c-862e-a2a43ab41f0a" (UID: "0ea6d4d1-f271-426c-862e-a2a43ab41f0a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:14:16.757624 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:16.757408    6996 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ea6d4d1-f271-426c-862e-a2a43ab41f0a-workload-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:14:16.758070 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:16.757972    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ea6d4d1-f271-426c-862e-a2a43ab41f0a-istiod-ca-cert" pod "0ea6d4d1-f271-426c-862e-a2a43ab41f0a" (UID: "0ea6d4d1-f271-426c-862e-a2a43ab41f0a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:14:16.758362 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:16.758309    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ea6d4d1-f271-426c-862e-a2a43ab41f0a-workload-certs" pod "0ea6d4d1-f271-426c-862e-a2a43ab41f0a" (UID: "0ea6d4d1-f271-426c-862e-a2a43ab41f0a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:14:16.758604 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:16.758563    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ea6d4d1-f271-426c-862e-a2a43ab41f0a-istio-data" pod "0ea6d4d1-f271-426c-862e-a2a43ab41f0a" (UID: "0ea6d4d1-f271-426c-862e-a2a43ab41f0a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:14:16.758688 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:16.758253    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ea6d4d1-f271-426c-862e-a2a43ab41f0a-credential-socket" pod "0ea6d4d1-f271-426c-862e-a2a43ab41f0a" (UID: "0ea6d4d1-f271-426c-862e-a2a43ab41f0a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:14:16.760893 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:16.760866    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0ea6d4d1-f271-426c-862e-a2a43ab41f0a-istio-podinfo" pod "0ea6d4d1-f271-426c-862e-a2a43ab41f0a" (UID: "0ea6d4d1-f271-426c-862e-a2a43ab41f0a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 20:14:16.761213 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:16.761186    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ea6d4d1-f271-426c-862e-a2a43ab41f0a-istio-token" pod "0ea6d4d1-f271-426c-862e-a2a43ab41f0a" (UID: "0ea6d4d1-f271-426c-862e-a2a43ab41f0a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:14:16.761347 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:16.761276    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ea6d4d1-f271-426c-862e-a2a43ab41f0a-kube-api-access-hschv" pod "0ea6d4d1-f271-426c-862e-a2a43ab41f0a" (UID: "0ea6d4d1-f271-426c-862e-a2a43ab41f0a"). InnerVolumeSpecName "kube-api-access-hschv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:14:16.761528 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:16.761466    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ea6d4d1-f271-426c-862e-a2a43ab41f0a-istio-envoy" pod "0ea6d4d1-f271-426c-862e-a2a43ab41f0a" (UID: "0ea6d4d1-f271-426c-862e-a2a43ab41f0a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:14:16.858663 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:16.858621    6996 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ea6d4d1-f271-426c-862e-a2a43ab41f0a-istio-data\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:14:16.858663 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:16.858658    6996 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ea6d4d1-f271-426c-862e-a2a43ab41f0a-workload-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:14:16.858663 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:16.858670    6996 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ea6d4d1-f271-426c-862e-a2a43ab41f0a-istio-envoy\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:14:16.858941 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:16.858679    6996 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ea6d4d1-f271-426c-862e-a2a43ab41f0a-istiod-ca-cert\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:14:16.858941 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:16.858689    6996 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ea6d4d1-f271-426c-862e-a2a43ab41f0a-istio-token\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:14:16.858941 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:16.858697    6996 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ea6d4d1-f271-426c-862e-a2a43ab41f0a-istio-podinfo\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:14:16.858941 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:16.858711    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hschv\" (UniqueName: \"kubernetes.io/projected/0ea6d4d1-f271-426c-862e-a2a43ab41f0a-kube-api-access-hschv\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:14:16.858941 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:16.858726    6996 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ea6d4d1-f271-426c-862e-a2a43ab41f0a-credential-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:14:17.338892 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:17.338845    6996 generic.go:356] "Generic (PLEG): container finished" podID="0ea6d4d1-f271-426c-862e-a2a43ab41f0a" containerID="79c4dc32987f4d6fe67b5f06a5c695b2fd82909a0901712e09bb023fc681e06e" exitCode=0
Oct 30 20:14:17.339119 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:17.338934    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr"
Oct 30 20:14:17.339119 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:17.338934    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr" event={"ID":"0ea6d4d1-f271-426c-862e-a2a43ab41f0a","Type":"ContainerDied","Data":"79c4dc32987f4d6fe67b5f06a5c695b2fd82909a0901712e09bb023fc681e06e"}
Oct 30 20:14:17.339119 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:17.339034    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr" event={"ID":"0ea6d4d1-f271-426c-862e-a2a43ab41f0a","Type":"ContainerDied","Data":"7bffc3bd6fee06c174cb7801d85954d941fa478ad0dc724a7a702755b91313c8"}
Oct 30 20:14:17.339119 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:17.339065    6996 scope.go:122] "RemoveContainer" containerID="79c4dc32987f4d6fe67b5f06a5c695b2fd82909a0901712e09bb023fc681e06e"
Oct 30 20:14:17.351035 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:17.351012    6996 scope.go:122] "RemoveContainer" containerID="79c4dc32987f4d6fe67b5f06a5c695b2fd82909a0901712e09bb023fc681e06e"
Oct 30 20:14:17.351307 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:14:17.351288    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79c4dc32987f4d6fe67b5f06a5c695b2fd82909a0901712e09bb023fc681e06e\": container with ID starting with 79c4dc32987f4d6fe67b5f06a5c695b2fd82909a0901712e09bb023fc681e06e not found: ID does not exist" containerID="79c4dc32987f4d6fe67b5f06a5c695b2fd82909a0901712e09bb023fc681e06e"
Oct 30 20:14:17.351385 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:17.351315    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79c4dc32987f4d6fe67b5f06a5c695b2fd82909a0901712e09bb023fc681e06e"} err="failed to get container status \"79c4dc32987f4d6fe67b5f06a5c695b2fd82909a0901712e09bb023fc681e06e\": rpc error: code = NotFound desc = could not find container \"79c4dc32987f4d6fe67b5f06a5c695b2fd82909a0901712e09bb023fc681e06e\": container with ID starting with 79c4dc32987f4d6fe67b5f06a5c695b2fd82909a0901712e09bb023fc681e06e not found: ID does not exist"
Oct 30 20:14:17.413726 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:17.413691    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr"]
Oct 30 20:14:17.446276 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:17.446240    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-b-969878-openshift-default-74f689c897-fjpbr"]
Oct 30 20:14:18.793347 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:18.793311    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0ea6d4d1-f271-426c-862e-a2a43ab41f0a" path="/var/lib/kubelet/pods/0ea6d4d1-f271-426c-862e-a2a43ab41f0a/volumes"
Oct 30 20:14:20.736689 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:20.736630    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw"
Oct 30 20:14:20.821110 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:20.821078    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8-kube-api-access-7qmrc\" (UniqueName: \"kubernetes.io/projected/ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8-kube-api-access-7qmrc\") pod \"ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8\" (UID: \"ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8\") "
Oct 30 20:14:20.821299 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:20.821125    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8-istio-data\") pod \"ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8\" (UID: \"ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8\") "
Oct 30 20:14:20.821299 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:20.821154    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8-credential-socket\") pod \"ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8\" (UID: \"ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8\") "
Oct 30 20:14:20.821299 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:20.821208    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8-istio-podinfo\") pod \"ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8\" (UID: \"ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8\") "
Oct 30 20:14:20.821299 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:20.821243    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8-istiod-ca-cert\") pod \"ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8\" (UID: \"ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8\") "
Oct 30 20:14:20.821578 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:20.821306    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8-workload-socket\") pod \"ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8\" (UID: \"ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8\") "
Oct 30 20:14:20.821578 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:20.821369    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8-istio-token\" (UniqueName: \"kubernetes.io/projected/ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8-istio-token\") pod \"ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8\" (UID: \"ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8\") "
Oct 30 20:14:20.821578 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:20.821425    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8-workload-certs\") pod \"ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8\" (UID: \"ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8\") "
Oct 30 20:14:20.821578 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:20.821533    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8-credential-socket" pod "ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8" (UID: "ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:14:20.821578 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:20.821550    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8-istio-data" pod "ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8" (UID: "ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:14:20.822259 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:20.821643    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8-istio-envoy\") pod \"ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8\" (UID: \"ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8\") "
Oct 30 20:14:20.822259 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:20.821722    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8-workload-socket" pod "ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8" (UID: "ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:14:20.822259 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:20.821807    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8-workload-certs" pod "ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8" (UID: "ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:14:20.822259 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:20.821987    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8-istiod-ca-cert" pod "ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8" (UID: "ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:14:20.822544 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:20.822394    6996 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8-istiod-ca-cert\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:14:20.822544 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:20.822421    6996 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8-workload-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:14:20.822544 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:20.822437    6996 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8-workload-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:14:20.822544 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:20.822453    6996 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8-istio-data\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:14:20.822544 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:20.822466    6996 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8-credential-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:14:20.823753 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:20.823730    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8-istio-podinfo" pod "ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8" (UID: "ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 20:14:20.824324 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:20.824307    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8-kube-api-access-7qmrc" pod "ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8" (UID: "ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8"). InnerVolumeSpecName "kube-api-access-7qmrc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:14:20.824534 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:20.824516    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8-istio-token" pod "ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8" (UID: "ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:14:20.824624 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:20.824544    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8-istio-envoy" pod "ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8" (UID: "ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:14:20.924330 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:20.924156    6996 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8-istio-envoy\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:14:20.924330 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:20.924191    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7qmrc\" (UniqueName: \"kubernetes.io/projected/ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8-kube-api-access-7qmrc\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:14:20.924330 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:20.924266    6996 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8-istio-podinfo\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:14:20.924330 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:20.924298    6996 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8-istio-token\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:14:21.405780 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:21.405748    6996 generic.go:356] "Generic (PLEG): container finished" podID="ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8" containerID="3794d879be1a019eab9cfaff47c6d5bb19470a96503b33abed996c7ab8f6c536" exitCode=0
Oct 30 20:14:21.405941 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:21.405799    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw"
Oct 30 20:14:21.405941 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:21.405832    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw" event={"ID":"ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8","Type":"ContainerDied","Data":"3794d879be1a019eab9cfaff47c6d5bb19470a96503b33abed996c7ab8f6c536"}
Oct 30 20:14:21.405941 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:21.405883    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw" event={"ID":"ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8","Type":"ContainerDied","Data":"5aa91cc14c6a0a28266ecec121a0d55937f1a86dbcc8797c8df7a19024323bb9"}
Oct 30 20:14:21.405941 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:21.405906    6996 scope.go:122] "RemoveContainer" containerID="3794d879be1a019eab9cfaff47c6d5bb19470a96503b33abed996c7ab8f6c536"
Oct 30 20:14:21.417582 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:21.417509    6996 scope.go:122] "RemoveContainer" containerID="3794d879be1a019eab9cfaff47c6d5bb19470a96503b33abed996c7ab8f6c536"
Oct 30 20:14:21.417796 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:14:21.417773    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3794d879be1a019eab9cfaff47c6d5bb19470a96503b33abed996c7ab8f6c536\": container with ID starting with 3794d879be1a019eab9cfaff47c6d5bb19470a96503b33abed996c7ab8f6c536 not found: ID does not exist" containerID="3794d879be1a019eab9cfaff47c6d5bb19470a96503b33abed996c7ab8f6c536"
Oct 30 20:14:21.417851 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:21.417806    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3794d879be1a019eab9cfaff47c6d5bb19470a96503b33abed996c7ab8f6c536"} err="failed to get container status \"3794d879be1a019eab9cfaff47c6d5bb19470a96503b33abed996c7ab8f6c536\": rpc error: code = NotFound desc = could not find container \"3794d879be1a019eab9cfaff47c6d5bb19470a96503b33abed996c7ab8f6c536\": container with ID starting with 3794d879be1a019eab9cfaff47c6d5bb19470a96503b33abed996c7ab8f6c536 not found: ID does not exist"
Oct 30 20:14:21.444189 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:21.444152    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw"]
Oct 30 20:14:21.446781 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:21.446755    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-k4mhw"]
Oct 30 20:14:22.795292 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:22.795264    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8" path="/var/lib/kubelet/pods/ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8/volumes"
Oct 30 20:14:25.699359 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:25.699321    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j"]
Oct 30 20:14:25.699780 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:25.699539    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j" podUID="974a4f8f-5f4a-4338-80b3-847f6e5ca080" containerName="istio-proxy" containerID="cri-o://a90814e71ccbaffcfb0f994387b99d9674dc19f86bf129593e0ffdd822862aac" gracePeriod=30
Oct 30 20:14:30.994127 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:30.994096    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j"
Oct 30 20:14:31.136459 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:31.136411    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/974a4f8f-5f4a-4338-80b3-847f6e5ca080-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/974a4f8f-5f4a-4338-80b3-847f6e5ca080-workload-certs\") pod \"974a4f8f-5f4a-4338-80b3-847f6e5ca080\" (UID: \"974a4f8f-5f4a-4338-80b3-847f6e5ca080\") "
Oct 30 20:14:31.136837 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:31.136713    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/974a4f8f-5f4a-4338-80b3-847f6e5ca080-workload-certs" pod "974a4f8f-5f4a-4338-80b3-847f6e5ca080" (UID: "974a4f8f-5f4a-4338-80b3-847f6e5ca080"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:14:31.136945 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:31.136813    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/974a4f8f-5f4a-4338-80b3-847f6e5ca080-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/974a4f8f-5f4a-4338-80b3-847f6e5ca080-istio-envoy\") pod \"974a4f8f-5f4a-4338-80b3-847f6e5ca080\" (UID: \"974a4f8f-5f4a-4338-80b3-847f6e5ca080\") "
Oct 30 20:14:31.137006 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:31.136952    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/974a4f8f-5f4a-4338-80b3-847f6e5ca080-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/974a4f8f-5f4a-4338-80b3-847f6e5ca080-istiod-ca-cert\") pod \"974a4f8f-5f4a-4338-80b3-847f6e5ca080\" (UID: \"974a4f8f-5f4a-4338-80b3-847f6e5ca080\") "
Oct 30 20:14:31.137185 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:31.137141    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/974a4f8f-5f4a-4338-80b3-847f6e5ca080-kube-api-access-xv4wt\" (UniqueName: \"kubernetes.io/projected/974a4f8f-5f4a-4338-80b3-847f6e5ca080-kube-api-access-xv4wt\") pod \"974a4f8f-5f4a-4338-80b3-847f6e5ca080\" (UID: \"974a4f8f-5f4a-4338-80b3-847f6e5ca080\") "
Oct 30 20:14:31.137185 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:31.137183    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/974a4f8f-5f4a-4338-80b3-847f6e5ca080-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/974a4f8f-5f4a-4338-80b3-847f6e5ca080-workload-socket\") pod \"974a4f8f-5f4a-4338-80b3-847f6e5ca080\" (UID: \"974a4f8f-5f4a-4338-80b3-847f6e5ca080\") "
Oct 30 20:14:31.137624 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:31.137292    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/974a4f8f-5f4a-4338-80b3-847f6e5ca080-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/974a4f8f-5f4a-4338-80b3-847f6e5ca080-istio-data\") pod \"974a4f8f-5f4a-4338-80b3-847f6e5ca080\" (UID: \"974a4f8f-5f4a-4338-80b3-847f6e5ca080\") "
Oct 30 20:14:31.138436 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:31.138403    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/974a4f8f-5f4a-4338-80b3-847f6e5ca080-istio-data" pod "974a4f8f-5f4a-4338-80b3-847f6e5ca080" (UID: "974a4f8f-5f4a-4338-80b3-847f6e5ca080"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:14:31.138608 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:31.138457    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/974a4f8f-5f4a-4338-80b3-847f6e5ca080-workload-socket" pod "974a4f8f-5f4a-4338-80b3-847f6e5ca080" (UID: "974a4f8f-5f4a-4338-80b3-847f6e5ca080"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:14:31.138608 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:31.138558    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/974a4f8f-5f4a-4338-80b3-847f6e5ca080-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/974a4f8f-5f4a-4338-80b3-847f6e5ca080-credential-socket\") pod \"974a4f8f-5f4a-4338-80b3-847f6e5ca080\" (UID: \"974a4f8f-5f4a-4338-80b3-847f6e5ca080\") "
Oct 30 20:14:31.138774 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:31.138642    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/974a4f8f-5f4a-4338-80b3-847f6e5ca080-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/974a4f8f-5f4a-4338-80b3-847f6e5ca080-istio-podinfo\") pod \"974a4f8f-5f4a-4338-80b3-847f6e5ca080\" (UID: \"974a4f8f-5f4a-4338-80b3-847f6e5ca080\") "
Oct 30 20:14:31.138774 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:31.138732    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/974a4f8f-5f4a-4338-80b3-847f6e5ca080-istio-token\" (UniqueName: \"kubernetes.io/projected/974a4f8f-5f4a-4338-80b3-847f6e5ca080-istio-token\") pod \"974a4f8f-5f4a-4338-80b3-847f6e5ca080\" (UID: \"974a4f8f-5f4a-4338-80b3-847f6e5ca080\") "
Oct 30 20:14:31.139455 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:31.138734    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/974a4f8f-5f4a-4338-80b3-847f6e5ca080-istiod-ca-cert" pod "974a4f8f-5f4a-4338-80b3-847f6e5ca080" (UID: "974a4f8f-5f4a-4338-80b3-847f6e5ca080"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:14:31.139738 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:31.139714    6996 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/974a4f8f-5f4a-4338-80b3-847f6e5ca080-istio-data\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:14:31.139830 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:31.139742    6996 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/974a4f8f-5f4a-4338-80b3-847f6e5ca080-workload-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:14:31.139830 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:31.139758    6996 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/974a4f8f-5f4a-4338-80b3-847f6e5ca080-istiod-ca-cert\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:14:31.139830 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:31.139759    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/974a4f8f-5f4a-4338-80b3-847f6e5ca080-credential-socket" pod "974a4f8f-5f4a-4338-80b3-847f6e5ca080" (UID: "974a4f8f-5f4a-4338-80b3-847f6e5ca080"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:14:31.139830 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:31.139773    6996 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/974a4f8f-5f4a-4338-80b3-847f6e5ca080-workload-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:14:31.140510 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:31.140450    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/974a4f8f-5f4a-4338-80b3-847f6e5ca080-kube-api-access-xv4wt" pod "974a4f8f-5f4a-4338-80b3-847f6e5ca080" (UID: "974a4f8f-5f4a-4338-80b3-847f6e5ca080"). InnerVolumeSpecName "kube-api-access-xv4wt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:14:31.140775 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:31.140740    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/974a4f8f-5f4a-4338-80b3-847f6e5ca080-istio-envoy" pod "974a4f8f-5f4a-4338-80b3-847f6e5ca080" (UID: "974a4f8f-5f4a-4338-80b3-847f6e5ca080"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:14:31.141191 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:31.141170    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/974a4f8f-5f4a-4338-80b3-847f6e5ca080-istio-podinfo" pod "974a4f8f-5f4a-4338-80b3-847f6e5ca080" (UID: "974a4f8f-5f4a-4338-80b3-847f6e5ca080"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 20:14:31.141593 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:31.141572    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/974a4f8f-5f4a-4338-80b3-847f6e5ca080-istio-token" pod "974a4f8f-5f4a-4338-80b3-847f6e5ca080" (UID: "974a4f8f-5f4a-4338-80b3-847f6e5ca080"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:14:31.221541 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:14:31.221439    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 20:14:31.241360 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:31.241329    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xv4wt\" (UniqueName: \"kubernetes.io/projected/974a4f8f-5f4a-4338-80b3-847f6e5ca080-kube-api-access-xv4wt\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:14:31.241360 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:31.241360    6996 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/974a4f8f-5f4a-4338-80b3-847f6e5ca080-credential-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:14:31.241583 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:31.241370    6996 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/974a4f8f-5f4a-4338-80b3-847f6e5ca080-istio-podinfo\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:14:31.241583 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:31.241379    6996 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/974a4f8f-5f4a-4338-80b3-847f6e5ca080-istio-token\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:14:31.241583 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:31.241388    6996 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/974a4f8f-5f4a-4338-80b3-847f6e5ca080-istio-envoy\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:14:31.502209 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:31.502120    6996 generic.go:356] "Generic (PLEG): container finished" podID="974a4f8f-5f4a-4338-80b3-847f6e5ca080" containerID="a90814e71ccbaffcfb0f994387b99d9674dc19f86bf129593e0ffdd822862aac" exitCode=0
Oct 30 20:14:31.502209 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:31.502170    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j"
Oct 30 20:14:31.502209 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:31.502199    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j" event={"ID":"974a4f8f-5f4a-4338-80b3-847f6e5ca080","Type":"ContainerDied","Data":"a90814e71ccbaffcfb0f994387b99d9674dc19f86bf129593e0ffdd822862aac"}
Oct 30 20:14:31.502442 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:31.502257    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j" event={"ID":"974a4f8f-5f4a-4338-80b3-847f6e5ca080","Type":"ContainerDied","Data":"2ee44a32cb31b4246ae7f348a515246d1787ab91d1ba7efd289b9bbdba72f052"}
Oct 30 20:14:31.502442 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:31.502281    6996 scope.go:122] "RemoveContainer" containerID="a90814e71ccbaffcfb0f994387b99d9674dc19f86bf129593e0ffdd822862aac"
Oct 30 20:14:31.514076 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:31.514051    6996 scope.go:122] "RemoveContainer" containerID="a90814e71ccbaffcfb0f994387b99d9674dc19f86bf129593e0ffdd822862aac"
Oct 30 20:14:31.514365 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:14:31.514345    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a90814e71ccbaffcfb0f994387b99d9674dc19f86bf129593e0ffdd822862aac\": container with ID starting with a90814e71ccbaffcfb0f994387b99d9674dc19f86bf129593e0ffdd822862aac not found: ID does not exist" containerID="a90814e71ccbaffcfb0f994387b99d9674dc19f86bf129593e0ffdd822862aac"
Oct 30 20:14:31.514425 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:31.514375    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a90814e71ccbaffcfb0f994387b99d9674dc19f86bf129593e0ffdd822862aac"} err="failed to get container status \"a90814e71ccbaffcfb0f994387b99d9674dc19f86bf129593e0ffdd822862aac\": rpc error: code = NotFound desc = could not find container \"a90814e71ccbaffcfb0f994387b99d9674dc19f86bf129593e0ffdd822862aac\": container with ID starting with a90814e71ccbaffcfb0f994387b99d9674dc19f86bf129593e0ffdd822862aac not found: ID does not exist"
Oct 30 20:14:31.529970 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:31.529942    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j"]
Oct 30 20:14:31.533343 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:31.533322    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-7cbcbb-openshift-default-666c99dff9-65j9j"]
Oct 30 20:14:32.793418 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:32.793386    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="974a4f8f-5f4a-4338-80b3-847f6e5ca080" path="/var/lib/kubelet/pods/974a4f8f-5f4a-4338-80b3-847f6e5ca080/volumes"
Oct 30 20:14:43.186321 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:43.186276    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 20:14:47.955541 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:47.951259    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp"]
Oct 30 20:14:47.955541 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:47.952519    6996 cpu_manager.go:414] "containerMap: removing container" podUID="ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8" containerName="istio-proxy"
Oct 30 20:14:47.955541 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:47.952541    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8" containerName="istio-proxy"
Oct 30 20:14:47.955541 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:47.952585    6996 cpu_manager.go:414] "containerMap: removing container" podUID="974a4f8f-5f4a-4338-80b3-847f6e5ca080" containerName="istio-proxy"
Oct 30 20:14:47.955541 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:47.952593    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="974a4f8f-5f4a-4338-80b3-847f6e5ca080" containerName="istio-proxy"
Oct 30 20:14:47.955541 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:47.952627    6996 cpu_manager.go:414] "containerMap: removing container" podUID="bae90963-7525-42cb-9cdc-fdb32da947e7" containerName="istio-proxy"
Oct 30 20:14:47.955541 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:47.952635    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bae90963-7525-42cb-9cdc-fdb32da947e7" containerName="istio-proxy"
Oct 30 20:14:47.955541 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:47.952652    6996 cpu_manager.go:414] "containerMap: removing container" podUID="799847f8-0366-4fba-a5a6-3e90bb4bba55" containerName="payload-processing"
Oct 30 20:14:47.955541 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:47.952661    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="799847f8-0366-4fba-a5a6-3e90bb4bba55" containerName="payload-processing"
Oct 30 20:14:47.955541 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:47.952687    6996 cpu_manager.go:414] "containerMap: removing container" podUID="0ea6d4d1-f271-426c-862e-a2a43ab41f0a" containerName="istio-proxy"
Oct 30 20:14:47.955541 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:47.952695    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0ea6d4d1-f271-426c-862e-a2a43ab41f0a" containerName="istio-proxy"
Oct 30 20:14:47.955541 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:47.952707    6996 cpu_manager.go:414] "containerMap: removing container" podUID="6d58ffd8-ffb2-4751-a622-11b584616c7d" containerName="istio-proxy"
Oct 30 20:14:47.955541 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:47.952715    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6d58ffd8-ffb2-4751-a622-11b584616c7d" containerName="istio-proxy"
Oct 30 20:14:47.955541 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:47.952902    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="974a4f8f-5f4a-4338-80b3-847f6e5ca080" containerName="istio-proxy"
Oct 30 20:14:47.955541 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:47.952926    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="bae90963-7525-42cb-9cdc-fdb32da947e7" containerName="istio-proxy"
Oct 30 20:14:47.955541 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:47.952937    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="799847f8-0366-4fba-a5a6-3e90bb4bba55" containerName="payload-processing"
Oct 30 20:14:47.955541 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:47.952952    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="0ea6d4d1-f271-426c-862e-a2a43ab41f0a" containerName="istio-proxy"
Oct 30 20:14:47.955541 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:47.952963    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="6d58ffd8-ffb2-4751-a622-11b584616c7d" containerName="istio-proxy"
Oct 30 20:14:47.955541 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:47.952981    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="ce1ed4ae-eab8-49a6-a68f-b7fe756a10a8" containerName="istio-proxy"
Oct 30 20:14:47.968433 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:47.959176    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp"
Oct 30 20:14:47.971816 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:47.971788    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-f63d20-openshift-default-dockercfg-stk29\""
Oct 30 20:14:47.982417 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:47.982380    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp"]
Oct 30 20:14:48.053372 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:48.025856    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a21bbe95-547d-41b0-b7c8-e1a296545947-istio-token\") pod \"e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp\" (UID: \"a21bbe95-547d-41b0-b7c8-e1a296545947\") " pod="openshift-ingress/e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp"
Oct 30 20:14:48.053931 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:48.053888    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a21bbe95-547d-41b0-b7c8-e1a296545947-workload-certs\") pod \"e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp\" (UID: \"a21bbe95-547d-41b0-b7c8-e1a296545947\") " pod="openshift-ingress/e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp"
Oct 30 20:14:48.054252 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:48.054215    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a21bbe95-547d-41b0-b7c8-e1a296545947-istio-podinfo\") pod \"e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp\" (UID: \"a21bbe95-547d-41b0-b7c8-e1a296545947\") " pod="openshift-ingress/e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp"
Oct 30 20:14:48.054567 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:48.054544    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a21bbe95-547d-41b0-b7c8-e1a296545947-istiod-ca-cert\") pod \"e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp\" (UID: \"a21bbe95-547d-41b0-b7c8-e1a296545947\") " pod="openshift-ingress/e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp"
Oct 30 20:14:48.054743 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:48.054721    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xg6w4\" (UniqueName: \"kubernetes.io/projected/a21bbe95-547d-41b0-b7c8-e1a296545947-kube-api-access-xg6w4\") pod \"e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp\" (UID: \"a21bbe95-547d-41b0-b7c8-e1a296545947\") " pod="openshift-ingress/e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp"
Oct 30 20:14:48.054936 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:48.054918    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a21bbe95-547d-41b0-b7c8-e1a296545947-istio-envoy\") pod \"e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp\" (UID: \"a21bbe95-547d-41b0-b7c8-e1a296545947\") " pod="openshift-ingress/e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp"
Oct 30 20:14:48.055181 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:48.055055    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a21bbe95-547d-41b0-b7c8-e1a296545947-workload-socket\") pod \"e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp\" (UID: \"a21bbe95-547d-41b0-b7c8-e1a296545947\") " pod="openshift-ingress/e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp"
Oct 30 20:14:48.055322 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:48.055305    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a21bbe95-547d-41b0-b7c8-e1a296545947-credential-socket\") pod \"e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp\" (UID: \"a21bbe95-547d-41b0-b7c8-e1a296545947\") " pod="openshift-ingress/e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp"
Oct 30 20:14:48.055439 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:48.055421    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a21bbe95-547d-41b0-b7c8-e1a296545947-istio-data\") pod \"e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp\" (UID: \"a21bbe95-547d-41b0-b7c8-e1a296545947\") " pod="openshift-ingress/e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp"
Oct 30 20:14:48.157143 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:48.157104    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a21bbe95-547d-41b0-b7c8-e1a296545947-istiod-ca-cert\") pod \"e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp\" (UID: \"a21bbe95-547d-41b0-b7c8-e1a296545947\") " pod="openshift-ingress/e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp"
Oct 30 20:14:48.157341 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:48.157176    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xg6w4\" (UniqueName: \"kubernetes.io/projected/a21bbe95-547d-41b0-b7c8-e1a296545947-kube-api-access-xg6w4\") pod \"e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp\" (UID: \"a21bbe95-547d-41b0-b7c8-e1a296545947\") " pod="openshift-ingress/e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp"
Oct 30 20:14:48.157341 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:48.157221    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a21bbe95-547d-41b0-b7c8-e1a296545947-istio-envoy\") pod \"e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp\" (UID: \"a21bbe95-547d-41b0-b7c8-e1a296545947\") " pod="openshift-ingress/e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp"
Oct 30 20:14:48.157341 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:48.157242    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a21bbe95-547d-41b0-b7c8-e1a296545947-workload-socket\") pod \"e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp\" (UID: \"a21bbe95-547d-41b0-b7c8-e1a296545947\") " pod="openshift-ingress/e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp"
Oct 30 20:14:48.157341 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:48.157273    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a21bbe95-547d-41b0-b7c8-e1a296545947-credential-socket\") pod \"e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp\" (UID: \"a21bbe95-547d-41b0-b7c8-e1a296545947\") " pod="openshift-ingress/e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp"
Oct 30 20:14:48.157341 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:48.157289    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a21bbe95-547d-41b0-b7c8-e1a296545947-istio-data\") pod \"e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp\" (UID: \"a21bbe95-547d-41b0-b7c8-e1a296545947\") " pod="openshift-ingress/e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp"
Oct 30 20:14:48.157683 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:48.157342    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a21bbe95-547d-41b0-b7c8-e1a296545947-istio-token\") pod \"e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp\" (UID: \"a21bbe95-547d-41b0-b7c8-e1a296545947\") " pod="openshift-ingress/e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp"
Oct 30 20:14:48.157683 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:48.157419    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a21bbe95-547d-41b0-b7c8-e1a296545947-workload-certs\") pod \"e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp\" (UID: \"a21bbe95-547d-41b0-b7c8-e1a296545947\") " pod="openshift-ingress/e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp"
Oct 30 20:14:48.157683 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:48.157543    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a21bbe95-547d-41b0-b7c8-e1a296545947-istio-podinfo\") pod \"e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp\" (UID: \"a21bbe95-547d-41b0-b7c8-e1a296545947\") " pod="openshift-ingress/e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp"
Oct 30 20:14:48.157847 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:48.157685    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a21bbe95-547d-41b0-b7c8-e1a296545947-workload-socket\") pod \"e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp\" (UID: \"a21bbe95-547d-41b0-b7c8-e1a296545947\") " pod="openshift-ingress/e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp"
Oct 30 20:14:48.157847 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:48.157720    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a21bbe95-547d-41b0-b7c8-e1a296545947-istio-data\") pod \"e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp\" (UID: \"a21bbe95-547d-41b0-b7c8-e1a296545947\") " pod="openshift-ingress/e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp"
Oct 30 20:14:48.157970 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:48.157887    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a21bbe95-547d-41b0-b7c8-e1a296545947-credential-socket\") pod \"e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp\" (UID: \"a21bbe95-547d-41b0-b7c8-e1a296545947\") " pod="openshift-ingress/e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp"
Oct 30 20:14:48.158029 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:48.157980    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a21bbe95-547d-41b0-b7c8-e1a296545947-workload-certs\") pod \"e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp\" (UID: \"a21bbe95-547d-41b0-b7c8-e1a296545947\") " pod="openshift-ingress/e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp"
Oct 30 20:14:48.158359 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:48.158337    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a21bbe95-547d-41b0-b7c8-e1a296545947-istiod-ca-cert\") pod \"e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp\" (UID: \"a21bbe95-547d-41b0-b7c8-e1a296545947\") " pod="openshift-ingress/e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp"
Oct 30 20:14:48.159974 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:48.159946    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a21bbe95-547d-41b0-b7c8-e1a296545947-istio-envoy\") pod \"e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp\" (UID: \"a21bbe95-547d-41b0-b7c8-e1a296545947\") " pod="openshift-ingress/e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp"
Oct 30 20:14:48.160284 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:48.160253    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a21bbe95-547d-41b0-b7c8-e1a296545947-istio-podinfo\") pod \"e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp\" (UID: \"a21bbe95-547d-41b0-b7c8-e1a296545947\") " pod="openshift-ingress/e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp"
Oct 30 20:14:48.170625 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:48.170594    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xg6w4\" (UniqueName: \"kubernetes.io/projected/a21bbe95-547d-41b0-b7c8-e1a296545947-kube-api-access-xg6w4\") pod \"e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp\" (UID: \"a21bbe95-547d-41b0-b7c8-e1a296545947\") " pod="openshift-ingress/e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp"
Oct 30 20:14:48.170761 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:48.170605    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a21bbe95-547d-41b0-b7c8-e1a296545947-istio-token\") pod \"e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp\" (UID: \"a21bbe95-547d-41b0-b7c8-e1a296545947\") " pod="openshift-ingress/e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp"
Oct 30 20:14:48.305833 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:48.305742    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp"
Oct 30 20:14:48.613423 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:48.613390    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp"]
Oct 30 20:14:48.613893 ip-10-0-141-42 kubenswrapper[6996]: W1030 20:14:48.613865    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda21bbe95_547d_41b0_b7c8_e1a296545947.slice/crio-261c442766844bf6b4c185098e498042bd7dfdf9df1d30385c844ca8e0632885 WatchSource:0}: Error finding container 261c442766844bf6b4c185098e498042bd7dfdf9df1d30385c844ca8e0632885: Status 404 returned error can't find the container with id 261c442766844bf6b4c185098e498042bd7dfdf9df1d30385c844ca8e0632885
Oct 30 20:14:48.617161 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:48.617119    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 20:14:48.617252 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:48.617193    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 20:14:48.617252 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:48.617241    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 20:14:48.632048 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:48.632019    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp" event={"ID":"a21bbe95-547d-41b0-b7c8-e1a296545947","Type":"ContainerStarted","Data":"261c442766844bf6b4c185098e498042bd7dfdf9df1d30385c844ca8e0632885"}
Oct 30 20:14:49.641078 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:49.641040    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp" event={"ID":"a21bbe95-547d-41b0-b7c8-e1a296545947","Type":"ContainerStarted","Data":"a482c0cafd0058dd322aed5b2a574316076d71cae879579d39f173e60ab077ef"}
Oct 30 20:14:49.669805 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:49.669740    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp" podStartSLOduration=2.669722322 podStartE2EDuration="2.669722322s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:14:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:14:49.664341791 +0000 UTC m=+2564.992459944" watchObservedRunningTime="2026-10-30 20:14:49.669722322 +0000 UTC m=+2564.997840477"
Oct 30 20:14:50.306123 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:50.306092    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp"
Oct 30 20:14:50.312560 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:50.312535    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp"
Oct 30 20:14:50.653835 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:50.653800    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp"
Oct 30 20:14:50.655348 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:14:50.655319    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp"
Oct 30 20:15:00.547139 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:00.547101    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-7cbcbb-29889855-hgcqt"]
Oct 30 20:15:00.552768 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:00.552746    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-7cbcbb-29889855-hgcqt"
Oct 30 20:15:00.567056 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:00.566743    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zgv8j\""
Oct 30 20:15:00.573387 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:00.573352    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-7cbcbb-29889855-hgcqt"]
Oct 30 20:15:00.678170 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:00.678116    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a68d2bd-f65c-4013-b700-3a0da95b301f-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w4-7cbcbb-29889855-hgcqt\" (UID: \"7a68d2bd-f65c-4013-b700-3a0da95b301f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-7cbcbb-29889855-hgcqt"
Oct 30 20:15:00.678373 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:00.678293    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfztm\" (UniqueName: \"kubernetes.io/projected/7a68d2bd-f65c-4013-b700-3a0da95b301f-kube-api-access-zfztm\") pod \"maas-api-key-cleanup-e2e-shared-b-w4-7cbcbb-29889855-hgcqt\" (UID: \"7a68d2bd-f65c-4013-b700-3a0da95b301f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-7cbcbb-29889855-hgcqt"
Oct 30 20:15:00.779309 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:00.779274    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a68d2bd-f65c-4013-b700-3a0da95b301f-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w4-7cbcbb-29889855-hgcqt\" (UID: \"7a68d2bd-f65c-4013-b700-3a0da95b301f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-7cbcbb-29889855-hgcqt"
Oct 30 20:15:00.779520 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:00.779369    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfztm\" (UniqueName: \"kubernetes.io/projected/7a68d2bd-f65c-4013-b700-3a0da95b301f-kube-api-access-zfztm\") pod \"maas-api-key-cleanup-e2e-shared-b-w4-7cbcbb-29889855-hgcqt\" (UID: \"7a68d2bd-f65c-4013-b700-3a0da95b301f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-7cbcbb-29889855-hgcqt"
Oct 30 20:15:00.779917 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:00.779893    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a68d2bd-f65c-4013-b700-3a0da95b301f-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w4-7cbcbb-29889855-hgcqt\" (UID: \"7a68d2bd-f65c-4013-b700-3a0da95b301f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-7cbcbb-29889855-hgcqt"
Oct 30 20:15:00.794052 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:00.794028    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfztm\" (UniqueName: \"kubernetes.io/projected/7a68d2bd-f65c-4013-b700-3a0da95b301f-kube-api-access-zfztm\") pod \"maas-api-key-cleanup-e2e-shared-b-w4-7cbcbb-29889855-hgcqt\" (UID: \"7a68d2bd-f65c-4013-b700-3a0da95b301f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-7cbcbb-29889855-hgcqt"
Oct 30 20:15:00.876972 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:00.876940    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-7cbcbb-29889855-hgcqt"
Oct 30 20:15:01.173520 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:01.173448    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-7cbcbb-29889855-hgcqt"]
Oct 30 20:15:01.174030 ip-10-0-141-42 kubenswrapper[6996]: W1030 20:15:01.174000    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a68d2bd_f65c_4013_b700_3a0da95b301f.slice/crio-03686a98e28702408f6a3cd380e51c60d5b1cbdacd94de2d321b34846bef8f90 WatchSource:0}: Error finding container 03686a98e28702408f6a3cd380e51c60d5b1cbdacd94de2d321b34846bef8f90: Status 404 returned error can't find the container with id 03686a98e28702408f6a3cd380e51c60d5b1cbdacd94de2d321b34846bef8f90
Oct 30 20:15:01.176344 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:01.176321    6996 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 20:15:01.193193 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:15:01.193160    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 20:15:01.741406 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:01.741367    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-7cbcbb-29889855-hgcqt" event={"ID":"7a68d2bd-f65c-4013-b700-3a0da95b301f","Type":"ContainerStarted","Data":"03686a98e28702408f6a3cd380e51c60d5b1cbdacd94de2d321b34846bef8f90"}
Oct 30 20:15:04.772107 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:04.772013    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-7cbcbb-29889855-hgcqt" event={"ID":"7a68d2bd-f65c-4013-b700-3a0da95b301f","Type":"ContainerStarted","Data":"b05ee17876f292b76ecb7487f776ce60973181151f07cdef68b8c9f0948c0e47"}
Oct 30 20:15:04.793589 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:04.793530    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-7cbcbb-29889855-hgcqt" podStartSLOduration=1.919310224 podStartE2EDuration="4.793512626s" totalImagesPullingTime="2.874202402s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:15:04.790497768 +0000 UTC m=+2580.118615916" watchObservedRunningTime="2026-10-30 20:15:04.793512626 +0000 UTC m=+2580.121630781"
Oct 30 20:15:05.781640 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:05.781600    6996 generic.go:356] "Generic (PLEG): container finished" podID="7a68d2bd-f65c-4013-b700-3a0da95b301f" containerID="b05ee17876f292b76ecb7487f776ce60973181151f07cdef68b8c9f0948c0e47" exitCode=60
Oct 30 20:15:05.782073 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:05.781690    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-7cbcbb-29889855-hgcqt" event={"ID":"7a68d2bd-f65c-4013-b700-3a0da95b301f","Type":"ContainerDied","Data":"b05ee17876f292b76ecb7487f776ce60973181151f07cdef68b8c9f0948c0e47"}
Oct 30 20:15:05.782073 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:05.781942    6996 scope.go:122] "RemoveContainer" containerID="b05ee17876f292b76ecb7487f776ce60973181151f07cdef68b8c9f0948c0e47"
Oct 30 20:15:06.791451 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:06.791399    6996 generic.go:356] "Generic (PLEG): container finished" podID="7a68d2bd-f65c-4013-b700-3a0da95b301f" containerID="4bddfe05d866ebca77f2455cbe53a56c3f1aec45f64392b4593c94b97760a6c5" exitCode=60
Oct 30 20:15:06.791941 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:06.791804    6996 scope.go:122] "RemoveContainer" containerID="4bddfe05d866ebca77f2455cbe53a56c3f1aec45f64392b4593c94b97760a6c5"
Oct 30 20:15:06.792110 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:15:06.792088    6996 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w4-7cbcbb-29889855-hgcqt_odh-ai-gateway-infra(7a68d2bd-f65c-4013-b700-3a0da95b301f)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-7cbcbb-29889855-hgcqt" podUID="7a68d2bd-f65c-4013-b700-3a0da95b301f"
Oct 30 20:15:06.795560 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:06.795532    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-7cbcbb-29889855-hgcqt" event={"ID":"7a68d2bd-f65c-4013-b700-3a0da95b301f","Type":"ContainerDied","Data":"4bddfe05d866ebca77f2455cbe53a56c3f1aec45f64392b4593c94b97760a6c5"}
Oct 30 20:15:06.795689 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:06.795590    6996 scope.go:122] "RemoveContainer" containerID="b05ee17876f292b76ecb7487f776ce60973181151f07cdef68b8c9f0948c0e47"
Oct 30 20:15:07.800474 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:07.800440    6996 scope.go:122] "RemoveContainer" containerID="4bddfe05d866ebca77f2455cbe53a56c3f1aec45f64392b4593c94b97760a6c5"
Oct 30 20:15:07.800901 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:15:07.800871    6996 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w4-7cbcbb-29889855-hgcqt_odh-ai-gateway-infra(7a68d2bd-f65c-4013-b700-3a0da95b301f)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-7cbcbb-29889855-hgcqt" podUID="7a68d2bd-f65c-4013-b700-3a0da95b301f"
Oct 30 20:15:17.784455 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:17.784421    6996 scope.go:122] "RemoveContainer" containerID="4bddfe05d866ebca77f2455cbe53a56c3f1aec45f64392b4593c94b97760a6c5"
Oct 30 20:15:19.179994 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:19.179912    6996 generic.go:356] "Generic (PLEG): container finished" podID="7a68d2bd-f65c-4013-b700-3a0da95b301f" containerID="ac4b835a02f9186a50c516abc9fdf8e0dfd880913dd081f4dea56d0f748206d7" exitCode=60
Oct 30 20:15:19.180767 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:19.179987    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-7cbcbb-29889855-hgcqt" event={"ID":"7a68d2bd-f65c-4013-b700-3a0da95b301f","Type":"ContainerDied","Data":"ac4b835a02f9186a50c516abc9fdf8e0dfd880913dd081f4dea56d0f748206d7"}
Oct 30 20:15:19.180767 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:19.180049    6996 scope.go:122] "RemoveContainer" containerID="4bddfe05d866ebca77f2455cbe53a56c3f1aec45f64392b4593c94b97760a6c5"
Oct 30 20:15:19.180767 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:19.180522    6996 scope.go:122] "RemoveContainer" containerID="ac4b835a02f9186a50c516abc9fdf8e0dfd880913dd081f4dea56d0f748206d7"
Oct 30 20:15:19.180995 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:15:19.180953    6996 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w4-7cbcbb-29889855-hgcqt_odh-ai-gateway-infra(7a68d2bd-f65c-4013-b700-3a0da95b301f)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-7cbcbb-29889855-hgcqt" podUID="7a68d2bd-f65c-4013-b700-3a0da95b301f"
Oct 30 20:15:19.302952 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:19.302918    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-f63d20-7bdf8df68b-clm5w"]
Oct 30 20:15:19.309578 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:19.309557    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f63d20-7bdf8df68b-clm5w"
Oct 30 20:15:19.316023 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:19.316001    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-f63d20-serving-cert\""
Oct 30 20:15:19.333650 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:19.333617    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-f63d20-7bdf8df68b-clm5w"]
Oct 30 20:15:19.394150 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:19.394119    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d1d1de99-5578-4b8a-b94d-c8c4a095d760-maas-api-tls\") pod \"maas-api-e2e-inf-a-f63d20-7bdf8df68b-clm5w\" (UID: \"d1d1de99-5578-4b8a-b94d-c8c4a095d760\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f63d20-7bdf8df68b-clm5w"
Oct 30 20:15:19.394385 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:19.394365    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1d1de99-5578-4b8a-b94d-c8c4a095d760-openshift-service-ca\") pod \"maas-api-e2e-inf-a-f63d20-7bdf8df68b-clm5w\" (UID: \"d1d1de99-5578-4b8a-b94d-c8c4a095d760\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f63d20-7bdf8df68b-clm5w"
Oct 30 20:15:19.394435 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:19.394425    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1d1de99-5578-4b8a-b94d-c8c4a095d760-metrics-certs\") pod \"maas-api-e2e-inf-a-f63d20-7bdf8df68b-clm5w\" (UID: \"d1d1de99-5578-4b8a-b94d-c8c4a095d760\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f63d20-7bdf8df68b-clm5w"
Oct 30 20:15:19.394536 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:19.394521    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpt76\" (UniqueName: \"kubernetes.io/projected/d1d1de99-5578-4b8a-b94d-c8c4a095d760-kube-api-access-vpt76\") pod \"maas-api-e2e-inf-a-f63d20-7bdf8df68b-clm5w\" (UID: \"d1d1de99-5578-4b8a-b94d-c8c4a095d760\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f63d20-7bdf8df68b-clm5w"
Oct 30 20:15:19.503691 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:19.503597    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1d1de99-5578-4b8a-b94d-c8c4a095d760-openshift-service-ca\") pod \"maas-api-e2e-inf-a-f63d20-7bdf8df68b-clm5w\" (UID: \"d1d1de99-5578-4b8a-b94d-c8c4a095d760\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f63d20-7bdf8df68b-clm5w"
Oct 30 20:15:19.504046 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:19.503979    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1d1de99-5578-4b8a-b94d-c8c4a095d760-metrics-certs\") pod \"maas-api-e2e-inf-a-f63d20-7bdf8df68b-clm5w\" (UID: \"d1d1de99-5578-4b8a-b94d-c8c4a095d760\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f63d20-7bdf8df68b-clm5w"
Oct 30 20:15:19.504256 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:19.504048    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpt76\" (UniqueName: \"kubernetes.io/projected/d1d1de99-5578-4b8a-b94d-c8c4a095d760-kube-api-access-vpt76\") pod \"maas-api-e2e-inf-a-f63d20-7bdf8df68b-clm5w\" (UID: \"d1d1de99-5578-4b8a-b94d-c8c4a095d760\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f63d20-7bdf8df68b-clm5w"
Oct 30 20:15:19.504256 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:19.504130    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d1d1de99-5578-4b8a-b94d-c8c4a095d760-maas-api-tls\") pod \"maas-api-e2e-inf-a-f63d20-7bdf8df68b-clm5w\" (UID: \"d1d1de99-5578-4b8a-b94d-c8c4a095d760\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f63d20-7bdf8df68b-clm5w"
Oct 30 20:15:19.504652 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:19.504425    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1d1de99-5578-4b8a-b94d-c8c4a095d760-openshift-service-ca\") pod \"maas-api-e2e-inf-a-f63d20-7bdf8df68b-clm5w\" (UID: \"d1d1de99-5578-4b8a-b94d-c8c4a095d760\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f63d20-7bdf8df68b-clm5w"
Oct 30 20:15:19.504652 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:15:19.504601    6996 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-f63d20-serving-cert: secret "maas-api-e2e-inf-a-f63d20-serving-cert" not found
Oct 30 20:15:19.504913 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:15:19.504674    6996 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1d1de99-5578-4b8a-b94d-c8c4a095d760-maas-api-tls podName:d1d1de99-5578-4b8a-b94d-c8c4a095d760 nodeName:}" failed. No retries permitted until 2026-10-30 20:15:20.004653633 +0000 UTC m=+2595.332771769 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d1d1de99-5578-4b8a-b94d-c8c4a095d760-maas-api-tls") pod "maas-api-e2e-inf-a-f63d20-7bdf8df68b-clm5w" (UID: "d1d1de99-5578-4b8a-b94d-c8c4a095d760") : secret "maas-api-e2e-inf-a-f63d20-serving-cert" not found
Oct 30 20:15:19.526853 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:19.526820    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1d1de99-5578-4b8a-b94d-c8c4a095d760-metrics-certs\") pod \"maas-api-e2e-inf-a-f63d20-7bdf8df68b-clm5w\" (UID: \"d1d1de99-5578-4b8a-b94d-c8c4a095d760\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f63d20-7bdf8df68b-clm5w"
Oct 30 20:15:19.552001 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:19.551955    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpt76\" (UniqueName: \"kubernetes.io/projected/d1d1de99-5578-4b8a-b94d-c8c4a095d760-kube-api-access-vpt76\") pod \"maas-api-e2e-inf-a-f63d20-7bdf8df68b-clm5w\" (UID: \"d1d1de99-5578-4b8a-b94d-c8c4a095d760\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f63d20-7bdf8df68b-clm5w"
Oct 30 20:15:20.060503 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:20.060091    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d1d1de99-5578-4b8a-b94d-c8c4a095d760-maas-api-tls\") pod \"maas-api-e2e-inf-a-f63d20-7bdf8df68b-clm5w\" (UID: \"d1d1de99-5578-4b8a-b94d-c8c4a095d760\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f63d20-7bdf8df68b-clm5w"
Oct 30 20:15:20.072855 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:20.072819    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d1d1de99-5578-4b8a-b94d-c8c4a095d760-maas-api-tls\") pod \"maas-api-e2e-inf-a-f63d20-7bdf8df68b-clm5w\" (UID: \"d1d1de99-5578-4b8a-b94d-c8c4a095d760\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f63d20-7bdf8df68b-clm5w"
Oct 30 20:15:20.220621 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:20.220585    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-7cbcbb-29889855-hgcqt"]
Oct 30 20:15:20.226400 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:20.226368    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f63d20-7bdf8df68b-clm5w"
Oct 30 20:15:20.502926 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:20.502896    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-7cbcbb-29889855-hgcqt"
Oct 30 20:15:20.600226 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:20.600144    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-f63d20-7bdf8df68b-clm5w"]
Oct 30 20:15:20.608515 ip-10-0-141-42 kubenswrapper[6996]: W1030 20:15:20.608348    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1d1de99_5578_4b8a_b94d_c8c4a095d760.slice/crio-99c7173122f8241c8c41cfa82eab150d14f1b0716cccf73848210127bd7f96ea WatchSource:0}: Error finding container 99c7173122f8241c8c41cfa82eab150d14f1b0716cccf73848210127bd7f96ea: Status 404 returned error can't find the container with id 99c7173122f8241c8c41cfa82eab150d14f1b0716cccf73848210127bd7f96ea
Oct 30 20:15:20.611673 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:20.611057    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7a68d2bd-f65c-4013-b700-3a0da95b301f-kube-api-access-zfztm\" (UniqueName: \"kubernetes.io/projected/7a68d2bd-f65c-4013-b700-3a0da95b301f-kube-api-access-zfztm\") pod \"7a68d2bd-f65c-4013-b700-3a0da95b301f\" (UID: \"7a68d2bd-f65c-4013-b700-3a0da95b301f\") "
Oct 30 20:15:20.611673 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:20.611144    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7a68d2bd-f65c-4013-b700-3a0da95b301f-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a68d2bd-f65c-4013-b700-3a0da95b301f-maas-api-service-ca\") pod \"7a68d2bd-f65c-4013-b700-3a0da95b301f\" (UID: \"7a68d2bd-f65c-4013-b700-3a0da95b301f\") "
Oct 30 20:15:20.612392 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:20.612366    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a68d2bd-f65c-4013-b700-3a0da95b301f-maas-api-service-ca" pod "7a68d2bd-f65c-4013-b700-3a0da95b301f" (UID: "7a68d2bd-f65c-4013-b700-3a0da95b301f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:15:20.614390 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:20.614366    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a68d2bd-f65c-4013-b700-3a0da95b301f-kube-api-access-zfztm" pod "7a68d2bd-f65c-4013-b700-3a0da95b301f" (UID: "7a68d2bd-f65c-4013-b700-3a0da95b301f"). InnerVolumeSpecName "kube-api-access-zfztm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:15:20.713180 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:20.713139    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zfztm\" (UniqueName: \"kubernetes.io/projected/7a68d2bd-f65c-4013-b700-3a0da95b301f-kube-api-access-zfztm\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:15:20.713180 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:20.713176    6996 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a68d2bd-f65c-4013-b700-3a0da95b301f-maas-api-service-ca\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:15:21.201522 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:21.201447    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f63d20-7bdf8df68b-clm5w" event={"ID":"d1d1de99-5578-4b8a-b94d-c8c4a095d760","Type":"ContainerStarted","Data":"40041b3b73f16d71ce6ee630d3555932a30bdd2c62fc5b8dfac6c75fae446034"}
Oct 30 20:15:21.201704 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:21.201528    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f63d20-7bdf8df68b-clm5w" event={"ID":"d1d1de99-5578-4b8a-b94d-c8c4a095d760","Type":"ContainerStarted","Data":"99c7173122f8241c8c41cfa82eab150d14f1b0716cccf73848210127bd7f96ea"}
Oct 30 20:15:21.201704 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:21.201588    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f63d20-7bdf8df68b-clm5w"
Oct 30 20:15:21.202904 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:21.202868    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-7cbcbb-29889855-hgcqt"
Oct 30 20:15:21.203030 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:21.202904    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-7cbcbb-29889855-hgcqt" event={"ID":"7a68d2bd-f65c-4013-b700-3a0da95b301f","Type":"ContainerDied","Data":"03686a98e28702408f6a3cd380e51c60d5b1cbdacd94de2d321b34846bef8f90"}
Oct 30 20:15:21.203030 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:21.202955    6996 scope.go:122] "RemoveContainer" containerID="ac4b835a02f9186a50c516abc9fdf8e0dfd880913dd081f4dea56d0f748206d7"
Oct 30 20:15:21.222803 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:21.222749    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f63d20-7bdf8df68b-clm5w" podStartSLOduration=1.746271685 podStartE2EDuration="2.222732835s" totalImagesPullingTime="476.46115ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:15:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:15:21.221217879 +0000 UTC m=+2596.549336034" watchObservedRunningTime="2026-10-30 20:15:21.222732835 +0000 UTC m=+2596.550850990"
Oct 30 20:15:21.242516 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:21.242472    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-7cbcbb-29889855-hgcqt"]
Oct 30 20:15:21.251085 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:21.251058    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-7cbcbb-29889855-hgcqt"]
Oct 30 20:15:22.792539 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:22.792505    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7a68d2bd-f65c-4013-b700-3a0da95b301f" path="/var/lib/kubelet/pods/7a68d2bd-f65c-4013-b700-3a0da95b301f/volumes"
Oct 30 20:15:27.217625 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:27.217593    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f63d20-7bdf8df68b-clm5w"
Oct 30 20:15:31.192035 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:15:31.191986    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 20:15:50.812057 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:50.812007    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 20:15:58.462420 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:15:58.462347    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-8c8241-79657d945b-5gwzv"]
Oct 30 20:16:01.191047 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:16:01.191004    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 20:16:03.463958 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:03.463902    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-extmod-a-8c8241-79657d945b-5gwzv" podUID="56aa21ff-1941-413f-99da-31997009371a" containerName="payload-processing" containerID="cri-o://94224f04750c1764f71a2c26afa4305d1a3767cce74e01166d9de07f1305274c" gracePeriod=25
Oct 30 20:16:05.907681 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:05.907648    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-extmod-a-8c8241-79657d945b-5gwzv" podUID="56aa21ff-1941-413f-99da-31997009371a" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 20:16:15.911982 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:15.911573    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-extmod-a-8c8241-79657d945b-5gwzv" podUID="56aa21ff-1941-413f-99da-31997009371a" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 20:16:20.273864 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:20.273797    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-c9bce8-66c79b8d4c-tr289"]
Oct 30 20:16:20.274385 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:20.274104    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c9bce8-66c79b8d4c-tr289" podUID="86effdca-6a15-4388-b069-9096d8c586c5" containerName="maas-api" containerID="cri-o://b479fd5b18f596ba8ccc8c241e6cfab4af0efa821b1f0e7562e332ac852870fc" gracePeriod=30
Oct 30 20:16:20.670063 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:20.670026    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c9bce8-66c79b8d4c-tr289"
Oct 30 20:16:20.702196 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:20.702156    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/86effdca-6a15-4388-b069-9096d8c586c5-metrics-certs\" (UniqueName: \"kubernetes.io/secret/86effdca-6a15-4388-b069-9096d8c586c5-metrics-certs\") pod \"86effdca-6a15-4388-b069-9096d8c586c5\" (UID: \"86effdca-6a15-4388-b069-9096d8c586c5\") "
Oct 30 20:16:20.702384 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:20.702220    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/86effdca-6a15-4388-b069-9096d8c586c5-kube-api-access-ddnb6\" (UniqueName: \"kubernetes.io/projected/86effdca-6a15-4388-b069-9096d8c586c5-kube-api-access-ddnb6\") pod \"86effdca-6a15-4388-b069-9096d8c586c5\" (UID: \"86effdca-6a15-4388-b069-9096d8c586c5\") "
Oct 30 20:16:20.702384 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:20.702272    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/86effdca-6a15-4388-b069-9096d8c586c5-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/86effdca-6a15-4388-b069-9096d8c586c5-openshift-service-ca\") pod \"86effdca-6a15-4388-b069-9096d8c586c5\" (UID: \"86effdca-6a15-4388-b069-9096d8c586c5\") "
Oct 30 20:16:20.702539 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:20.702453    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/86effdca-6a15-4388-b069-9096d8c586c5-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/86effdca-6a15-4388-b069-9096d8c586c5-maas-api-tls\") pod \"86effdca-6a15-4388-b069-9096d8c586c5\" (UID: \"86effdca-6a15-4388-b069-9096d8c586c5\") "
Oct 30 20:16:20.706340 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:20.705905    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86effdca-6a15-4388-b069-9096d8c586c5-maas-api-tls" pod "86effdca-6a15-4388-b069-9096d8c586c5" (UID: "86effdca-6a15-4388-b069-9096d8c586c5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:16:20.710556 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:20.707363    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86effdca-6a15-4388-b069-9096d8c586c5-openshift-service-ca" pod "86effdca-6a15-4388-b069-9096d8c586c5" (UID: "86effdca-6a15-4388-b069-9096d8c586c5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:16:20.723523 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:20.710963    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86effdca-6a15-4388-b069-9096d8c586c5-kube-api-access-ddnb6" pod "86effdca-6a15-4388-b069-9096d8c586c5" (UID: "86effdca-6a15-4388-b069-9096d8c586c5"). InnerVolumeSpecName "kube-api-access-ddnb6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:16:20.723523 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:20.717639    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86effdca-6a15-4388-b069-9096d8c586c5-metrics-certs" pod "86effdca-6a15-4388-b069-9096d8c586c5" (UID: "86effdca-6a15-4388-b069-9096d8c586c5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:16:20.806712 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:20.806561    6996 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/86effdca-6a15-4388-b069-9096d8c586c5-metrics-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:16:20.806712 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:20.806599    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ddnb6\" (UniqueName: \"kubernetes.io/projected/86effdca-6a15-4388-b069-9096d8c586c5-kube-api-access-ddnb6\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:16:20.806712 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:20.806616    6996 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/86effdca-6a15-4388-b069-9096d8c586c5-openshift-service-ca\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:16:20.806712 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:20.806632    6996 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/86effdca-6a15-4388-b069-9096d8c586c5-maas-api-tls\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:16:20.927226 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:20.927136    6996 generic.go:356] "Generic (PLEG): container finished" podID="86effdca-6a15-4388-b069-9096d8c586c5" containerID="b479fd5b18f596ba8ccc8c241e6cfab4af0efa821b1f0e7562e332ac852870fc" exitCode=0
Oct 30 20:16:20.927226 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:20.927216    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c9bce8-66c79b8d4c-tr289"
Oct 30 20:16:20.927492 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:20.927217    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c9bce8-66c79b8d4c-tr289" event={"ID":"86effdca-6a15-4388-b069-9096d8c586c5","Type":"ContainerDied","Data":"b479fd5b18f596ba8ccc8c241e6cfab4af0efa821b1f0e7562e332ac852870fc"}
Oct 30 20:16:20.927492 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:20.927354    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-c9bce8-66c79b8d4c-tr289" event={"ID":"86effdca-6a15-4388-b069-9096d8c586c5","Type":"ContainerDied","Data":"2ba306816f598215ba2dd142575f7b562002971efd4158890b0839813fb85e10"}
Oct 30 20:16:20.927492 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:20.927385    6996 scope.go:122] "RemoveContainer" containerID="b479fd5b18f596ba8ccc8c241e6cfab4af0efa821b1f0e7562e332ac852870fc"
Oct 30 20:16:20.939524 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:20.939472    6996 scope.go:122] "RemoveContainer" containerID="b479fd5b18f596ba8ccc8c241e6cfab4af0efa821b1f0e7562e332ac852870fc"
Oct 30 20:16:20.939825 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:16:20.939798    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b479fd5b18f596ba8ccc8c241e6cfab4af0efa821b1f0e7562e332ac852870fc\": container with ID starting with b479fd5b18f596ba8ccc8c241e6cfab4af0efa821b1f0e7562e332ac852870fc not found: ID does not exist" containerID="b479fd5b18f596ba8ccc8c241e6cfab4af0efa821b1f0e7562e332ac852870fc"
Oct 30 20:16:20.939922 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:20.939838    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b479fd5b18f596ba8ccc8c241e6cfab4af0efa821b1f0e7562e332ac852870fc"} err="failed to get container status \"b479fd5b18f596ba8ccc8c241e6cfab4af0efa821b1f0e7562e332ac852870fc\": rpc error: code = NotFound desc = could not find container \"b479fd5b18f596ba8ccc8c241e6cfab4af0efa821b1f0e7562e332ac852870fc\": container with ID starting with b479fd5b18f596ba8ccc8c241e6cfab4af0efa821b1f0e7562e332ac852870fc not found: ID does not exist"
Oct 30 20:16:20.951214 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:20.951188    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-c9bce8-66c79b8d4c-tr289"]
Oct 30 20:16:20.954821 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:20.954802    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-c9bce8-66c79b8d4c-tr289"]
Oct 30 20:16:22.793212 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:22.793167    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="86effdca-6a15-4388-b069-9096d8c586c5" path="/var/lib/kubelet/pods/86effdca-6a15-4388-b069-9096d8c586c5/volumes"
Oct 30 20:16:23.763329 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:23.763300    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-8c8241-79657d945b-5gwzv"
Oct 30 20:16:23.841897 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:23.841864    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c257e7-54b9f7d969-54pbk"]
Oct 30 20:16:23.842785 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:23.842762    6996 cpu_manager.go:414] "containerMap: removing container" podUID="56aa21ff-1941-413f-99da-31997009371a" containerName="payload-processing"
Oct 30 20:16:23.842785 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:23.842784    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="56aa21ff-1941-413f-99da-31997009371a" containerName="payload-processing"
Oct 30 20:16:23.843234 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:23.842802    6996 cpu_manager.go:414] "containerMap: removing container" podUID="7a68d2bd-f65c-4013-b700-3a0da95b301f" containerName="cleanup"
Oct 30 20:16:23.843234 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:23.842810    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7a68d2bd-f65c-4013-b700-3a0da95b301f" containerName="cleanup"
Oct 30 20:16:23.843234 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:23.842833    6996 cpu_manager.go:414] "containerMap: removing container" podUID="7a68d2bd-f65c-4013-b700-3a0da95b301f" containerName="cleanup"
Oct 30 20:16:23.843234 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:23.842843    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7a68d2bd-f65c-4013-b700-3a0da95b301f" containerName="cleanup"
Oct 30 20:16:23.843234 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:23.842873    6996 cpu_manager.go:414] "containerMap: removing container" podUID="7a68d2bd-f65c-4013-b700-3a0da95b301f" containerName="cleanup"
Oct 30 20:16:23.843234 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:23.842887    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7a68d2bd-f65c-4013-b700-3a0da95b301f" containerName="cleanup"
Oct 30 20:16:23.843234 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:23.842899    6996 cpu_manager.go:414] "containerMap: removing container" podUID="86effdca-6a15-4388-b069-9096d8c586c5" containerName="maas-api"
Oct 30 20:16:23.843234 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:23.842906    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="86effdca-6a15-4388-b069-9096d8c586c5" containerName="maas-api"
Oct 30 20:16:23.843234 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:23.843121    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="56aa21ff-1941-413f-99da-31997009371a" containerName="payload-processing"
Oct 30 20:16:23.843234 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:23.843139    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="7a68d2bd-f65c-4013-b700-3a0da95b301f" containerName="cleanup"
Oct 30 20:16:23.843234 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:23.843150    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="7a68d2bd-f65c-4013-b700-3a0da95b301f" containerName="cleanup"
Oct 30 20:16:23.843234 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:23.843162    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="86effdca-6a15-4388-b069-9096d8c586c5" containerName="maas-api"
Oct 30 20:16:23.847136 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:23.847117    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c257e7-54b9f7d969-54pbk"
Oct 30 20:16:23.849984 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:23.849961    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/56aa21ff-1941-413f-99da-31997009371a-kube-api-access-b586r\" (UniqueName: \"kubernetes.io/projected/56aa21ff-1941-413f-99da-31997009371a-kube-api-access-b586r\") pod \"56aa21ff-1941-413f-99da-31997009371a\" (UID: \"56aa21ff-1941-413f-99da-31997009371a\") "
Oct 30 20:16:23.850090 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:23.850000    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/56aa21ff-1941-413f-99da-31997009371a-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56aa21ff-1941-413f-99da-31997009371a-plugins-config-volume\") pod \"56aa21ff-1941-413f-99da-31997009371a\" (UID: \"56aa21ff-1941-413f-99da-31997009371a\") "
Oct 30 20:16:23.850788 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:23.850754    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56aa21ff-1941-413f-99da-31997009371a-plugins-config-volume" pod "56aa21ff-1941-413f-99da-31997009371a" (UID: "56aa21ff-1941-413f-99da-31997009371a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:16:23.852677 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:23.852570    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-c257e7-serving-cert\""
Oct 30 20:16:23.852677 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:23.852638    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56aa21ff-1941-413f-99da-31997009371a-kube-api-access-b586r" pod "56aa21ff-1941-413f-99da-31997009371a" (UID: "56aa21ff-1941-413f-99da-31997009371a"). InnerVolumeSpecName "kube-api-access-b586r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:16:23.881683 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:23.881609    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c257e7-54b9f7d969-54pbk"]
Oct 30 20:16:23.951388 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:23.951346    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60867c87-cc28-4053-92d6-7f3f6024edc2-metrics-certs\") pod \"maas-api-e2e-inf-b-c257e7-54b9f7d969-54pbk\" (UID: \"60867c87-cc28-4053-92d6-7f3f6024edc2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c257e7-54b9f7d969-54pbk"
Oct 30 20:16:23.951620 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:23.951468    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60867c87-cc28-4053-92d6-7f3f6024edc2-maas-api-tls\") pod \"maas-api-e2e-inf-b-c257e7-54b9f7d969-54pbk\" (UID: \"60867c87-cc28-4053-92d6-7f3f6024edc2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c257e7-54b9f7d969-54pbk"
Oct 30 20:16:23.951620 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:23.951562    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60867c87-cc28-4053-92d6-7f3f6024edc2-openshift-service-ca\") pod \"maas-api-e2e-inf-b-c257e7-54b9f7d969-54pbk\" (UID: \"60867c87-cc28-4053-92d6-7f3f6024edc2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c257e7-54b9f7d969-54pbk"
Oct 30 20:16:23.951620 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:23.951602    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xxw6\" (UniqueName: \"kubernetes.io/projected/60867c87-cc28-4053-92d6-7f3f6024edc2-kube-api-access-4xxw6\") pod \"maas-api-e2e-inf-b-c257e7-54b9f7d969-54pbk\" (UID: \"60867c87-cc28-4053-92d6-7f3f6024edc2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c257e7-54b9f7d969-54pbk"
Oct 30 20:16:23.951893 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:23.951862    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b586r\" (UniqueName: \"kubernetes.io/projected/56aa21ff-1941-413f-99da-31997009371a-kube-api-access-b586r\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:16:23.951893 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:23.951884    6996 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56aa21ff-1941-413f-99da-31997009371a-plugins-config-volume\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:16:23.958444 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:23.958413    6996 generic.go:356] "Generic (PLEG): container finished" podID="56aa21ff-1941-413f-99da-31997009371a" containerID="94224f04750c1764f71a2c26afa4305d1a3767cce74e01166d9de07f1305274c" exitCode=0
Oct 30 20:16:23.958651 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:23.958473    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-8c8241-79657d945b-5gwzv"
Oct 30 20:16:23.958651 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:23.958513    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-8c8241-79657d945b-5gwzv" event={"ID":"56aa21ff-1941-413f-99da-31997009371a","Type":"ContainerDied","Data":"94224f04750c1764f71a2c26afa4305d1a3767cce74e01166d9de07f1305274c"}
Oct 30 20:16:23.958651 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:23.958595    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-8c8241-79657d945b-5gwzv" event={"ID":"56aa21ff-1941-413f-99da-31997009371a","Type":"ContainerDied","Data":"4526ef898d1ac2bb51d6b7dc9d2a6bced43e1b1995d7f5fd9c08007fabd76cda"}
Oct 30 20:16:23.958651 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:23.958629    6996 scope.go:122] "RemoveContainer" containerID="94224f04750c1764f71a2c26afa4305d1a3767cce74e01166d9de07f1305274c"
Oct 30 20:16:23.978532 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:23.976157    6996 scope.go:122] "RemoveContainer" containerID="94224f04750c1764f71a2c26afa4305d1a3767cce74e01166d9de07f1305274c"
Oct 30 20:16:23.978532 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:16:23.976681    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94224f04750c1764f71a2c26afa4305d1a3767cce74e01166d9de07f1305274c\": container with ID starting with 94224f04750c1764f71a2c26afa4305d1a3767cce74e01166d9de07f1305274c not found: ID does not exist" containerID="94224f04750c1764f71a2c26afa4305d1a3767cce74e01166d9de07f1305274c"
Oct 30 20:16:23.978532 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:23.976724    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94224f04750c1764f71a2c26afa4305d1a3767cce74e01166d9de07f1305274c"} err="failed to get container status \"94224f04750c1764f71a2c26afa4305d1a3767cce74e01166d9de07f1305274c\": rpc error: code = NotFound desc = could not find container \"94224f04750c1764f71a2c26afa4305d1a3767cce74e01166d9de07f1305274c\": container with ID starting with 94224f04750c1764f71a2c26afa4305d1a3767cce74e01166d9de07f1305274c not found: ID does not exist"
Oct 30 20:16:24.044213 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:24.042749    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-8c8241-79657d945b-5gwzv"]
Oct 30 20:16:24.046963 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:24.046929    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-8c8241-79657d945b-5gwzv"]
Oct 30 20:16:24.052891 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:24.052862    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60867c87-cc28-4053-92d6-7f3f6024edc2-metrics-certs\") pod \"maas-api-e2e-inf-b-c257e7-54b9f7d969-54pbk\" (UID: \"60867c87-cc28-4053-92d6-7f3f6024edc2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c257e7-54b9f7d969-54pbk"
Oct 30 20:16:24.053058 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:24.052921    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60867c87-cc28-4053-92d6-7f3f6024edc2-maas-api-tls\") pod \"maas-api-e2e-inf-b-c257e7-54b9f7d969-54pbk\" (UID: \"60867c87-cc28-4053-92d6-7f3f6024edc2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c257e7-54b9f7d969-54pbk"
Oct 30 20:16:24.053058 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:24.052961    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60867c87-cc28-4053-92d6-7f3f6024edc2-openshift-service-ca\") pod \"maas-api-e2e-inf-b-c257e7-54b9f7d969-54pbk\" (UID: \"60867c87-cc28-4053-92d6-7f3f6024edc2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c257e7-54b9f7d969-54pbk"
Oct 30 20:16:24.053058 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:24.052990    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xxw6\" (UniqueName: \"kubernetes.io/projected/60867c87-cc28-4053-92d6-7f3f6024edc2-kube-api-access-4xxw6\") pod \"maas-api-e2e-inf-b-c257e7-54b9f7d969-54pbk\" (UID: \"60867c87-cc28-4053-92d6-7f3f6024edc2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c257e7-54b9f7d969-54pbk"
Oct 30 20:16:24.053718 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:24.053693    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60867c87-cc28-4053-92d6-7f3f6024edc2-openshift-service-ca\") pod \"maas-api-e2e-inf-b-c257e7-54b9f7d969-54pbk\" (UID: \"60867c87-cc28-4053-92d6-7f3f6024edc2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c257e7-54b9f7d969-54pbk"
Oct 30 20:16:24.055575 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:24.055551    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60867c87-cc28-4053-92d6-7f3f6024edc2-metrics-certs\") pod \"maas-api-e2e-inf-b-c257e7-54b9f7d969-54pbk\" (UID: \"60867c87-cc28-4053-92d6-7f3f6024edc2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c257e7-54b9f7d969-54pbk"
Oct 30 20:16:24.055575 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:24.055569    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60867c87-cc28-4053-92d6-7f3f6024edc2-maas-api-tls\") pod \"maas-api-e2e-inf-b-c257e7-54b9f7d969-54pbk\" (UID: \"60867c87-cc28-4053-92d6-7f3f6024edc2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c257e7-54b9f7d969-54pbk"
Oct 30 20:16:24.065894 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:24.065863    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xxw6\" (UniqueName: \"kubernetes.io/projected/60867c87-cc28-4053-92d6-7f3f6024edc2-kube-api-access-4xxw6\") pod \"maas-api-e2e-inf-b-c257e7-54b9f7d969-54pbk\" (UID: \"60867c87-cc28-4053-92d6-7f3f6024edc2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c257e7-54b9f7d969-54pbk"
Oct 30 20:16:24.192721 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:24.192686    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c257e7-54b9f7d969-54pbk"
Oct 30 20:16:24.529774 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:24.506747    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c257e7-54b9f7d969-54pbk"]
Oct 30 20:16:24.537729 ip-10-0-141-42 kubenswrapper[6996]: W1030 20:16:24.531432    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60867c87_cc28_4053_92d6_7f3f6024edc2.slice/crio-3d754c414dbf3eaa19cbbd5126e2600aae065c7b32f55a68b903f247a383ba98 WatchSource:0}: Error finding container 3d754c414dbf3eaa19cbbd5126e2600aae065c7b32f55a68b903f247a383ba98: Status 404 returned error can't find the container with id 3d754c414dbf3eaa19cbbd5126e2600aae065c7b32f55a68b903f247a383ba98
Oct 30 20:16:24.802436 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:24.802359    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="56aa21ff-1941-413f-99da-31997009371a" path="/var/lib/kubelet/pods/56aa21ff-1941-413f-99da-31997009371a/volumes"
Oct 30 20:16:24.971186 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:24.971144    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c257e7-54b9f7d969-54pbk" event={"ID":"60867c87-cc28-4053-92d6-7f3f6024edc2","Type":"ContainerStarted","Data":"3d754c414dbf3eaa19cbbd5126e2600aae065c7b32f55a68b903f247a383ba98"}
Oct 30 20:16:25.981596 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:25.981561    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c257e7-54b9f7d969-54pbk" event={"ID":"60867c87-cc28-4053-92d6-7f3f6024edc2","Type":"ContainerStarted","Data":"7e887c2e07cf97d28b7c5a610abf9cf98dc9202883b4a9b66c0b128f7e0e93c8"}
Oct 30 20:16:25.982034 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:25.981694    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c257e7-54b9f7d969-54pbk"
Oct 30 20:16:26.046951 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:26.046887    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c257e7-54b9f7d969-54pbk" podStartSLOduration=2.5702500329999998 podStartE2EDuration="3.046866272s" totalImagesPullingTime="476.616239ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:16:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:16:26.045829333 +0000 UTC m=+2661.373947486" watchObservedRunningTime="2026-10-30 20:16:26.046866272 +0000 UTC m=+2661.374984426"
Oct 30 20:16:31.190898 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:16:31.190851    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 20:16:31.995619 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:31.995586    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c257e7-54b9f7d969-54pbk"
Oct 30 20:16:42.688415 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:42.688373    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 20:16:43.478539 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:43.476729    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-969878-d8f4759b7-kdsjv"]
Oct 30 20:16:43.478539 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:43.476988    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-969878-d8f4759b7-kdsjv" podUID="89a06251-4c13-4b46-99fb-6b3db0c4ddf6" containerName="maas-api" containerID="cri-o://99b2de11388801ed32596c535fe72ae5288e673c9cd4367f0c854a38c3762861" gracePeriod=30
Oct 30 20:16:43.897116 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:43.897058    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-969878-d8f4759b7-kdsjv"
Oct 30 20:16:44.070503 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:44.069220    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/89a06251-4c13-4b46-99fb-6b3db0c4ddf6-kube-api-access-nzrfx\" (UniqueName: \"kubernetes.io/projected/89a06251-4c13-4b46-99fb-6b3db0c4ddf6-kube-api-access-nzrfx\") pod \"89a06251-4c13-4b46-99fb-6b3db0c4ddf6\" (UID: \"89a06251-4c13-4b46-99fb-6b3db0c4ddf6\") "
Oct 30 20:16:44.070503 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:44.069372    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/89a06251-4c13-4b46-99fb-6b3db0c4ddf6-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/89a06251-4c13-4b46-99fb-6b3db0c4ddf6-openshift-service-ca\") pod \"89a06251-4c13-4b46-99fb-6b3db0c4ddf6\" (UID: \"89a06251-4c13-4b46-99fb-6b3db0c4ddf6\") "
Oct 30 20:16:44.070503 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:44.069417    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/89a06251-4c13-4b46-99fb-6b3db0c4ddf6-metrics-certs\" (UniqueName: \"kubernetes.io/secret/89a06251-4c13-4b46-99fb-6b3db0c4ddf6-metrics-certs\") pod \"89a06251-4c13-4b46-99fb-6b3db0c4ddf6\" (UID: \"89a06251-4c13-4b46-99fb-6b3db0c4ddf6\") "
Oct 30 20:16:44.070503 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:44.070036    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/89a06251-4c13-4b46-99fb-6b3db0c4ddf6-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89a06251-4c13-4b46-99fb-6b3db0c4ddf6-maas-api-tls\") pod \"89a06251-4c13-4b46-99fb-6b3db0c4ddf6\" (UID: \"89a06251-4c13-4b46-99fb-6b3db0c4ddf6\") "
Oct 30 20:16:44.083564 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:44.071145    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89a06251-4c13-4b46-99fb-6b3db0c4ddf6-openshift-service-ca" pod "89a06251-4c13-4b46-99fb-6b3db0c4ddf6" (UID: "89a06251-4c13-4b46-99fb-6b3db0c4ddf6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:16:44.083564 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:44.073509    6996 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/89a06251-4c13-4b46-99fb-6b3db0c4ddf6-openshift-service-ca\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:16:44.083564 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:44.079864    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/89a06251-4c13-4b46-99fb-6b3db0c4ddf6-metrics-certs" pod "89a06251-4c13-4b46-99fb-6b3db0c4ddf6" (UID: "89a06251-4c13-4b46-99fb-6b3db0c4ddf6"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:16:44.083564 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:44.081603    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/89a06251-4c13-4b46-99fb-6b3db0c4ddf6-maas-api-tls" pod "89a06251-4c13-4b46-99fb-6b3db0c4ddf6" (UID: "89a06251-4c13-4b46-99fb-6b3db0c4ddf6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:16:44.083564 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:44.082053    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89a06251-4c13-4b46-99fb-6b3db0c4ddf6-kube-api-access-nzrfx" pod "89a06251-4c13-4b46-99fb-6b3db0c4ddf6" (UID: "89a06251-4c13-4b46-99fb-6b3db0c4ddf6"). InnerVolumeSpecName "kube-api-access-nzrfx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:16:44.137976 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:44.134655    6996 generic.go:356] "Generic (PLEG): container finished" podID="89a06251-4c13-4b46-99fb-6b3db0c4ddf6" containerID="99b2de11388801ed32596c535fe72ae5288e673c9cd4367f0c854a38c3762861" exitCode=0
Oct 30 20:16:44.137976 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:44.134840    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-969878-d8f4759b7-kdsjv" event={"ID":"89a06251-4c13-4b46-99fb-6b3db0c4ddf6","Type":"ContainerDied","Data":"99b2de11388801ed32596c535fe72ae5288e673c9cd4367f0c854a38c3762861"}
Oct 30 20:16:44.137976 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:44.134925    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-969878-d8f4759b7-kdsjv" event={"ID":"89a06251-4c13-4b46-99fb-6b3db0c4ddf6","Type":"ContainerDied","Data":"edc3339fd991b871b6ed3140d97d370514456eb0ea45affc101c2f5e903abc95"}
Oct 30 20:16:44.137976 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:44.134960    6996 scope.go:122] "RemoveContainer" containerID="99b2de11388801ed32596c535fe72ae5288e673c9cd4367f0c854a38c3762861"
Oct 30 20:16:44.137976 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:44.135139    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-969878-d8f4759b7-kdsjv"
Oct 30 20:16:44.193396 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:44.192580    6996 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89a06251-4c13-4b46-99fb-6b3db0c4ddf6-metrics-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:16:44.193396 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:44.192614    6996 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89a06251-4c13-4b46-99fb-6b3db0c4ddf6-maas-api-tls\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:16:44.193396 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:44.192631    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nzrfx\" (UniqueName: \"kubernetes.io/projected/89a06251-4c13-4b46-99fb-6b3db0c4ddf6-kube-api-access-nzrfx\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:16:44.195924 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:44.195888    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-969878-d8f4759b7-kdsjv"]
Oct 30 20:16:44.196666 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:44.196190    6996 scope.go:122] "RemoveContainer" containerID="99b2de11388801ed32596c535fe72ae5288e673c9cd4367f0c854a38c3762861"
Oct 30 20:16:44.197265 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:16:44.197189    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99b2de11388801ed32596c535fe72ae5288e673c9cd4367f0c854a38c3762861\": container with ID starting with 99b2de11388801ed32596c535fe72ae5288e673c9cd4367f0c854a38c3762861 not found: ID does not exist" containerID="99b2de11388801ed32596c535fe72ae5288e673c9cd4367f0c854a38c3762861"
Oct 30 20:16:44.197265 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:44.197224    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99b2de11388801ed32596c535fe72ae5288e673c9cd4367f0c854a38c3762861"} err="failed to get container status \"99b2de11388801ed32596c535fe72ae5288e673c9cd4367f0c854a38c3762861\": rpc error: code = NotFound desc = could not find container \"99b2de11388801ed32596c535fe72ae5288e673c9cd4367f0c854a38c3762861\": container with ID starting with 99b2de11388801ed32596c535fe72ae5288e673c9cd4367f0c854a38c3762861 not found: ID does not exist"
Oct 30 20:16:44.199989 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:44.199967    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-969878-d8f4759b7-kdsjv"]
Oct 30 20:16:44.795331 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:16:44.795080    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="89a06251-4c13-4b46-99fb-6b3db0c4ddf6" path="/var/lib/kubelet/pods/89a06251-4c13-4b46-99fb-6b3db0c4ddf6/volumes"
Oct 30 20:17:00.876343 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:00.876307    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md"]
Oct 30 20:17:00.877300 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:00.877241    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md" podUID="06a16fac-9e8d-4724-a949-a8d31f13002d" containerName="istio-proxy" containerID="cri-o://59bb9556f598b6daedb4c97d1431d4962734088763306faace05e43981e642dd" gracePeriod=30
Oct 30 20:17:01.198109 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:17:01.198016    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 20:17:06.052257 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:06.052214    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-b7431c-5dbb549c45-hsk6h"]
Oct 30 20:17:06.052960 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:06.052505    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-b7431c-5dbb549c45-hsk6h" podUID="80f2d362-3093-42b0-a3c0-c8bf25c43a84" containerName="maas-api" containerID="cri-o://3de50dc33a2c987755a08c5085df845677106e6247f55677ea258253cce66173" gracePeriod=30
Oct 30 20:17:06.208291 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:06.208266    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md"
Oct 30 20:17:06.223094 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:06.223063    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/06a16fac-9e8d-4724-a949-a8d31f13002d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06a16fac-9e8d-4724-a949-a8d31f13002d-credential-socket\") pod \"06a16fac-9e8d-4724-a949-a8d31f13002d\" (UID: \"06a16fac-9e8d-4724-a949-a8d31f13002d\") "
Oct 30 20:17:06.223295 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:06.223171    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/06a16fac-9e8d-4724-a949-a8d31f13002d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06a16fac-9e8d-4724-a949-a8d31f13002d-workload-certs\") pod \"06a16fac-9e8d-4724-a949-a8d31f13002d\" (UID: \"06a16fac-9e8d-4724-a949-a8d31f13002d\") "
Oct 30 20:17:06.223295 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:06.223211    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/06a16fac-9e8d-4724-a949-a8d31f13002d-kube-api-access-8jtjc\" (UniqueName: \"kubernetes.io/projected/06a16fac-9e8d-4724-a949-a8d31f13002d-kube-api-access-8jtjc\") pod \"06a16fac-9e8d-4724-a949-a8d31f13002d\" (UID: \"06a16fac-9e8d-4724-a949-a8d31f13002d\") "
Oct 30 20:17:06.223528 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:06.223316    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/06a16fac-9e8d-4724-a949-a8d31f13002d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06a16fac-9e8d-4724-a949-a8d31f13002d-workload-socket\") pod \"06a16fac-9e8d-4724-a949-a8d31f13002d\" (UID: \"06a16fac-9e8d-4724-a949-a8d31f13002d\") "
Oct 30 20:17:06.223528 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:06.223348    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/06a16fac-9e8d-4724-a949-a8d31f13002d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06a16fac-9e8d-4724-a949-a8d31f13002d-istio-data\") pod \"06a16fac-9e8d-4724-a949-a8d31f13002d\" (UID: \"06a16fac-9e8d-4724-a949-a8d31f13002d\") "
Oct 30 20:17:06.223528 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:06.223362    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06a16fac-9e8d-4724-a949-a8d31f13002d-credential-socket" pod "06a16fac-9e8d-4724-a949-a8d31f13002d" (UID: "06a16fac-9e8d-4724-a949-a8d31f13002d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:17:06.223528 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:06.223389    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/06a16fac-9e8d-4724-a949-a8d31f13002d-istio-token\" (UniqueName: \"kubernetes.io/projected/06a16fac-9e8d-4724-a949-a8d31f13002d-istio-token\") pod \"06a16fac-9e8d-4724-a949-a8d31f13002d\" (UID: \"06a16fac-9e8d-4724-a949-a8d31f13002d\") "
Oct 30 20:17:06.223528 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:06.223431    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/06a16fac-9e8d-4724-a949-a8d31f13002d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06a16fac-9e8d-4724-a949-a8d31f13002d-istio-envoy\") pod \"06a16fac-9e8d-4724-a949-a8d31f13002d\" (UID: \"06a16fac-9e8d-4724-a949-a8d31f13002d\") "
Oct 30 20:17:06.223528 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:06.223460    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/06a16fac-9e8d-4724-a949-a8d31f13002d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06a16fac-9e8d-4724-a949-a8d31f13002d-istio-podinfo\") pod \"06a16fac-9e8d-4724-a949-a8d31f13002d\" (UID: \"06a16fac-9e8d-4724-a949-a8d31f13002d\") "
Oct 30 20:17:06.223970 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:06.223535    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06a16fac-9e8d-4724-a949-a8d31f13002d-workload-certs" pod "06a16fac-9e8d-4724-a949-a8d31f13002d" (UID: "06a16fac-9e8d-4724-a949-a8d31f13002d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:17:06.223970 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:06.223555    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/06a16fac-9e8d-4724-a949-a8d31f13002d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06a16fac-9e8d-4724-a949-a8d31f13002d-istiod-ca-cert\") pod \"06a16fac-9e8d-4724-a949-a8d31f13002d\" (UID: \"06a16fac-9e8d-4724-a949-a8d31f13002d\") "
Oct 30 20:17:06.223970 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:06.223621    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06a16fac-9e8d-4724-a949-a8d31f13002d-istio-data" pod "06a16fac-9e8d-4724-a949-a8d31f13002d" (UID: "06a16fac-9e8d-4724-a949-a8d31f13002d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:17:06.224516 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:06.224176    6996 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06a16fac-9e8d-4724-a949-a8d31f13002d-istio-data\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:17:06.224516 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:06.224193    6996 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06a16fac-9e8d-4724-a949-a8d31f13002d-credential-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:17:06.224516 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:06.224204    6996 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06a16fac-9e8d-4724-a949-a8d31f13002d-workload-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:17:06.224692 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:06.224577    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06a16fac-9e8d-4724-a949-a8d31f13002d-istiod-ca-cert" pod "06a16fac-9e8d-4724-a949-a8d31f13002d" (UID: "06a16fac-9e8d-4724-a949-a8d31f13002d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:17:06.224692 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:06.224640    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06a16fac-9e8d-4724-a949-a8d31f13002d-workload-socket" pod "06a16fac-9e8d-4724-a949-a8d31f13002d" (UID: "06a16fac-9e8d-4724-a949-a8d31f13002d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:17:06.226132 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:06.226102    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06a16fac-9e8d-4724-a949-a8d31f13002d-istio-envoy" pod "06a16fac-9e8d-4724-a949-a8d31f13002d" (UID: "06a16fac-9e8d-4724-a949-a8d31f13002d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:17:06.226921 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:06.226900    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06a16fac-9e8d-4724-a949-a8d31f13002d-kube-api-access-8jtjc" pod "06a16fac-9e8d-4724-a949-a8d31f13002d" (UID: "06a16fac-9e8d-4724-a949-a8d31f13002d"). InnerVolumeSpecName "kube-api-access-8jtjc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:17:06.228359 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:06.228329    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/06a16fac-9e8d-4724-a949-a8d31f13002d-istio-podinfo" pod "06a16fac-9e8d-4724-a949-a8d31f13002d" (UID: "06a16fac-9e8d-4724-a949-a8d31f13002d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 20:17:06.228456 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:06.228431    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06a16fac-9e8d-4724-a949-a8d31f13002d-istio-token" pod "06a16fac-9e8d-4724-a949-a8d31f13002d" (UID: "06a16fac-9e8d-4724-a949-a8d31f13002d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:17:06.326277 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:06.326233    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8jtjc\" (UniqueName: \"kubernetes.io/projected/06a16fac-9e8d-4724-a949-a8d31f13002d-kube-api-access-8jtjc\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:17:06.326277 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:06.326279    6996 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06a16fac-9e8d-4724-a949-a8d31f13002d-workload-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:17:06.326524 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:06.326296    6996 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06a16fac-9e8d-4724-a949-a8d31f13002d-istio-token\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:17:06.326524 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:06.326310    6996 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06a16fac-9e8d-4724-a949-a8d31f13002d-istio-envoy\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:17:06.326524 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:06.326324    6996 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06a16fac-9e8d-4724-a949-a8d31f13002d-istio-podinfo\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:17:06.326524 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:06.326340    6996 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06a16fac-9e8d-4724-a949-a8d31f13002d-istiod-ca-cert\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:17:06.334235 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:06.334199    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-b7431c-5dbb549c45-hsk6h"
Oct 30 20:17:06.461561 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:06.452047    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/80f2d362-3093-42b0-a3c0-c8bf25c43a84-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80f2d362-3093-42b0-a3c0-c8bf25c43a84-maas-api-tls\") pod \"80f2d362-3093-42b0-a3c0-c8bf25c43a84\" (UID: \"80f2d362-3093-42b0-a3c0-c8bf25c43a84\") "
Oct 30 20:17:06.461561 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:06.452157    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/80f2d362-3093-42b0-a3c0-c8bf25c43a84-kube-api-access-n7qdz\" (UniqueName: \"kubernetes.io/projected/80f2d362-3093-42b0-a3c0-c8bf25c43a84-kube-api-access-n7qdz\") pod \"80f2d362-3093-42b0-a3c0-c8bf25c43a84\" (UID: \"80f2d362-3093-42b0-a3c0-c8bf25c43a84\") "
Oct 30 20:17:06.461561 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:06.452419    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/80f2d362-3093-42b0-a3c0-c8bf25c43a84-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80f2d362-3093-42b0-a3c0-c8bf25c43a84-openshift-service-ca\") pod \"80f2d362-3093-42b0-a3c0-c8bf25c43a84\" (UID: \"80f2d362-3093-42b0-a3c0-c8bf25c43a84\") "
Oct 30 20:17:06.461561 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:06.452467    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/80f2d362-3093-42b0-a3c0-c8bf25c43a84-metrics-certs\" (UniqueName: \"kubernetes.io/secret/80f2d362-3093-42b0-a3c0-c8bf25c43a84-metrics-certs\") pod \"80f2d362-3093-42b0-a3c0-c8bf25c43a84\" (UID: \"80f2d362-3093-42b0-a3c0-c8bf25c43a84\") "
Oct 30 20:17:06.474099 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:06.469444    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80f2d362-3093-42b0-a3c0-c8bf25c43a84-openshift-service-ca" pod "80f2d362-3093-42b0-a3c0-c8bf25c43a84" (UID: "80f2d362-3093-42b0-a3c0-c8bf25c43a84"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:17:06.474099 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:06.471320    6996 generic.go:356] "Generic (PLEG): container finished" podID="06a16fac-9e8d-4724-a949-a8d31f13002d" containerID="59bb9556f598b6daedb4c97d1431d4962734088763306faace05e43981e642dd" exitCode=0
Oct 30 20:17:06.474099 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:06.471389    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md"
Oct 30 20:17:06.474099 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:06.471434    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md" event={"ID":"06a16fac-9e8d-4724-a949-a8d31f13002d","Type":"ContainerDied","Data":"59bb9556f598b6daedb4c97d1431d4962734088763306faace05e43981e642dd"}
Oct 30 20:17:06.474099 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:06.471502    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md" event={"ID":"06a16fac-9e8d-4724-a949-a8d31f13002d","Type":"ContainerDied","Data":"3650f0279d486971dcae4e2663bc0e45fbfc7408b5835126d1b953d35238711f"}
Oct 30 20:17:06.474099 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:06.471531    6996 scope.go:122] "RemoveContainer" containerID="59bb9556f598b6daedb4c97d1431d4962734088763306faace05e43981e642dd"
Oct 30 20:17:06.474099 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:06.472622    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80f2d362-3093-42b0-a3c0-c8bf25c43a84-kube-api-access-n7qdz" pod "80f2d362-3093-42b0-a3c0-c8bf25c43a84" (UID: "80f2d362-3093-42b0-a3c0-c8bf25c43a84"). InnerVolumeSpecName "kube-api-access-n7qdz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:17:06.474099 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:06.473199    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/80f2d362-3093-42b0-a3c0-c8bf25c43a84-maas-api-tls" pod "80f2d362-3093-42b0-a3c0-c8bf25c43a84" (UID: "80f2d362-3093-42b0-a3c0-c8bf25c43a84"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:17:06.476342 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:06.476137    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-b7431c-5dbb549c45-hsk6h"
Oct 30 20:17:06.476530 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:06.476505    6996 generic.go:356] "Generic (PLEG): container finished" podID="80f2d362-3093-42b0-a3c0-c8bf25c43a84" containerID="3de50dc33a2c987755a08c5085df845677106e6247f55677ea258253cce66173" exitCode=0
Oct 30 20:17:06.477750 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:06.476639    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-b7431c-5dbb549c45-hsk6h" event={"ID":"80f2d362-3093-42b0-a3c0-c8bf25c43a84","Type":"ContainerDied","Data":"3de50dc33a2c987755a08c5085df845677106e6247f55677ea258253cce66173"}
Oct 30 20:17:06.477937 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:06.477910    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-b7431c-5dbb549c45-hsk6h" event={"ID":"80f2d362-3093-42b0-a3c0-c8bf25c43a84","Type":"ContainerDied","Data":"6f68858ec028d1c2f25b549737ea3ce4318fb6ef7aaa6eb4070a10878f08ceae"}
Oct 30 20:17:06.491270 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:06.491177    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/80f2d362-3093-42b0-a3c0-c8bf25c43a84-metrics-certs" pod "80f2d362-3093-42b0-a3c0-c8bf25c43a84" (UID: "80f2d362-3093-42b0-a3c0-c8bf25c43a84"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:17:06.497403 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:06.496824    6996 scope.go:122] "RemoveContainer" containerID="59bb9556f598b6daedb4c97d1431d4962734088763306faace05e43981e642dd"
Oct 30 20:17:06.497403 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:17:06.497187    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59bb9556f598b6daedb4c97d1431d4962734088763306faace05e43981e642dd\": container with ID starting with 59bb9556f598b6daedb4c97d1431d4962734088763306faace05e43981e642dd not found: ID does not exist" containerID="59bb9556f598b6daedb4c97d1431d4962734088763306faace05e43981e642dd"
Oct 30 20:17:06.497403 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:06.497221    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59bb9556f598b6daedb4c97d1431d4962734088763306faace05e43981e642dd"} err="failed to get container status \"59bb9556f598b6daedb4c97d1431d4962734088763306faace05e43981e642dd\": rpc error: code = NotFound desc = could not find container \"59bb9556f598b6daedb4c97d1431d4962734088763306faace05e43981e642dd\": container with ID starting with 59bb9556f598b6daedb4c97d1431d4962734088763306faace05e43981e642dd not found: ID does not exist"
Oct 30 20:17:06.497403 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:06.497238    6996 scope.go:122] "RemoveContainer" containerID="3de50dc33a2c987755a08c5085df845677106e6247f55677ea258253cce66173"
Oct 30 20:17:06.515592 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:06.515553    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md"]
Oct 30 20:17:06.517233 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:06.517216    6996 scope.go:122] "RemoveContainer" containerID="3de50dc33a2c987755a08c5085df845677106e6247f55677ea258253cce66173"
Oct 30 20:17:06.517765 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:17:06.517733    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3de50dc33a2c987755a08c5085df845677106e6247f55677ea258253cce66173\": container with ID starting with 3de50dc33a2c987755a08c5085df845677106e6247f55677ea258253cce66173 not found: ID does not exist" containerID="3de50dc33a2c987755a08c5085df845677106e6247f55677ea258253cce66173"
Oct 30 20:17:06.517859 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:06.517781    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-b-d97cc8-openshift-default-665557959c-vf4md"]
Oct 30 20:17:06.517859 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:06.517776    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3de50dc33a2c987755a08c5085df845677106e6247f55677ea258253cce66173"} err="failed to get container status \"3de50dc33a2c987755a08c5085df845677106e6247f55677ea258253cce66173\": rpc error: code = NotFound desc = could not find container \"3de50dc33a2c987755a08c5085df845677106e6247f55677ea258253cce66173\": container with ID starting with 3de50dc33a2c987755a08c5085df845677106e6247f55677ea258253cce66173 not found: ID does not exist"
Oct 30 20:17:06.554978 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:06.554924    6996 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80f2d362-3093-42b0-a3c0-c8bf25c43a84-openshift-service-ca\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:17:06.554978 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:06.554971    6996 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80f2d362-3093-42b0-a3c0-c8bf25c43a84-metrics-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:17:06.554978 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:06.554987    6996 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80f2d362-3093-42b0-a3c0-c8bf25c43a84-maas-api-tls\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:17:06.555368 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:06.555002    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n7qdz\" (UniqueName: \"kubernetes.io/projected/80f2d362-3093-42b0-a3c0-c8bf25c43a84-kube-api-access-n7qdz\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:17:06.795634 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:06.795595    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="06a16fac-9e8d-4724-a949-a8d31f13002d" path="/var/lib/kubelet/pods/06a16fac-9e8d-4724-a949-a8d31f13002d/volumes"
Oct 30 20:17:06.802429 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:06.802396    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-b7431c-5dbb549c45-hsk6h"]
Oct 30 20:17:06.808354 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:06.808320    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-b7431c-5dbb549c45-hsk6h"]
Oct 30 20:17:08.794627 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:08.794597    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="80f2d362-3093-42b0-a3c0-c8bf25c43a84" path="/var/lib/kubelet/pods/80f2d362-3093-42b0-a3c0-c8bf25c43a84/volumes"
Oct 30 20:17:11.909083 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:11.909043    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-0eeba5-75c6455594-8kbxt"]
Oct 30 20:17:15.087951 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:15.087912    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b24a1a-6f55488c68-pk5m5"]
Oct 30 20:17:15.089797 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:15.089762    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b24a1a-6f55488c68-pk5m5" podUID="33ca6985-3896-45c4-bb2f-b58fb5fb40b4" containerName="maas-api" containerID="cri-o://7d9978fccdad9b9b9e7c286ceaa4460a362d6d2527fd4c157ce647bf90575476" gracePeriod=30
Oct 30 20:17:15.369088 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:15.369061    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b24a1a-6f55488c68-pk5m5"
Oct 30 20:17:15.457833 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:15.457794    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/33ca6985-3896-45c4-bb2f-b58fb5fb40b4-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33ca6985-3896-45c4-bb2f-b58fb5fb40b4-maas-api-tls\") pod \"33ca6985-3896-45c4-bb2f-b58fb5fb40b4\" (UID: \"33ca6985-3896-45c4-bb2f-b58fb5fb40b4\") "
Oct 30 20:17:15.458031 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:15.457891    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/33ca6985-3896-45c4-bb2f-b58fb5fb40b4-kube-api-access-rxbhk\" (UniqueName: \"kubernetes.io/projected/33ca6985-3896-45c4-bb2f-b58fb5fb40b4-kube-api-access-rxbhk\") pod \"33ca6985-3896-45c4-bb2f-b58fb5fb40b4\" (UID: \"33ca6985-3896-45c4-bb2f-b58fb5fb40b4\") "
Oct 30 20:17:15.458031 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:15.457931    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/33ca6985-3896-45c4-bb2f-b58fb5fb40b4-metrics-certs\" (UniqueName: \"kubernetes.io/secret/33ca6985-3896-45c4-bb2f-b58fb5fb40b4-metrics-certs\") pod \"33ca6985-3896-45c4-bb2f-b58fb5fb40b4\" (UID: \"33ca6985-3896-45c4-bb2f-b58fb5fb40b4\") "
Oct 30 20:17:15.458175 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:15.458063    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/33ca6985-3896-45c4-bb2f-b58fb5fb40b4-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33ca6985-3896-45c4-bb2f-b58fb5fb40b4-openshift-service-ca\") pod \"33ca6985-3896-45c4-bb2f-b58fb5fb40b4\" (UID: \"33ca6985-3896-45c4-bb2f-b58fb5fb40b4\") "
Oct 30 20:17:15.458519 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:15.458453    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33ca6985-3896-45c4-bb2f-b58fb5fb40b4-openshift-service-ca" pod "33ca6985-3896-45c4-bb2f-b58fb5fb40b4" (UID: "33ca6985-3896-45c4-bb2f-b58fb5fb40b4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:17:15.458786 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:15.458766    6996 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33ca6985-3896-45c4-bb2f-b58fb5fb40b4-openshift-service-ca\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:17:15.460139 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:15.460106    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33ca6985-3896-45c4-bb2f-b58fb5fb40b4-kube-api-access-rxbhk" pod "33ca6985-3896-45c4-bb2f-b58fb5fb40b4" (UID: "33ca6985-3896-45c4-bb2f-b58fb5fb40b4"). InnerVolumeSpecName "kube-api-access-rxbhk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:17:15.460261 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:15.460184    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33ca6985-3896-45c4-bb2f-b58fb5fb40b4-maas-api-tls" pod "33ca6985-3896-45c4-bb2f-b58fb5fb40b4" (UID: "33ca6985-3896-45c4-bb2f-b58fb5fb40b4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:17:15.460440 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:15.460425    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33ca6985-3896-45c4-bb2f-b58fb5fb40b4-metrics-certs" pod "33ca6985-3896-45c4-bb2f-b58fb5fb40b4" (UID: "33ca6985-3896-45c4-bb2f-b58fb5fb40b4"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:17:15.578503 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:15.577638    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rxbhk\" (UniqueName: \"kubernetes.io/projected/33ca6985-3896-45c4-bb2f-b58fb5fb40b4-kube-api-access-rxbhk\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:17:15.578503 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:15.577688    6996 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/33ca6985-3896-45c4-bb2f-b58fb5fb40b4-metrics-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:17:15.578503 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:15.577706    6996 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33ca6985-3896-45c4-bb2f-b58fb5fb40b4-maas-api-tls\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:17:15.632283 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:15.632156    6996 generic.go:356] "Generic (PLEG): container finished" podID="33ca6985-3896-45c4-bb2f-b58fb5fb40b4" containerID="7d9978fccdad9b9b9e7c286ceaa4460a362d6d2527fd4c157ce647bf90575476" exitCode=0
Oct 30 20:17:15.632471 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:15.632295    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b24a1a-6f55488c68-pk5m5" event={"ID":"33ca6985-3896-45c4-bb2f-b58fb5fb40b4","Type":"ContainerDied","Data":"7d9978fccdad9b9b9e7c286ceaa4460a362d6d2527fd4c157ce647bf90575476"}
Oct 30 20:17:15.632471 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:15.632345    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b24a1a-6f55488c68-pk5m5" event={"ID":"33ca6985-3896-45c4-bb2f-b58fb5fb40b4","Type":"ContainerDied","Data":"aa75121ce3e2b73162e98326aaa5f65d02b97661b25a5f6ebac3dbb84ca0610b"}
Oct 30 20:17:15.632471 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:15.632373    6996 scope.go:122] "RemoveContainer" containerID="7d9978fccdad9b9b9e7c286ceaa4460a362d6d2527fd4c157ce647bf90575476"
Oct 30 20:17:15.632703 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:15.632539    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b24a1a-6f55488c68-pk5m5"
Oct 30 20:17:15.671165 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:15.671136    6996 scope.go:122] "RemoveContainer" containerID="7d9978fccdad9b9b9e7c286ceaa4460a362d6d2527fd4c157ce647bf90575476"
Oct 30 20:17:15.671554 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:17:15.671529    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d9978fccdad9b9b9e7c286ceaa4460a362d6d2527fd4c157ce647bf90575476\": container with ID starting with 7d9978fccdad9b9b9e7c286ceaa4460a362d6d2527fd4c157ce647bf90575476 not found: ID does not exist" containerID="7d9978fccdad9b9b9e7c286ceaa4460a362d6d2527fd4c157ce647bf90575476"
Oct 30 20:17:15.671650 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:15.671564    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d9978fccdad9b9b9e7c286ceaa4460a362d6d2527fd4c157ce647bf90575476"} err="failed to get container status \"7d9978fccdad9b9b9e7c286ceaa4460a362d6d2527fd4c157ce647bf90575476\": rpc error: code = NotFound desc = could not find container \"7d9978fccdad9b9b9e7c286ceaa4460a362d6d2527fd4c157ce647bf90575476\": container with ID starting with 7d9978fccdad9b9b9e7c286ceaa4460a362d6d2527fd4c157ce647bf90575476 not found: ID does not exist"
Oct 30 20:17:15.681124 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:15.681093    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b24a1a-6f55488c68-pk5m5"]
Oct 30 20:17:15.686438 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:15.686410    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b24a1a-6f55488c68-pk5m5"]
Oct 30 20:17:16.792416 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:16.792380    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="33ca6985-3896-45c4-bb2f-b58fb5fb40b4" path="/var/lib/kubelet/pods/33ca6985-3896-45c4-bb2f-b58fb5fb40b4/volumes"
Oct 30 20:17:16.909972 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:16.909908    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0eeba5-75c6455594-8kbxt" podUID="3bc8f341-bbf4-42d4-941a-0e636de353fe" containerName="payload-processing" containerID="cri-o://d6536549fd3e0ab44191c445d54cb36fdfc58da198afac152de207958d1da78d" gracePeriod=25
Oct 30 20:17:17.198706 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:17.198672    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0eeba5-75c6455594-8kbxt"
Oct 30 20:17:17.303253 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:17.303222    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3bc8f341-bbf4-42d4-941a-0e636de353fe-kube-api-access-7t8kc\" (UniqueName: \"kubernetes.io/projected/3bc8f341-bbf4-42d4-941a-0e636de353fe-kube-api-access-7t8kc\") pod \"3bc8f341-bbf4-42d4-941a-0e636de353fe\" (UID: \"3bc8f341-bbf4-42d4-941a-0e636de353fe\") "
Oct 30 20:17:17.303438 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:17.303425    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3bc8f341-bbf4-42d4-941a-0e636de353fe-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3bc8f341-bbf4-42d4-941a-0e636de353fe-plugins-config-volume\") pod \"3bc8f341-bbf4-42d4-941a-0e636de353fe\" (UID: \"3bc8f341-bbf4-42d4-941a-0e636de353fe\") "
Oct 30 20:17:17.303798 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:17.303772    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3bc8f341-bbf4-42d4-941a-0e636de353fe-plugins-config-volume" pod "3bc8f341-bbf4-42d4-941a-0e636de353fe" (UID: "3bc8f341-bbf4-42d4-941a-0e636de353fe"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:17:17.303959 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:17.303944    6996 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3bc8f341-bbf4-42d4-941a-0e636de353fe-plugins-config-volume\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:17:17.305425 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:17.305404    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3bc8f341-bbf4-42d4-941a-0e636de353fe-kube-api-access-7t8kc" pod "3bc8f341-bbf4-42d4-941a-0e636de353fe" (UID: "3bc8f341-bbf4-42d4-941a-0e636de353fe"). InnerVolumeSpecName "kube-api-access-7t8kc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:17:17.405263 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:17.405186    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7t8kc\" (UniqueName: \"kubernetes.io/projected/3bc8f341-bbf4-42d4-941a-0e636de353fe-kube-api-access-7t8kc\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:17:17.650253 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:17.650212    6996 generic.go:356] "Generic (PLEG): container finished" podID="3bc8f341-bbf4-42d4-941a-0e636de353fe" containerID="d6536549fd3e0ab44191c445d54cb36fdfc58da198afac152de207958d1da78d" exitCode=0
Oct 30 20:17:17.650445 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:17.650264    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0eeba5-75c6455594-8kbxt"
Oct 30 20:17:17.650445 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:17.650298    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0eeba5-75c6455594-8kbxt" event={"ID":"3bc8f341-bbf4-42d4-941a-0e636de353fe","Type":"ContainerDied","Data":"d6536549fd3e0ab44191c445d54cb36fdfc58da198afac152de207958d1da78d"}
Oct 30 20:17:17.650445 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:17.650358    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0eeba5-75c6455594-8kbxt" event={"ID":"3bc8f341-bbf4-42d4-941a-0e636de353fe","Type":"ContainerDied","Data":"8b285f0f30e2c27c4f08034772bb823bd7596499b2848a677bf7da8e021e6883"}
Oct 30 20:17:17.650445 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:17.650390    6996 scope.go:122] "RemoveContainer" containerID="d6536549fd3e0ab44191c445d54cb36fdfc58da198afac152de207958d1da78d"
Oct 30 20:17:17.661463 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:17.661436    6996 scope.go:122] "RemoveContainer" containerID="d6536549fd3e0ab44191c445d54cb36fdfc58da198afac152de207958d1da78d"
Oct 30 20:17:17.661758 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:17:17.661738    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6536549fd3e0ab44191c445d54cb36fdfc58da198afac152de207958d1da78d\": container with ID starting with d6536549fd3e0ab44191c445d54cb36fdfc58da198afac152de207958d1da78d not found: ID does not exist" containerID="d6536549fd3e0ab44191c445d54cb36fdfc58da198afac152de207958d1da78d"
Oct 30 20:17:17.661847 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:17.661765    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6536549fd3e0ab44191c445d54cb36fdfc58da198afac152de207958d1da78d"} err="failed to get container status \"d6536549fd3e0ab44191c445d54cb36fdfc58da198afac152de207958d1da78d\": rpc error: code = NotFound desc = could not find container \"d6536549fd3e0ab44191c445d54cb36fdfc58da198afac152de207958d1da78d\": container with ID starting with d6536549fd3e0ab44191c445d54cb36fdfc58da198afac152de207958d1da78d not found: ID does not exist"
Oct 30 20:17:17.675634 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:17.675612    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-0eeba5-75c6455594-8kbxt"]
Oct 30 20:17:17.680570 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:17.680547    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-0eeba5-75c6455594-8kbxt"]
Oct 30 20:17:18.792287 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:18.792254    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3bc8f341-bbf4-42d4-941a-0e636de353fe" path="/var/lib/kubelet/pods/3bc8f341-bbf4-42d4-941a-0e636de353fe/volumes"
Oct 30 20:17:22.378905 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:22.378860    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 20:17:31.252201 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:17:31.252159    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 20:17:33.769623 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:33.769535    6996 generic.go:356] "Generic (PLEG): container finished" podID="a21bbe95-547d-41b0-b7c8-e1a296545947" containerID="a482c0cafd0058dd322aed5b2a574316076d71cae879579d39f173e60ab077ef" exitCode=0
Oct 30 20:17:33.770181 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:33.769612    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp" event={"ID":"a21bbe95-547d-41b0-b7c8-e1a296545947","Type":"ContainerDied","Data":"a482c0cafd0058dd322aed5b2a574316076d71cae879579d39f173e60ab077ef"}
Oct 30 20:17:33.770181 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:33.770111    6996 scope.go:122] "RemoveContainer" containerID="a482c0cafd0058dd322aed5b2a574316076d71cae879579d39f173e60ab077ef"
Oct 30 20:17:33.771877 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:33.771847    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 20:17:33.771962 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:33.771920    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 20:17:33.771962 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:33.771953    6996 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 20:17:34.778620 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:34.778449    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp" event={"ID":"a21bbe95-547d-41b0-b7c8-e1a296545947","Type":"ContainerStarted","Data":"7d73432516dfe591caae8bf776dadb8a4926e3847b29ae549e7e6f47989e1edd"}
Oct 30 20:17:35.097614 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:35.097580    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-7cbcbb-6b88874567-48rg5"]
Oct 30 20:17:35.097824 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:35.097789    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-7cbcbb-6b88874567-48rg5" podUID="8d38b232-9550-4272-8174-7c457b1f4b38" containerName="maas-api" containerID="cri-o://604a1503ebd138f50c5602c813c6dc05ea9b2ac0ed643cb881e30629e911c968" gracePeriod=30
Oct 30 20:17:35.307014 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:35.306920    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp"
Oct 30 20:17:35.317176 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:35.308925    6996 patch_prober.go:29] interesting pod/e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.169:15021/healthz/ready\": dial tcp 10.134.0.169:15021: connect: connection refused" start-of-body=
Oct 30 20:17:35.317176 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:35.309003    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp" podUID="a21bbe95-547d-41b0-b7c8-e1a296545947" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.169:15021/healthz/ready\": dial tcp 10.134.0.169:15021: connect: connection refused"
Oct 30 20:17:35.619266 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:35.619238    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-7cbcbb-6b88874567-48rg5"
Oct 30 20:17:35.709590 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:35.706291    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp"
Oct 30 20:17:35.761789 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:35.761751    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8d38b232-9550-4272-8174-7c457b1f4b38-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d38b232-9550-4272-8174-7c457b1f4b38-openshift-service-ca\") pod \"8d38b232-9550-4272-8174-7c457b1f4b38\" (UID: \"8d38b232-9550-4272-8174-7c457b1f4b38\") "
Oct 30 20:17:35.762108 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:35.762083    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8d38b232-9550-4272-8174-7c457b1f4b38-metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d38b232-9550-4272-8174-7c457b1f4b38-metrics-certs\") pod \"8d38b232-9550-4272-8174-7c457b1f4b38\" (UID: \"8d38b232-9550-4272-8174-7c457b1f4b38\") "
Oct 30 20:17:35.762327 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:35.762309    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8d38b232-9550-4272-8174-7c457b1f4b38-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8d38b232-9550-4272-8174-7c457b1f4b38-maas-api-tls\") pod \"8d38b232-9550-4272-8174-7c457b1f4b38\" (UID: \"8d38b232-9550-4272-8174-7c457b1f4b38\") "
Oct 30 20:17:35.762566 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:35.762545    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8d38b232-9550-4272-8174-7c457b1f4b38-kube-api-access-c9mzn\" (UniqueName: \"kubernetes.io/projected/8d38b232-9550-4272-8174-7c457b1f4b38-kube-api-access-c9mzn\") pod \"8d38b232-9550-4272-8174-7c457b1f4b38\" (UID: \"8d38b232-9550-4272-8174-7c457b1f4b38\") "
Oct 30 20:17:35.773833 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:35.772161    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d38b232-9550-4272-8174-7c457b1f4b38-openshift-service-ca" pod "8d38b232-9550-4272-8174-7c457b1f4b38" (UID: "8d38b232-9550-4272-8174-7c457b1f4b38"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:17:35.774330 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:35.774288    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d38b232-9550-4272-8174-7c457b1f4b38-metrics-certs" pod "8d38b232-9550-4272-8174-7c457b1f4b38" (UID: "8d38b232-9550-4272-8174-7c457b1f4b38"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:17:35.776370 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:35.776330    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d38b232-9550-4272-8174-7c457b1f4b38-kube-api-access-c9mzn" pod "8d38b232-9550-4272-8174-7c457b1f4b38" (UID: "8d38b232-9550-4272-8174-7c457b1f4b38"). InnerVolumeSpecName "kube-api-access-c9mzn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:17:35.783427 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:35.783010    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d38b232-9550-4272-8174-7c457b1f4b38-maas-api-tls" pod "8d38b232-9550-4272-8174-7c457b1f4b38" (UID: "8d38b232-9550-4272-8174-7c457b1f4b38"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:17:35.803532 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:35.793191    6996 generic.go:356] "Generic (PLEG): container finished" podID="8d38b232-9550-4272-8174-7c457b1f4b38" containerID="604a1503ebd138f50c5602c813c6dc05ea9b2ac0ed643cb881e30629e911c968" exitCode=0
Oct 30 20:17:35.803532 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:35.793260    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-7cbcbb-6b88874567-48rg5"
Oct 30 20:17:35.803532 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:35.793735    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-7cbcbb-6b88874567-48rg5" event={"ID":"8d38b232-9550-4272-8174-7c457b1f4b38","Type":"ContainerDied","Data":"604a1503ebd138f50c5602c813c6dc05ea9b2ac0ed643cb881e30629e911c968"}
Oct 30 20:17:35.803532 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:35.793792    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-7cbcbb-6b88874567-48rg5" event={"ID":"8d38b232-9550-4272-8174-7c457b1f4b38","Type":"ContainerDied","Data":"0e7f30b10952560e49963a86d87be704de9e0ad292b41c06e4c9660be5c05473"}
Oct 30 20:17:35.803532 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:35.793822    6996 scope.go:122] "RemoveContainer" containerID="604a1503ebd138f50c5602c813c6dc05ea9b2ac0ed643cb881e30629e911c968"
Oct 30 20:17:35.813154 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:35.813131    6996 scope.go:122] "RemoveContainer" containerID="604a1503ebd138f50c5602c813c6dc05ea9b2ac0ed643cb881e30629e911c968"
Oct 30 20:17:35.813515 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:17:35.813461    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"604a1503ebd138f50c5602c813c6dc05ea9b2ac0ed643cb881e30629e911c968\": container with ID starting with 604a1503ebd138f50c5602c813c6dc05ea9b2ac0ed643cb881e30629e911c968 not found: ID does not exist" containerID="604a1503ebd138f50c5602c813c6dc05ea9b2ac0ed643cb881e30629e911c968"
Oct 30 20:17:35.813515 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:35.813506    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"604a1503ebd138f50c5602c813c6dc05ea9b2ac0ed643cb881e30629e911c968"} err="failed to get container status \"604a1503ebd138f50c5602c813c6dc05ea9b2ac0ed643cb881e30629e911c968\": rpc error: code = NotFound desc = could not find container \"604a1503ebd138f50c5602c813c6dc05ea9b2ac0ed643cb881e30629e911c968\": container with ID starting with 604a1503ebd138f50c5602c813c6dc05ea9b2ac0ed643cb881e30629e911c968 not found: ID does not exist"
Oct 30 20:17:35.823666 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:35.823640    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-7cbcbb-6b88874567-48rg5"]
Oct 30 20:17:35.831381 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:35.831358    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-7cbcbb-6b88874567-48rg5"]
Oct 30 20:17:35.875786 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:35.867885    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c9mzn\" (UniqueName: \"kubernetes.io/projected/8d38b232-9550-4272-8174-7c457b1f4b38-kube-api-access-c9mzn\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:17:35.875786 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:35.867918    6996 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d38b232-9550-4272-8174-7c457b1f4b38-openshift-service-ca\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:17:35.875786 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:35.867938    6996 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d38b232-9550-4272-8174-7c457b1f4b38-metrics-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:17:35.875786 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:35.867966    6996 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8d38b232-9550-4272-8174-7c457b1f4b38-maas-api-tls\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:17:36.326254 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:36.321640    6996 patch_prober.go:29] interesting pod/e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.169:15021/healthz/ready\": dial tcp 10.134.0.169:15021: connect: connection refused" start-of-body=
Oct 30 20:17:36.326254 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:36.321710    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp" podUID="a21bbe95-547d-41b0-b7c8-e1a296545947" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.169:15021/healthz/ready\": dial tcp 10.134.0.169:15021: connect: connection refused"
Oct 30 20:17:36.822193 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:36.822158    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8d38b232-9550-4272-8174-7c457b1f4b38" path="/var/lib/kubelet/pods/8d38b232-9550-4272-8174-7c457b1f4b38/volumes"
Oct 30 20:17:37.306884 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:37.306792    6996 patch_prober.go:29] interesting pod/e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.169:15021/healthz/ready\": dial tcp 10.134.0.169:15021: connect: connection refused" start-of-body=
Oct 30 20:17:37.306884 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:37.306866    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp" podUID="a21bbe95-547d-41b0-b7c8-e1a296545947" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.169:15021/healthz/ready\": dial tcp 10.134.0.169:15021: connect: connection refused"
Oct 30 20:17:38.306342 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:38.306305    6996 patch_prober.go:29] interesting pod/e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.169:15021/healthz/ready\": dial tcp 10.134.0.169:15021: connect: connection refused" start-of-body=
Oct 30 20:17:38.306774 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:38.306374    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp" podUID="a21bbe95-547d-41b0-b7c8-e1a296545947" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.169:15021/healthz/ready\": dial tcp 10.134.0.169:15021: connect: connection refused"
Oct 30 20:17:39.306724 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:39.306689    6996 patch_prober.go:29] interesting pod/e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.169:15021/healthz/ready\": dial tcp 10.134.0.169:15021: connect: connection refused" start-of-body=
Oct 30 20:17:39.307227 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:39.306750    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp" podUID="a21bbe95-547d-41b0-b7c8-e1a296545947" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.169:15021/healthz/ready\": dial tcp 10.134.0.169:15021: connect: connection refused"
Oct 30 20:17:40.307171 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:40.307129    6996 patch_prober.go:29] interesting pod/e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.169:15021/healthz/ready\": dial tcp 10.134.0.169:15021: connect: connection refused" start-of-body=
Oct 30 20:17:40.307590 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:40.307210    6996 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp" podUID="a21bbe95-547d-41b0-b7c8-e1a296545947" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.169:15021/healthz/ready\": dial tcp 10.134.0.169:15021: connect: connection refused"
Oct 30 20:17:41.314039 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:41.314002    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp"
Oct 30 20:17:41.315011 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:41.314991    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp"
Oct 30 20:17:45.099465 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:45.099424    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-54d4c4-58c99d6b5d-v5j9h"]
Oct 30 20:17:45.100029 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:45.099784    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-54d4c4-58c99d6b5d-v5j9h" podUID="4b366fcc-1688-4082-bb67-b2f05afdd019" containerName="maas-api" containerID="cri-o://ba0ab181ce16ddbc61e5cd758944ac95a5875f97f76034b94d044c3425e5064d" gracePeriod=30
Oct 30 20:17:45.482053 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:45.482024    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-54d4c4-58c99d6b5d-v5j9h"
Oct 30 20:17:45.594098 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:45.594065    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4b366fcc-1688-4082-bb67-b2f05afdd019-metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b366fcc-1688-4082-bb67-b2f05afdd019-metrics-certs\") pod \"4b366fcc-1688-4082-bb67-b2f05afdd019\" (UID: \"4b366fcc-1688-4082-bb67-b2f05afdd019\") "
Oct 30 20:17:45.594281 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:45.594200    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4b366fcc-1688-4082-bb67-b2f05afdd019-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b366fcc-1688-4082-bb67-b2f05afdd019-openshift-service-ca\") pod \"4b366fcc-1688-4082-bb67-b2f05afdd019\" (UID: \"4b366fcc-1688-4082-bb67-b2f05afdd019\") "
Oct 30 20:17:45.594281 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:45.594239    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4b366fcc-1688-4082-bb67-b2f05afdd019-kube-api-access-xjpvg\" (UniqueName: \"kubernetes.io/projected/4b366fcc-1688-4082-bb67-b2f05afdd019-kube-api-access-xjpvg\") pod \"4b366fcc-1688-4082-bb67-b2f05afdd019\" (UID: \"4b366fcc-1688-4082-bb67-b2f05afdd019\") "
Oct 30 20:17:45.594378 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:45.594319    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4b366fcc-1688-4082-bb67-b2f05afdd019-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b366fcc-1688-4082-bb67-b2f05afdd019-maas-api-tls\") pod \"4b366fcc-1688-4082-bb67-b2f05afdd019\" (UID: \"4b366fcc-1688-4082-bb67-b2f05afdd019\") "
Oct 30 20:17:45.594636 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:45.594608    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b366fcc-1688-4082-bb67-b2f05afdd019-openshift-service-ca" pod "4b366fcc-1688-4082-bb67-b2f05afdd019" (UID: "4b366fcc-1688-4082-bb67-b2f05afdd019"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:17:45.594867 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:45.594847    6996 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b366fcc-1688-4082-bb67-b2f05afdd019-openshift-service-ca\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:17:45.596295 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:45.596266    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b366fcc-1688-4082-bb67-b2f05afdd019-metrics-certs" pod "4b366fcc-1688-4082-bb67-b2f05afdd019" (UID: "4b366fcc-1688-4082-bb67-b2f05afdd019"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:17:45.596431 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:45.596417    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b366fcc-1688-4082-bb67-b2f05afdd019-maas-api-tls" pod "4b366fcc-1688-4082-bb67-b2f05afdd019" (UID: "4b366fcc-1688-4082-bb67-b2f05afdd019"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:17:45.596631 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:45.596610    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b366fcc-1688-4082-bb67-b2f05afdd019-kube-api-access-xjpvg" pod "4b366fcc-1688-4082-bb67-b2f05afdd019" (UID: "4b366fcc-1688-4082-bb67-b2f05afdd019"). InnerVolumeSpecName "kube-api-access-xjpvg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:17:45.702965 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:45.700592    6996 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b366fcc-1688-4082-bb67-b2f05afdd019-metrics-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:17:45.702965 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:45.700645    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xjpvg\" (UniqueName: \"kubernetes.io/projected/4b366fcc-1688-4082-bb67-b2f05afdd019-kube-api-access-xjpvg\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:17:45.702965 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:45.700660    6996 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b366fcc-1688-4082-bb67-b2f05afdd019-maas-api-tls\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:17:45.922349 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:45.922309    6996 generic.go:356] "Generic (PLEG): container finished" podID="4b366fcc-1688-4082-bb67-b2f05afdd019" containerID="ba0ab181ce16ddbc61e5cd758944ac95a5875f97f76034b94d044c3425e5064d" exitCode=0
Oct 30 20:17:45.922585 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:45.922367    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-54d4c4-58c99d6b5d-v5j9h"
Oct 30 20:17:45.922585 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:45.922371    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-54d4c4-58c99d6b5d-v5j9h" event={"ID":"4b366fcc-1688-4082-bb67-b2f05afdd019","Type":"ContainerDied","Data":"ba0ab181ce16ddbc61e5cd758944ac95a5875f97f76034b94d044c3425e5064d"}
Oct 30 20:17:45.922585 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:45.922554    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-54d4c4-58c99d6b5d-v5j9h" event={"ID":"4b366fcc-1688-4082-bb67-b2f05afdd019","Type":"ContainerDied","Data":"3174097a29ddae621bc8c89748445544d49f0a944cc945e1523f1c3365a3b3c2"}
Oct 30 20:17:45.922774 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:45.922591    6996 scope.go:122] "RemoveContainer" containerID="ba0ab181ce16ddbc61e5cd758944ac95a5875f97f76034b94d044c3425e5064d"
Oct 30 20:17:45.933469 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:45.933449    6996 scope.go:122] "RemoveContainer" containerID="ba0ab181ce16ddbc61e5cd758944ac95a5875f97f76034b94d044c3425e5064d"
Oct 30 20:17:45.933762 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:17:45.933741    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba0ab181ce16ddbc61e5cd758944ac95a5875f97f76034b94d044c3425e5064d\": container with ID starting with ba0ab181ce16ddbc61e5cd758944ac95a5875f97f76034b94d044c3425e5064d not found: ID does not exist" containerID="ba0ab181ce16ddbc61e5cd758944ac95a5875f97f76034b94d044c3425e5064d"
Oct 30 20:17:45.933830 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:45.933776    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba0ab181ce16ddbc61e5cd758944ac95a5875f97f76034b94d044c3425e5064d"} err="failed to get container status \"ba0ab181ce16ddbc61e5cd758944ac95a5875f97f76034b94d044c3425e5064d\": rpc error: code = NotFound desc = could not find container \"ba0ab181ce16ddbc61e5cd758944ac95a5875f97f76034b94d044c3425e5064d\": container with ID starting with ba0ab181ce16ddbc61e5cd758944ac95a5875f97f76034b94d044c3425e5064d not found: ID does not exist"
Oct 30 20:17:45.946335 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:45.946310    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-54d4c4-58c99d6b5d-v5j9h"]
Oct 30 20:17:45.958063 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:45.958013    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-54d4c4-58c99d6b5d-v5j9h"]
Oct 30 20:17:46.793409 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:46.793357    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4b366fcc-1688-4082-bb67-b2f05afdd019" path="/var/lib/kubelet/pods/4b366fcc-1688-4082-bb67-b2f05afdd019/volumes"
Oct 30 20:17:47.583628 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:47.583276    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-779fa3-56465c7cdd-wvqr2"]
Oct 30 20:17:47.583858 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:47.583632    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-779fa3-56465c7cdd-wvqr2" podUID="3bdb062c-f6d3-4d06-9c69-b9d0b8e11726" containerName="maas-api" containerID="cri-o://93a1b24e2ea3ec2537f1b826706cd6cbcf00856ecdfefec2c01ac71fa6054358" gracePeriod=30
Oct 30 20:17:47.881636 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:47.881609    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-779fa3-56465c7cdd-wvqr2"
Oct 30 20:17:47.967503 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:47.959445    6996 generic.go:356] "Generic (PLEG): container finished" podID="3bdb062c-f6d3-4d06-9c69-b9d0b8e11726" containerID="93a1b24e2ea3ec2537f1b826706cd6cbcf00856ecdfefec2c01ac71fa6054358" exitCode=0
Oct 30 20:17:47.967503 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:47.959555    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-779fa3-56465c7cdd-wvqr2" event={"ID":"3bdb062c-f6d3-4d06-9c69-b9d0b8e11726","Type":"ContainerDied","Data":"93a1b24e2ea3ec2537f1b826706cd6cbcf00856ecdfefec2c01ac71fa6054358"}
Oct 30 20:17:47.967503 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:47.959889    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-779fa3-56465c7cdd-wvqr2" event={"ID":"3bdb062c-f6d3-4d06-9c69-b9d0b8e11726","Type":"ContainerDied","Data":"cd6501746bf67cdd78c9feccc071ec4ce76d3185ba4b424398bfaafb29f609ed"}
Oct 30 20:17:47.967503 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:47.960001    6996 scope.go:122] "RemoveContainer" containerID="93a1b24e2ea3ec2537f1b826706cd6cbcf00856ecdfefec2c01ac71fa6054358"
Oct 30 20:17:47.967503 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:47.960169    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-779fa3-56465c7cdd-wvqr2"
Oct 30 20:17:47.967503 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:47.961635    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3bdb062c-f6d3-4d06-9c69-b9d0b8e11726-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3bdb062c-f6d3-4d06-9c69-b9d0b8e11726-openshift-service-ca\") pod \"3bdb062c-f6d3-4d06-9c69-b9d0b8e11726\" (UID: \"3bdb062c-f6d3-4d06-9c69-b9d0b8e11726\") "
Oct 30 20:17:47.967503 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:47.961732    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3bdb062c-f6d3-4d06-9c69-b9d0b8e11726-kube-api-access-nrs67\" (UniqueName: \"kubernetes.io/projected/3bdb062c-f6d3-4d06-9c69-b9d0b8e11726-kube-api-access-nrs67\") pod \"3bdb062c-f6d3-4d06-9c69-b9d0b8e11726\" (UID: \"3bdb062c-f6d3-4d06-9c69-b9d0b8e11726\") "
Oct 30 20:17:47.967503 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:47.961843    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3bdb062c-f6d3-4d06-9c69-b9d0b8e11726-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3bdb062c-f6d3-4d06-9c69-b9d0b8e11726-maas-api-tls\") pod \"3bdb062c-f6d3-4d06-9c69-b9d0b8e11726\" (UID: \"3bdb062c-f6d3-4d06-9c69-b9d0b8e11726\") "
Oct 30 20:17:47.967503 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:47.961935    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3bdb062c-f6d3-4d06-9c69-b9d0b8e11726-metrics-certs\" (UniqueName: \"kubernetes.io/secret/3bdb062c-f6d3-4d06-9c69-b9d0b8e11726-metrics-certs\") pod \"3bdb062c-f6d3-4d06-9c69-b9d0b8e11726\" (UID: \"3bdb062c-f6d3-4d06-9c69-b9d0b8e11726\") "
Oct 30 20:17:47.967503 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:47.963543    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3bdb062c-f6d3-4d06-9c69-b9d0b8e11726-openshift-service-ca" pod "3bdb062c-f6d3-4d06-9c69-b9d0b8e11726" (UID: "3bdb062c-f6d3-4d06-9c69-b9d0b8e11726"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:17:48.017705 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:48.014724    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3bdb062c-f6d3-4d06-9c69-b9d0b8e11726-metrics-certs" pod "3bdb062c-f6d3-4d06-9c69-b9d0b8e11726" (UID: "3bdb062c-f6d3-4d06-9c69-b9d0b8e11726"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:17:48.036712 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:48.036669    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3bdb062c-f6d3-4d06-9c69-b9d0b8e11726-kube-api-access-nrs67" pod "3bdb062c-f6d3-4d06-9c69-b9d0b8e11726" (UID: "3bdb062c-f6d3-4d06-9c69-b9d0b8e11726"). InnerVolumeSpecName "kube-api-access-nrs67". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:17:48.038941 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:48.038919    6996 scope.go:122] "RemoveContainer" containerID="93a1b24e2ea3ec2537f1b826706cd6cbcf00856ecdfefec2c01ac71fa6054358"
Oct 30 20:17:48.039474 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:17:48.039438    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93a1b24e2ea3ec2537f1b826706cd6cbcf00856ecdfefec2c01ac71fa6054358\": container with ID starting with 93a1b24e2ea3ec2537f1b826706cd6cbcf00856ecdfefec2c01ac71fa6054358 not found: ID does not exist" containerID="93a1b24e2ea3ec2537f1b826706cd6cbcf00856ecdfefec2c01ac71fa6054358"
Oct 30 20:17:48.039580 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:48.039503    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93a1b24e2ea3ec2537f1b826706cd6cbcf00856ecdfefec2c01ac71fa6054358"} err="failed to get container status \"93a1b24e2ea3ec2537f1b826706cd6cbcf00856ecdfefec2c01ac71fa6054358\": rpc error: code = NotFound desc = could not find container \"93a1b24e2ea3ec2537f1b826706cd6cbcf00856ecdfefec2c01ac71fa6054358\": container with ID starting with 93a1b24e2ea3ec2537f1b826706cd6cbcf00856ecdfefec2c01ac71fa6054358 not found: ID does not exist"
Oct 30 20:17:48.044158 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:48.044124    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3bdb062c-f6d3-4d06-9c69-b9d0b8e11726-maas-api-tls" pod "3bdb062c-f6d3-4d06-9c69-b9d0b8e11726" (UID: "3bdb062c-f6d3-4d06-9c69-b9d0b8e11726"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:17:48.080038 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:48.078728    6996 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3bdb062c-f6d3-4d06-9c69-b9d0b8e11726-openshift-service-ca\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:17:48.080038 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:48.078768    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nrs67\" (UniqueName: \"kubernetes.io/projected/3bdb062c-f6d3-4d06-9c69-b9d0b8e11726-kube-api-access-nrs67\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:17:48.080038 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:48.078787    6996 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3bdb062c-f6d3-4d06-9c69-b9d0b8e11726-maas-api-tls\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:17:48.080038 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:48.078801    6996 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3bdb062c-f6d3-4d06-9c69-b9d0b8e11726-metrics-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:17:48.283809 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:48.283773    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-779fa3-56465c7cdd-wvqr2"]
Oct 30 20:17:48.291208 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:48.291176    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-779fa3-56465c7cdd-wvqr2"]
Oct 30 20:17:48.792754 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:48.792719    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3bdb062c-f6d3-4d06-9c69-b9d0b8e11726" path="/var/lib/kubelet/pods/3bdb062c-f6d3-4d06-9c69-b9d0b8e11726/volumes"
Oct 30 20:17:49.583140 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:49.583104    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-2a3e1b/distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk"]
Oct 30 20:17:49.590768 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:49.590738    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-2a3e1b/distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq"]
Oct 30 20:17:49.601584 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:49.601549    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr"]
Oct 30 20:17:57.395049 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:17:57.395009    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 20:18:01.192364 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:18:01.192317    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 20:18:02.009895 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:02.009852    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b"]
Oct 30 20:18:04.474453 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:04.474413    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-f06af6-77784df88-qg6p6"]
Oct 30 20:18:04.605894 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:04.605848    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk" podUID="a1857d2d-3b40-4e63-be33-be4ba476f926" containerName="main" containerID="cri-o://333b252c6267e56044991c7b47299163c325d1b3ea1e70604a319e9615a1ce52" gracePeriod=45
Oct 30 20:18:04.612319 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:04.612284    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq" podUID="89009638-003d-4a5d-ab13-1f8c8f64d3b7" containerName="main" containerID="cri-o://cd497be8ff51e29fca5d267706805e3208fb8324980217626924a5fed9a7bd35" gracePeriod=45
Oct 30 20:18:04.623111 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:04.623078    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr" podUID="0e470010-625d-443f-81ec-9bf965988212" containerName="main" containerID="cri-o://1ea96199554c4db1446c246d6c52c0fb1b5334aeb9ab8ad5d1d30748d398d204" gracePeriod=45
Oct 30 20:18:04.900287 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:04.897325    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr" podUID="0e470010-625d-443f-81ec-9bf965988212" containerName="main" probeResult="failure" output="Get \"https://10.134.0.72:8000/ready\": dial tcp 10.134.0.72:8000: connect: connection refused"
Oct 30 20:18:04.902821 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:04.902267    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk" podUID="a1857d2d-3b40-4e63-be33-be4ba476f926" containerName="main" probeResult="failure" output="Get \"https://10.134.0.90:8000/ready\": dial tcp 10.134.0.90:8000: connect: connection refused"
Oct 30 20:18:05.034615 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.034590    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq"
Oct 30 20:18:05.063284 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.056833    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr"
Oct 30 20:18:05.086649 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.084499    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk"
Oct 30 20:18:05.155983 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.155929    6996 generic.go:356] "Generic (PLEG): container finished" podID="89009638-003d-4a5d-ab13-1f8c8f64d3b7" containerID="cd497be8ff51e29fca5d267706805e3208fb8324980217626924a5fed9a7bd35" exitCode=0
Oct 30 20:18:05.156278 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.156056    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq"
Oct 30 20:18:05.158501 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.156075    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq" event={"ID":"89009638-003d-4a5d-ab13-1f8c8f64d3b7","Type":"ContainerDied","Data":"cd497be8ff51e29fca5d267706805e3208fb8324980217626924a5fed9a7bd35"}
Oct 30 20:18:05.158501 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.156667    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq" event={"ID":"89009638-003d-4a5d-ab13-1f8c8f64d3b7","Type":"ContainerDied","Data":"e74597a4c9e6ac335f1115fb4600bee2261c72e63c30becba614802a9daa3fc6"}
Oct 30 20:18:05.158501 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.156696    6996 scope.go:122] "RemoveContainer" containerID="cd497be8ff51e29fca5d267706805e3208fb8324980217626924a5fed9a7bd35"
Oct 30 20:18:05.159786 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.159449    6996 generic.go:356] "Generic (PLEG): container finished" podID="0e470010-625d-443f-81ec-9bf965988212" containerID="1ea96199554c4db1446c246d6c52c0fb1b5334aeb9ab8ad5d1d30748d398d204" exitCode=0
Oct 30 20:18:05.159786 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.159600    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr" event={"ID":"0e470010-625d-443f-81ec-9bf965988212","Type":"ContainerDied","Data":"1ea96199554c4db1446c246d6c52c0fb1b5334aeb9ab8ad5d1d30748d398d204"}
Oct 30 20:18:05.159786 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.159633    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr" event={"ID":"0e470010-625d-443f-81ec-9bf965988212","Type":"ContainerDied","Data":"01ea573abfd479ce3f1ed75336667b85ca0b8841885f82e63cacbe3688045ffc"}
Oct 30 20:18:05.159786 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.159705    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr"
Oct 30 20:18:05.169499 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.162285    6996 generic.go:356] "Generic (PLEG): container finished" podID="a1857d2d-3b40-4e63-be33-be4ba476f926" containerID="333b252c6267e56044991c7b47299163c325d1b3ea1e70604a319e9615a1ce52" exitCode=0
Oct 30 20:18:05.169499 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.162364    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk" event={"ID":"a1857d2d-3b40-4e63-be33-be4ba476f926","Type":"ContainerDied","Data":"333b252c6267e56044991c7b47299163c325d1b3ea1e70604a319e9615a1ce52"}
Oct 30 20:18:05.169499 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.162398    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk" event={"ID":"a1857d2d-3b40-4e63-be33-be4ba476f926","Type":"ContainerDied","Data":"92aae822cebf644c7224bef16724d9985654afb4dd677b98965c5170e5ef20a1"}
Oct 30 20:18:05.169499 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.162525    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk"
Oct 30 20:18:05.186348 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.186317    6996 scope.go:122] "RemoveContainer" containerID="cd497be8ff51e29fca5d267706805e3208fb8324980217626924a5fed9a7bd35"
Oct 30 20:18:05.186842 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:18:05.186810    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd497be8ff51e29fca5d267706805e3208fb8324980217626924a5fed9a7bd35\": container with ID starting with cd497be8ff51e29fca5d267706805e3208fb8324980217626924a5fed9a7bd35 not found: ID does not exist" containerID="cd497be8ff51e29fca5d267706805e3208fb8324980217626924a5fed9a7bd35"
Oct 30 20:18:05.186978 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.186854    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd497be8ff51e29fca5d267706805e3208fb8324980217626924a5fed9a7bd35"} err="failed to get container status \"cd497be8ff51e29fca5d267706805e3208fb8324980217626924a5fed9a7bd35\": rpc error: code = NotFound desc = could not find container \"cd497be8ff51e29fca5d267706805e3208fb8324980217626924a5fed9a7bd35\": container with ID starting with cd497be8ff51e29fca5d267706805e3208fb8324980217626924a5fed9a7bd35 not found: ID does not exist"
Oct 30 20:18:05.186978 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.186888    6996 scope.go:122] "RemoveContainer" containerID="1ea96199554c4db1446c246d6c52c0fb1b5334aeb9ab8ad5d1d30748d398d204"
Oct 30 20:18:05.198842 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.198822    6996 scope.go:122] "RemoveContainer" containerID="1ea96199554c4db1446c246d6c52c0fb1b5334aeb9ab8ad5d1d30748d398d204"
Oct 30 20:18:05.199163 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:18:05.199139    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ea96199554c4db1446c246d6c52c0fb1b5334aeb9ab8ad5d1d30748d398d204\": container with ID starting with 1ea96199554c4db1446c246d6c52c0fb1b5334aeb9ab8ad5d1d30748d398d204 not found: ID does not exist" containerID="1ea96199554c4db1446c246d6c52c0fb1b5334aeb9ab8ad5d1d30748d398d204"
Oct 30 20:18:05.199268 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.199171    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ea96199554c4db1446c246d6c52c0fb1b5334aeb9ab8ad5d1d30748d398d204"} err="failed to get container status \"1ea96199554c4db1446c246d6c52c0fb1b5334aeb9ab8ad5d1d30748d398d204\": rpc error: code = NotFound desc = could not find container \"1ea96199554c4db1446c246d6c52c0fb1b5334aeb9ab8ad5d1d30748d398d204\": container with ID starting with 1ea96199554c4db1446c246d6c52c0fb1b5334aeb9ab8ad5d1d30748d398d204 not found: ID does not exist"
Oct 30 20:18:05.199268 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.199194    6996 scope.go:122] "RemoveContainer" containerID="333b252c6267e56044991c7b47299163c325d1b3ea1e70604a319e9615a1ce52"
Oct 30 20:18:05.208824 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.208796    6996 scope.go:122] "RemoveContainer" containerID="333b252c6267e56044991c7b47299163c325d1b3ea1e70604a319e9615a1ce52"
Oct 30 20:18:05.209144 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:18:05.209122    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"333b252c6267e56044991c7b47299163c325d1b3ea1e70604a319e9615a1ce52\": container with ID starting with 333b252c6267e56044991c7b47299163c325d1b3ea1e70604a319e9615a1ce52 not found: ID does not exist" containerID="333b252c6267e56044991c7b47299163c325d1b3ea1e70604a319e9615a1ce52"
Oct 30 20:18:05.209249 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.209150    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"333b252c6267e56044991c7b47299163c325d1b3ea1e70604a319e9615a1ce52"} err="failed to get container status \"333b252c6267e56044991c7b47299163c325d1b3ea1e70604a319e9615a1ce52\": rpc error: code = NotFound desc = could not find container \"333b252c6267e56044991c7b47299163c325d1b3ea1e70604a319e9615a1ce52\": container with ID starting with 333b252c6267e56044991c7b47299163c325d1b3ea1e70604a319e9615a1ce52 not found: ID does not exist"
Oct 30 20:18:05.236227 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.236192    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/89009638-003d-4a5d-ab13-1f8c8f64d3b7-dshm\" (UniqueName: \"kubernetes.io/empty-dir/89009638-003d-4a5d-ab13-1f8c8f64d3b7-dshm\") pod \"89009638-003d-4a5d-ab13-1f8c8f64d3b7\" (UID: \"89009638-003d-4a5d-ab13-1f8c8f64d3b7\") "
Oct 30 20:18:05.236387 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.236255    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0e470010-625d-443f-81ec-9bf965988212-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0e470010-625d-443f-81ec-9bf965988212-model-cache\") pod \"0e470010-625d-443f-81ec-9bf965988212\" (UID: \"0e470010-625d-443f-81ec-9bf965988212\") "
Oct 30 20:18:05.236387 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.236311    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a1857d2d-3b40-4e63-be33-be4ba476f926-dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1857d2d-3b40-4e63-be33-be4ba476f926-dshm\") pod \"a1857d2d-3b40-4e63-be33-be4ba476f926\" (UID: \"a1857d2d-3b40-4e63-be33-be4ba476f926\") "
Oct 30 20:18:05.236387 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.236339    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0e470010-625d-443f-81ec-9bf965988212-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0e470010-625d-443f-81ec-9bf965988212-tmp-dir\") pod \"0e470010-625d-443f-81ec-9bf965988212\" (UID: \"0e470010-625d-443f-81ec-9bf965988212\") "
Oct 30 20:18:05.236387 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.236379    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0e470010-625d-443f-81ec-9bf965988212-dshm\" (UniqueName: \"kubernetes.io/empty-dir/0e470010-625d-443f-81ec-9bf965988212-dshm\") pod \"0e470010-625d-443f-81ec-9bf965988212\" (UID: \"0e470010-625d-443f-81ec-9bf965988212\") "
Oct 30 20:18:05.236665 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.236454    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/89009638-003d-4a5d-ab13-1f8c8f64d3b7-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89009638-003d-4a5d-ab13-1f8c8f64d3b7-model-cache\") pod \"89009638-003d-4a5d-ab13-1f8c8f64d3b7\" (UID: \"89009638-003d-4a5d-ab13-1f8c8f64d3b7\") "
Oct 30 20:18:05.236665 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.236531    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/89009638-003d-4a5d-ab13-1f8c8f64d3b7-home\" (UniqueName: \"kubernetes.io/empty-dir/89009638-003d-4a5d-ab13-1f8c8f64d3b7-home\") pod \"89009638-003d-4a5d-ab13-1f8c8f64d3b7\" (UID: \"89009638-003d-4a5d-ab13-1f8c8f64d3b7\") "
Oct 30 20:18:05.236665 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.236576    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a1857d2d-3b40-4e63-be33-be4ba476f926-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1857d2d-3b40-4e63-be33-be4ba476f926-model-cache\") pod \"a1857d2d-3b40-4e63-be33-be4ba476f926\" (UID: \"a1857d2d-3b40-4e63-be33-be4ba476f926\") "
Oct 30 20:18:05.236665 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.236662    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a1857d2d-3b40-4e63-be33-be4ba476f926-kube-api-access-p4xlr\" (UniqueName: \"kubernetes.io/projected/a1857d2d-3b40-4e63-be33-be4ba476f926-kube-api-access-p4xlr\") pod \"a1857d2d-3b40-4e63-be33-be4ba476f926\" (UID: \"a1857d2d-3b40-4e63-be33-be4ba476f926\") "
Oct 30 20:18:05.236983 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.236707    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/89009638-003d-4a5d-ab13-1f8c8f64d3b7-tls-certs\" (UniqueName: \"kubernetes.io/secret/89009638-003d-4a5d-ab13-1f8c8f64d3b7-tls-certs\") pod \"89009638-003d-4a5d-ab13-1f8c8f64d3b7\" (UID: \"89009638-003d-4a5d-ab13-1f8c8f64d3b7\") "
Oct 30 20:18:05.236983 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.236754    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/89009638-003d-4a5d-ab13-1f8c8f64d3b7-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89009638-003d-4a5d-ab13-1f8c8f64d3b7-tmp-dir\") pod \"89009638-003d-4a5d-ab13-1f8c8f64d3b7\" (UID: \"89009638-003d-4a5d-ab13-1f8c8f64d3b7\") "
Oct 30 20:18:05.236983 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.236811    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0e470010-625d-443f-81ec-9bf965988212-tls-certs\" (UniqueName: \"kubernetes.io/secret/0e470010-625d-443f-81ec-9bf965988212-tls-certs\") pod \"0e470010-625d-443f-81ec-9bf965988212\" (UID: \"0e470010-625d-443f-81ec-9bf965988212\") "
Oct 30 20:18:05.236983 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.236872    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0e470010-625d-443f-81ec-9bf965988212-home\" (UniqueName: \"kubernetes.io/empty-dir/0e470010-625d-443f-81ec-9bf965988212-home\") pod \"0e470010-625d-443f-81ec-9bf965988212\" (UID: \"0e470010-625d-443f-81ec-9bf965988212\") "
Oct 30 20:18:05.236983 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.236919    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a1857d2d-3b40-4e63-be33-be4ba476f926-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1857d2d-3b40-4e63-be33-be4ba476f926-tmp-dir\") pod \"a1857d2d-3b40-4e63-be33-be4ba476f926\" (UID: \"a1857d2d-3b40-4e63-be33-be4ba476f926\") "
Oct 30 20:18:05.236983 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.236950    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0e470010-625d-443f-81ec-9bf965988212-kube-api-access-wvdtg\" (UniqueName: \"kubernetes.io/projected/0e470010-625d-443f-81ec-9bf965988212-kube-api-access-wvdtg\") pod \"0e470010-625d-443f-81ec-9bf965988212\" (UID: \"0e470010-625d-443f-81ec-9bf965988212\") "
Oct 30 20:18:05.236983 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.236978    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a1857d2d-3b40-4e63-be33-be4ba476f926-tls-certs\" (UniqueName: \"kubernetes.io/secret/a1857d2d-3b40-4e63-be33-be4ba476f926-tls-certs\") pod \"a1857d2d-3b40-4e63-be33-be4ba476f926\" (UID: \"a1857d2d-3b40-4e63-be33-be4ba476f926\") "
Oct 30 20:18:05.237636 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.237024    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/89009638-003d-4a5d-ab13-1f8c8f64d3b7-kube-api-access-x6kwl\" (UniqueName: \"kubernetes.io/projected/89009638-003d-4a5d-ab13-1f8c8f64d3b7-kube-api-access-x6kwl\") pod \"89009638-003d-4a5d-ab13-1f8c8f64d3b7\" (UID: \"89009638-003d-4a5d-ab13-1f8c8f64d3b7\") "
Oct 30 20:18:05.237636 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.237052    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a1857d2d-3b40-4e63-be33-be4ba476f926-home\" (UniqueName: \"kubernetes.io/empty-dir/a1857d2d-3b40-4e63-be33-be4ba476f926-home\") pod \"a1857d2d-3b40-4e63-be33-be4ba476f926\" (UID: \"a1857d2d-3b40-4e63-be33-be4ba476f926\") "
Oct 30 20:18:05.240792 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.237022    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89009638-003d-4a5d-ab13-1f8c8f64d3b7-model-cache" pod "89009638-003d-4a5d-ab13-1f8c8f64d3b7" (UID: "89009638-003d-4a5d-ab13-1f8c8f64d3b7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:18:05.240939 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.237332    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e470010-625d-443f-81ec-9bf965988212-tmp-dir" pod "0e470010-625d-443f-81ec-9bf965988212" (UID: "0e470010-625d-443f-81ec-9bf965988212"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:18:05.240939 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.237425    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e470010-625d-443f-81ec-9bf965988212-model-cache" pod "0e470010-625d-443f-81ec-9bf965988212" (UID: "0e470010-625d-443f-81ec-9bf965988212"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:18:05.240939 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.237962    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1857d2d-3b40-4e63-be33-be4ba476f926-home" pod "a1857d2d-3b40-4e63-be33-be4ba476f926" (UID: "a1857d2d-3b40-4e63-be33-be4ba476f926"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:18:05.240939 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.239006    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89009638-003d-4a5d-ab13-1f8c8f64d3b7-dshm" pod "89009638-003d-4a5d-ab13-1f8c8f64d3b7" (UID: "89009638-003d-4a5d-ab13-1f8c8f64d3b7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:18:05.240939 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.239238    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e470010-625d-443f-81ec-9bf965988212-home" pod "0e470010-625d-443f-81ec-9bf965988212" (UID: "0e470010-625d-443f-81ec-9bf965988212"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:18:05.240939 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.239975    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1857d2d-3b40-4e63-be33-be4ba476f926-kube-api-access-p4xlr" pod "a1857d2d-3b40-4e63-be33-be4ba476f926" (UID: "a1857d2d-3b40-4e63-be33-be4ba476f926"). InnerVolumeSpecName "kube-api-access-p4xlr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:18:05.240939 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.240198    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89009638-003d-4a5d-ab13-1f8c8f64d3b7-home" pod "89009638-003d-4a5d-ab13-1f8c8f64d3b7" (UID: "89009638-003d-4a5d-ab13-1f8c8f64d3b7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:18:05.240939 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.240371    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1857d2d-3b40-4e63-be33-be4ba476f926-model-cache" pod "a1857d2d-3b40-4e63-be33-be4ba476f926" (UID: "a1857d2d-3b40-4e63-be33-be4ba476f926"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:18:05.240939 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.240586    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89009638-003d-4a5d-ab13-1f8c8f64d3b7-tmp-dir" pod "89009638-003d-4a5d-ab13-1f8c8f64d3b7" (UID: "89009638-003d-4a5d-ab13-1f8c8f64d3b7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:18:05.240939 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.240758    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1857d2d-3b40-4e63-be33-be4ba476f926-tmp-dir" pod "a1857d2d-3b40-4e63-be33-be4ba476f926" (UID: "a1857d2d-3b40-4e63-be33-be4ba476f926"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:18:05.241992 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.241965    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e470010-625d-443f-81ec-9bf965988212-tls-certs" pod "0e470010-625d-443f-81ec-9bf965988212" (UID: "0e470010-625d-443f-81ec-9bf965988212"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:18:05.242720 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.242693    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/89009638-003d-4a5d-ab13-1f8c8f64d3b7-tls-certs" pod "89009638-003d-4a5d-ab13-1f8c8f64d3b7" (UID: "89009638-003d-4a5d-ab13-1f8c8f64d3b7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:18:05.243108 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.243083    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89009638-003d-4a5d-ab13-1f8c8f64d3b7-kube-api-access-x6kwl" pod "89009638-003d-4a5d-ab13-1f8c8f64d3b7" (UID: "89009638-003d-4a5d-ab13-1f8c8f64d3b7"). InnerVolumeSpecName "kube-api-access-x6kwl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:18:05.245522 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.245372    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e470010-625d-443f-81ec-9bf965988212-kube-api-access-wvdtg" pod "0e470010-625d-443f-81ec-9bf965988212" (UID: "0e470010-625d-443f-81ec-9bf965988212"). InnerVolumeSpecName "kube-api-access-wvdtg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:18:05.245522 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.245406    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e470010-625d-443f-81ec-9bf965988212-dshm" pod "0e470010-625d-443f-81ec-9bf965988212" (UID: "0e470010-625d-443f-81ec-9bf965988212"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:18:05.246657 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.246631    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1857d2d-3b40-4e63-be33-be4ba476f926-dshm" pod "a1857d2d-3b40-4e63-be33-be4ba476f926" (UID: "a1857d2d-3b40-4e63-be33-be4ba476f926"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:18:05.247132 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.247107    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a1857d2d-3b40-4e63-be33-be4ba476f926-tls-certs" pod "a1857d2d-3b40-4e63-be33-be4ba476f926" (UID: "a1857d2d-3b40-4e63-be33-be4ba476f926"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:18:05.339119 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.339087    6996 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89009638-003d-4a5d-ab13-1f8c8f64d3b7-dshm\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:05.339119 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.339118    6996 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0e470010-625d-443f-81ec-9bf965988212-model-cache\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:05.339334 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.339128    6996 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1857d2d-3b40-4e63-be33-be4ba476f926-dshm\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:05.339334 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.339137    6996 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0e470010-625d-443f-81ec-9bf965988212-tmp-dir\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:05.339334 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.339145    6996 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0e470010-625d-443f-81ec-9bf965988212-dshm\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:05.339334 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.339155    6996 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89009638-003d-4a5d-ab13-1f8c8f64d3b7-model-cache\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:05.339334 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.339163    6996 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89009638-003d-4a5d-ab13-1f8c8f64d3b7-home\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:05.339334 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.339172    6996 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1857d2d-3b40-4e63-be33-be4ba476f926-model-cache\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:05.339334 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.339183    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p4xlr\" (UniqueName: \"kubernetes.io/projected/a1857d2d-3b40-4e63-be33-be4ba476f926-kube-api-access-p4xlr\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:05.339334 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.339193    6996 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89009638-003d-4a5d-ab13-1f8c8f64d3b7-tls-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:05.339334 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.339201    6996 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89009638-003d-4a5d-ab13-1f8c8f64d3b7-tmp-dir\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:05.339334 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.339209    6996 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e470010-625d-443f-81ec-9bf965988212-tls-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:05.339334 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.339217    6996 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0e470010-625d-443f-81ec-9bf965988212-home\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:05.339334 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.339225    6996 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1857d2d-3b40-4e63-be33-be4ba476f926-tmp-dir\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:05.339334 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.339233    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wvdtg\" (UniqueName: \"kubernetes.io/projected/0e470010-625d-443f-81ec-9bf965988212-kube-api-access-wvdtg\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:05.339334 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.339242    6996 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1857d2d-3b40-4e63-be33-be4ba476f926-tls-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:05.339334 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.339250    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x6kwl\" (UniqueName: \"kubernetes.io/projected/89009638-003d-4a5d-ab13-1f8c8f64d3b7-kube-api-access-x6kwl\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:05.339334 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.339257    6996 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1857d2d-3b40-4e63-be33-be4ba476f926-home\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:05.486103 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.486072    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-2a3e1b/distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq"]
Oct 30 20:18:05.492850 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.492825    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-2a3e1b/distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq"]
Oct 30 20:18:05.502837 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.502815    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-2a3e1b/distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk"]
Oct 30 20:18:05.507638 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.507614    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-2a3e1b/distinct-2-w4-2a3e1b-kserve-7f6cfdc556-d5rdk"]
Oct 30 20:18:05.518144 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.518122    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr"]
Oct 30 20:18:05.523043 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.523021    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-2a3e1b/premium-w4-2a3e1b-kserve-7599454564-4dcwr"]
Oct 30 20:18:05.902684 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:05.902597    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2a3e1b/distinct-w4-2a3e1b-kserve-c87c85c9-pdvzq" podUID="89009638-003d-4a5d-ab13-1f8c8f64d3b7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.88:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:18:06.793447 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:06.793413    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0e470010-625d-443f-81ec-9bf965988212" path="/var/lib/kubelet/pods/0e470010-625d-443f-81ec-9bf965988212/volumes"
Oct 30 20:18:06.794001 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:06.793984    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="89009638-003d-4a5d-ab13-1f8c8f64d3b7" path="/var/lib/kubelet/pods/89009638-003d-4a5d-ab13-1f8c8f64d3b7/volumes"
Oct 30 20:18:06.794549 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:06.794529    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a1857d2d-3b40-4e63-be33-be4ba476f926" path="/var/lib/kubelet/pods/a1857d2d-3b40-4e63-be33-be4ba476f926/volumes"
Oct 30 20:18:07.364249 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:07.364211    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-c1fdd6-76556ddfb-w9t9z"]
Oct 30 20:18:07.381133 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:07.381098    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-c1fdd6-5d68c879bc-ccgjb"]
Oct 30 20:18:09.474889 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:09.474820    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w2-f06af6-77784df88-qg6p6" podUID="0451a825-071c-4214-b5ea-800baa964cde" containerName="payload-processing" containerID="cri-o://03553bdb3ed85902eb05ef420ac75c54d97c5785e6f2bc3a1bf95eafccd8e97b" gracePeriod=25
Oct 30 20:18:12.365216 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:12.365156    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w3-c1fdd6-76556ddfb-w9t9z" podUID="71cedf55-9f63-4c56-b75f-ab67bb24d556" containerName="payload-processing" containerID="cri-o://3c3409a29bf819acfdd659f6777163d4330cd0d2df4cc59d2965841b5a781cc7" gracePeriod=25
Oct 30 20:18:12.381579 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:12.381544    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-c1fdd6-5d68c879bc-ccgjb" podUID="dc5d56e5-1eef-44d3-97c0-84e9af887161" containerName="payload-pre-processing" containerID="cri-o://0fe16b0c2359fc34bdf49b70910a721d291931704937bbade12905f4a23132d1" gracePeriod=25
Oct 30 20:18:14.041360 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:14.041323    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w3-c1fdd6-76556ddfb-w9t9z" podUID="71cedf55-9f63-4c56-b75f-ab67bb24d556" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 20:18:14.041799 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:14.041323    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-c1fdd6-5d68c879bc-ccgjb" podUID="dc5d56e5-1eef-44d3-97c0-84e9af887161" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 20:18:17.047188 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:17.047145    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b" podUID="19fe4fba-09ae-4620-b4d2-0ef8c2463d71" containerName="main" containerID="cri-o://8b1c7515b4a1b8e30a0d3ffb924e6c9741ab6f63e2beb13342b9cc7ca91763a3" gracePeriod=45
Oct 30 20:18:17.270802 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:17.270709    6996 generic.go:356] "Generic (PLEG): container finished" podID="19fe4fba-09ae-4620-b4d2-0ef8c2463d71" containerID="8b1c7515b4a1b8e30a0d3ffb924e6c9741ab6f63e2beb13342b9cc7ca91763a3" exitCode=0
Oct 30 20:18:17.270802 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:17.270762    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b" event={"ID":"19fe4fba-09ae-4620-b4d2-0ef8c2463d71","Type":"ContainerDied","Data":"8b1c7515b4a1b8e30a0d3ffb924e6c9741ab6f63e2beb13342b9cc7ca91763a3"}
Oct 30 20:18:17.350910 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:17.350884    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b"
Oct 30 20:18:17.363427 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:17.363400    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/19fe4fba-09ae-4620-b4d2-0ef8c2463d71-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19fe4fba-09ae-4620-b4d2-0ef8c2463d71-tmp-dir\") pod \"19fe4fba-09ae-4620-b4d2-0ef8c2463d71\" (UID: \"19fe4fba-09ae-4620-b4d2-0ef8c2463d71\") "
Oct 30 20:18:17.363589 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:17.363459    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/19fe4fba-09ae-4620-b4d2-0ef8c2463d71-kube-api-access-77mb5\" (UniqueName: \"kubernetes.io/projected/19fe4fba-09ae-4620-b4d2-0ef8c2463d71-kube-api-access-77mb5\") pod \"19fe4fba-09ae-4620-b4d2-0ef8c2463d71\" (UID: \"19fe4fba-09ae-4620-b4d2-0ef8c2463d71\") "
Oct 30 20:18:17.363589 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:17.363517    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/19fe4fba-09ae-4620-b4d2-0ef8c2463d71-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19fe4fba-09ae-4620-b4d2-0ef8c2463d71-model-cache\") pod \"19fe4fba-09ae-4620-b4d2-0ef8c2463d71\" (UID: \"19fe4fba-09ae-4620-b4d2-0ef8c2463d71\") "
Oct 30 20:18:17.363589 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:17.363560    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/19fe4fba-09ae-4620-b4d2-0ef8c2463d71-tls-certs\" (UniqueName: \"kubernetes.io/secret/19fe4fba-09ae-4620-b4d2-0ef8c2463d71-tls-certs\") pod \"19fe4fba-09ae-4620-b4d2-0ef8c2463d71\" (UID: \"19fe4fba-09ae-4620-b4d2-0ef8c2463d71\") "
Oct 30 20:18:17.363714 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:17.363651    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/19fe4fba-09ae-4620-b4d2-0ef8c2463d71-dshm\" (UniqueName: \"kubernetes.io/empty-dir/19fe4fba-09ae-4620-b4d2-0ef8c2463d71-dshm\") pod \"19fe4fba-09ae-4620-b4d2-0ef8c2463d71\" (UID: \"19fe4fba-09ae-4620-b4d2-0ef8c2463d71\") "
Oct 30 20:18:17.363714 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:17.363681    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/19fe4fba-09ae-4620-b4d2-0ef8c2463d71-home\" (UniqueName: \"kubernetes.io/empty-dir/19fe4fba-09ae-4620-b4d2-0ef8c2463d71-home\") pod \"19fe4fba-09ae-4620-b4d2-0ef8c2463d71\" (UID: \"19fe4fba-09ae-4620-b4d2-0ef8c2463d71\") "
Oct 30 20:18:17.363813 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:17.363712    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19fe4fba-09ae-4620-b4d2-0ef8c2463d71-tmp-dir" pod "19fe4fba-09ae-4620-b4d2-0ef8c2463d71" (UID: "19fe4fba-09ae-4620-b4d2-0ef8c2463d71"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:18:17.363890 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:17.363867    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19fe4fba-09ae-4620-b4d2-0ef8c2463d71-model-cache" pod "19fe4fba-09ae-4620-b4d2-0ef8c2463d71" (UID: "19fe4fba-09ae-4620-b4d2-0ef8c2463d71"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:18:17.364099 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:17.364072    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19fe4fba-09ae-4620-b4d2-0ef8c2463d71-home" pod "19fe4fba-09ae-4620-b4d2-0ef8c2463d71" (UID: "19fe4fba-09ae-4620-b4d2-0ef8c2463d71"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:18:17.364422 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:17.364402    6996 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19fe4fba-09ae-4620-b4d2-0ef8c2463d71-tmp-dir\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:17.364532 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:17.364429    6996 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19fe4fba-09ae-4620-b4d2-0ef8c2463d71-model-cache\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:17.364532 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:17.364446    6996 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19fe4fba-09ae-4620-b4d2-0ef8c2463d71-home\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:17.365949 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:17.365925    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19fe4fba-09ae-4620-b4d2-0ef8c2463d71-dshm" pod "19fe4fba-09ae-4620-b4d2-0ef8c2463d71" (UID: "19fe4fba-09ae-4620-b4d2-0ef8c2463d71"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:18:17.366033 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:17.365956    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19fe4fba-09ae-4620-b4d2-0ef8c2463d71-kube-api-access-77mb5" pod "19fe4fba-09ae-4620-b4d2-0ef8c2463d71" (UID: "19fe4fba-09ae-4620-b4d2-0ef8c2463d71"). InnerVolumeSpecName "kube-api-access-77mb5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:18:17.366033 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:17.365982    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19fe4fba-09ae-4620-b4d2-0ef8c2463d71-tls-certs" pod "19fe4fba-09ae-4620-b4d2-0ef8c2463d71" (UID: "19fe4fba-09ae-4620-b4d2-0ef8c2463d71"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:18:17.466434 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:17.466376    6996 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19fe4fba-09ae-4620-b4d2-0ef8c2463d71-dshm\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:17.466434 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:17.466425    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-77mb5\" (UniqueName: \"kubernetes.io/projected/19fe4fba-09ae-4620-b4d2-0ef8c2463d71-kube-api-access-77mb5\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:17.466434 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:17.466442    6996 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19fe4fba-09ae-4620-b4d2-0ef8c2463d71-tls-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:18.280634 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:18.280605    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b"
Oct 30 20:18:18.281159 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:18.280599    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b" event={"ID":"19fe4fba-09ae-4620-b4d2-0ef8c2463d71","Type":"ContainerDied","Data":"fa233be42d378d8c68f922d4cfd16c7b0945bff4699e72dbdce82a4e077327e5"}
Oct 30 20:18:18.281159 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:18.280740    6996 scope.go:122] "RemoveContainer" containerID="8b1c7515b4a1b8e30a0d3ffb924e6c9741ab6f63e2beb13342b9cc7ca91763a3"
Oct 30 20:18:18.320573 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:18.320539    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b"]
Oct 30 20:18:18.323536 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:18.323511    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-c1fdd6/distinct-2-w3-c1fdd6-kserve-76d78bd84f-25b8b"]
Oct 30 20:18:18.792230 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:18.792195    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="19fe4fba-09ae-4620-b4d2-0ef8c2463d71" path="/var/lib/kubelet/pods/19fe4fba-09ae-4620-b4d2-0ef8c2463d71/volumes"
Oct 30 20:18:19.093254 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:19.093212    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w2-f06af6-77784df88-qg6p6" podUID="0451a825-071c-4214-b5ea-800baa964cde" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 20:18:23.582071 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:23.582006    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-f06af6-7f99b6c86c-tdw5h"]
Oct 30 20:18:23.582746 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:23.582294    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-f06af6-7f99b6c86c-tdw5h" podUID="6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a" containerName="maas-api" containerID="cri-o://4cd55008fc5acfa4051d2e74b8e657826e9c18d5859900ad2df3bfaff7eab51f" gracePeriod=30
Oct 30 20:18:23.946676 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:23.943669    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-f06af6-7f99b6c86c-tdw5h"
Oct 30 20:18:24.041621 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:24.041580    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w3-c1fdd6-76556ddfb-w9t9z" podUID="71cedf55-9f63-4c56-b75f-ab67bb24d556" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 20:18:24.041854 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:24.041581    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-c1fdd6-5d68c879bc-ccgjb" podUID="dc5d56e5-1eef-44d3-97c0-84e9af887161" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 20:18:24.076354 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:24.076317    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a-metrics-certs\" (UniqueName: \"kubernetes.io/secret/6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a-metrics-certs\") pod \"6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a\" (UID: \"6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a\") "
Oct 30 20:18:24.076597 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:24.076420    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a-maas-api-tls\") pod \"6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a\" (UID: \"6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a\") "
Oct 30 20:18:24.076597 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:24.076504    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a-kube-api-access-lqj78\" (UniqueName: \"kubernetes.io/projected/6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a-kube-api-access-lqj78\") pod \"6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a\" (UID: \"6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a\") "
Oct 30 20:18:24.076713 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:24.076632    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a-openshift-service-ca\") pod \"6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a\" (UID: \"6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a\") "
Oct 30 20:18:24.077030 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:24.076996    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a-openshift-service-ca" pod "6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a" (UID: "6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:18:24.077254 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:24.077238    6996 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a-openshift-service-ca\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:24.078631 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:24.078604    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a-maas-api-tls" pod "6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a" (UID: "6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:18:24.078732 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:24.078670    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a-metrics-certs" pod "6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a" (UID: "6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:18:24.078732 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:24.078707    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a-kube-api-access-lqj78" pod "6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a" (UID: "6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a"). InnerVolumeSpecName "kube-api-access-lqj78". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:18:24.179060 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:24.179022    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lqj78\" (UniqueName: \"kubernetes.io/projected/6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a-kube-api-access-lqj78\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:24.179060 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:24.179061    6996 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a-metrics-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:24.179276 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:24.179078    6996 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a-maas-api-tls\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:24.323518 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:24.323464    6996 generic.go:356] "Generic (PLEG): container finished" podID="6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a" containerID="4cd55008fc5acfa4051d2e74b8e657826e9c18d5859900ad2df3bfaff7eab51f" exitCode=0
Oct 30 20:18:24.323688 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:24.323537    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-f06af6-7f99b6c86c-tdw5h"
Oct 30 20:18:24.323688 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:24.323533    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-f06af6-7f99b6c86c-tdw5h" event={"ID":"6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a","Type":"ContainerDied","Data":"4cd55008fc5acfa4051d2e74b8e657826e9c18d5859900ad2df3bfaff7eab51f"}
Oct 30 20:18:24.323688 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:24.323660    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-f06af6-7f99b6c86c-tdw5h" event={"ID":"6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a","Type":"ContainerDied","Data":"8b6ce75b50711c5bc42af0f91abda8932e479c6f9eb9eb4f93f7d6e17977fd73"}
Oct 30 20:18:24.323688 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:24.323682    6996 scope.go:122] "RemoveContainer" containerID="4cd55008fc5acfa4051d2e74b8e657826e9c18d5859900ad2df3bfaff7eab51f"
Oct 30 20:18:24.335150 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:24.335130    6996 scope.go:122] "RemoveContainer" containerID="4cd55008fc5acfa4051d2e74b8e657826e9c18d5859900ad2df3bfaff7eab51f"
Oct 30 20:18:24.335417 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:18:24.335399    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4cd55008fc5acfa4051d2e74b8e657826e9c18d5859900ad2df3bfaff7eab51f\": container with ID starting with 4cd55008fc5acfa4051d2e74b8e657826e9c18d5859900ad2df3bfaff7eab51f not found: ID does not exist" containerID="4cd55008fc5acfa4051d2e74b8e657826e9c18d5859900ad2df3bfaff7eab51f"
Oct 30 20:18:24.335453 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:24.335425    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4cd55008fc5acfa4051d2e74b8e657826e9c18d5859900ad2df3bfaff7eab51f"} err="failed to get container status \"4cd55008fc5acfa4051d2e74b8e657826e9c18d5859900ad2df3bfaff7eab51f\": rpc error: code = NotFound desc = could not find container \"4cd55008fc5acfa4051d2e74b8e657826e9c18d5859900ad2df3bfaff7eab51f\": container with ID starting with 4cd55008fc5acfa4051d2e74b8e657826e9c18d5859900ad2df3bfaff7eab51f not found: ID does not exist"
Oct 30 20:18:24.348296 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:24.348245    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-f06af6-7f99b6c86c-tdw5h"]
Oct 30 20:18:24.353691 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:24.353665    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-f06af6-7f99b6c86c-tdw5h"]
Oct 30 20:18:24.803319 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:24.803242    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a" path="/var/lib/kubelet/pods/6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a/volumes"
Oct 30 20:18:25.998507 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:25.997904    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-c1fdd6-c87447bf7-8r6gk"]
Oct 30 20:18:25.998507 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:25.998173    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-c1fdd6-c87447bf7-8r6gk" podUID="34320df2-abbd-4508-91e0-bad01e9f5c25" containerName="maas-api" containerID="cri-o://054931ed9d39b08455ddf923d7ba89892bab8bb4ec8297c8e0b162c2b98e153b" gracePeriod=30
Oct 30 20:18:26.069338 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:26.069300    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-c1fdd6-c87447bf7-8r6gk" podUID="34320df2-abbd-4508-91e0-bad01e9f5c25" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.65:8443/health\": dial tcp 10.134.0.65:8443: connect: connection refused"
Oct 30 20:18:26.291170 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:26.291139    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-c1fdd6-c87447bf7-8r6gk"
Oct 30 20:18:26.464934 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:26.418496    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/34320df2-abbd-4508-91e0-bad01e9f5c25-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/34320df2-abbd-4508-91e0-bad01e9f5c25-maas-api-tls\") pod \"34320df2-abbd-4508-91e0-bad01e9f5c25\" (UID: \"34320df2-abbd-4508-91e0-bad01e9f5c25\") "
Oct 30 20:18:26.464934 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:26.418550    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/34320df2-abbd-4508-91e0-bad01e9f5c25-metrics-certs\" (UniqueName: \"kubernetes.io/secret/34320df2-abbd-4508-91e0-bad01e9f5c25-metrics-certs\") pod \"34320df2-abbd-4508-91e0-bad01e9f5c25\" (UID: \"34320df2-abbd-4508-91e0-bad01e9f5c25\") "
Oct 30 20:18:26.464934 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:26.418759    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/34320df2-abbd-4508-91e0-bad01e9f5c25-kube-api-access-xdhqz\" (UniqueName: \"kubernetes.io/projected/34320df2-abbd-4508-91e0-bad01e9f5c25-kube-api-access-xdhqz\") pod \"34320df2-abbd-4508-91e0-bad01e9f5c25\" (UID: \"34320df2-abbd-4508-91e0-bad01e9f5c25\") "
Oct 30 20:18:26.464934 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:26.418815    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/34320df2-abbd-4508-91e0-bad01e9f5c25-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/34320df2-abbd-4508-91e0-bad01e9f5c25-openshift-service-ca\") pod \"34320df2-abbd-4508-91e0-bad01e9f5c25\" (UID: \"34320df2-abbd-4508-91e0-bad01e9f5c25\") "
Oct 30 20:18:26.464934 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:26.420856    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/34320df2-abbd-4508-91e0-bad01e9f5c25-openshift-service-ca" pod "34320df2-abbd-4508-91e0-bad01e9f5c25" (UID: "34320df2-abbd-4508-91e0-bad01e9f5c25"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:18:26.464934 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:26.423795    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/34320df2-abbd-4508-91e0-bad01e9f5c25-maas-api-tls" pod "34320df2-abbd-4508-91e0-bad01e9f5c25" (UID: "34320df2-abbd-4508-91e0-bad01e9f5c25"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:18:26.479093 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:26.474910    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/34320df2-abbd-4508-91e0-bad01e9f5c25-metrics-certs" pod "34320df2-abbd-4508-91e0-bad01e9f5c25" (UID: "34320df2-abbd-4508-91e0-bad01e9f5c25"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:18:26.487588 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:26.486122    6996 generic.go:356] "Generic (PLEG): container finished" podID="34320df2-abbd-4508-91e0-bad01e9f5c25" containerID="054931ed9d39b08455ddf923d7ba89892bab8bb4ec8297c8e0b162c2b98e153b" exitCode=0
Oct 30 20:18:26.487588 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:26.486383    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-c1fdd6-c87447bf7-8r6gk" event={"ID":"34320df2-abbd-4508-91e0-bad01e9f5c25","Type":"ContainerDied","Data":"054931ed9d39b08455ddf923d7ba89892bab8bb4ec8297c8e0b162c2b98e153b"}
Oct 30 20:18:26.487588 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:26.486437    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-c1fdd6-c87447bf7-8r6gk" event={"ID":"34320df2-abbd-4508-91e0-bad01e9f5c25","Type":"ContainerDied","Data":"838273cc8dca0cc8e9b1645cf829cff31f329e67008b23fcd450b00c9811fb40"}
Oct 30 20:18:26.487588 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:26.486465    6996 scope.go:122] "RemoveContainer" containerID="054931ed9d39b08455ddf923d7ba89892bab8bb4ec8297c8e0b162c2b98e153b"
Oct 30 20:18:26.487588 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:26.486641    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-c1fdd6-c87447bf7-8r6gk"
Oct 30 20:18:26.491442 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:26.491397    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34320df2-abbd-4508-91e0-bad01e9f5c25-kube-api-access-xdhqz" pod "34320df2-abbd-4508-91e0-bad01e9f5c25" (UID: "34320df2-abbd-4508-91e0-bad01e9f5c25"). InnerVolumeSpecName "kube-api-access-xdhqz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:18:26.504719 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:26.504685    6996 scope.go:122] "RemoveContainer" containerID="054931ed9d39b08455ddf923d7ba89892bab8bb4ec8297c8e0b162c2b98e153b"
Oct 30 20:18:26.505093 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:18:26.505069    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"054931ed9d39b08455ddf923d7ba89892bab8bb4ec8297c8e0b162c2b98e153b\": container with ID starting with 054931ed9d39b08455ddf923d7ba89892bab8bb4ec8297c8e0b162c2b98e153b not found: ID does not exist" containerID="054931ed9d39b08455ddf923d7ba89892bab8bb4ec8297c8e0b162c2b98e153b"
Oct 30 20:18:26.505251 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:26.505232    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"054931ed9d39b08455ddf923d7ba89892bab8bb4ec8297c8e0b162c2b98e153b"} err="failed to get container status \"054931ed9d39b08455ddf923d7ba89892bab8bb4ec8297c8e0b162c2b98e153b\": rpc error: code = NotFound desc = could not find container \"054931ed9d39b08455ddf923d7ba89892bab8bb4ec8297c8e0b162c2b98e153b\": container with ID starting with 054931ed9d39b08455ddf923d7ba89892bab8bb4ec8297c8e0b162c2b98e153b not found: ID does not exist"
Oct 30 20:18:26.520193 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:26.520132    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xdhqz\" (UniqueName: \"kubernetes.io/projected/34320df2-abbd-4508-91e0-bad01e9f5c25-kube-api-access-xdhqz\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:26.520193 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:26.520160    6996 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/34320df2-abbd-4508-91e0-bad01e9f5c25-openshift-service-ca\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:26.520193 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:26.520172    6996 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/34320df2-abbd-4508-91e0-bad01e9f5c25-maas-api-tls\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:26.520193 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:26.520180    6996 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/34320df2-abbd-4508-91e0-bad01e9f5c25-metrics-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:26.816579 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:26.816540    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-c1fdd6-c87447bf7-8r6gk"]
Oct 30 20:18:26.818757 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:26.818723    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-c1fdd6-c87447bf7-8r6gk"]
Oct 30 20:18:28.444882 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:28.444840    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 20:18:28.792940 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:28.792850    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="34320df2-abbd-4508-91e0-bad01e9f5c25" path="/var/lib/kubelet/pods/34320df2-abbd-4508-91e0-bad01e9f5c25/volumes"
Oct 30 20:18:29.093534 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:29.093492    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w2-f06af6-77784df88-qg6p6" podUID="0451a825-071c-4214-b5ea-800baa964cde" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 20:18:29.873661 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:29.873626    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-f06af6-77784df88-qg6p6"
Oct 30 20:18:29.999002 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:29.993110    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0451a825-071c-4214-b5ea-800baa964cde-kube-api-access-2rpc8\" (UniqueName: \"kubernetes.io/projected/0451a825-071c-4214-b5ea-800baa964cde-kube-api-access-2rpc8\") pod \"0451a825-071c-4214-b5ea-800baa964cde\" (UID: \"0451a825-071c-4214-b5ea-800baa964cde\") "
Oct 30 20:18:29.999002 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:29.993418    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0451a825-071c-4214-b5ea-800baa964cde-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0451a825-071c-4214-b5ea-800baa964cde-plugins-config-volume\") pod \"0451a825-071c-4214-b5ea-800baa964cde\" (UID: \"0451a825-071c-4214-b5ea-800baa964cde\") "
Oct 30 20:18:29.999002 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:29.994706    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0451a825-071c-4214-b5ea-800baa964cde-plugins-config-volume" pod "0451a825-071c-4214-b5ea-800baa964cde" (UID: "0451a825-071c-4214-b5ea-800baa964cde"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:18:29.999002 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:29.995209    6996 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0451a825-071c-4214-b5ea-800baa964cde-plugins-config-volume\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:30.007124 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:30.007081    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0451a825-071c-4214-b5ea-800baa964cde-kube-api-access-2rpc8" pod "0451a825-071c-4214-b5ea-800baa964cde" (UID: "0451a825-071c-4214-b5ea-800baa964cde"). InnerVolumeSpecName "kube-api-access-2rpc8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:18:30.096744 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:30.096667    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2rpc8\" (UniqueName: \"kubernetes.io/projected/0451a825-071c-4214-b5ea-800baa964cde-kube-api-access-2rpc8\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:30.558507 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:30.557737    6996 generic.go:356] "Generic (PLEG): container finished" podID="0451a825-071c-4214-b5ea-800baa964cde" containerID="03553bdb3ed85902eb05ef420ac75c54d97c5785e6f2bc3a1bf95eafccd8e97b" exitCode=0
Oct 30 20:18:30.558507 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:30.557819    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-f06af6-77784df88-qg6p6" event={"ID":"0451a825-071c-4214-b5ea-800baa964cde","Type":"ContainerDied","Data":"03553bdb3ed85902eb05ef420ac75c54d97c5785e6f2bc3a1bf95eafccd8e97b"}
Oct 30 20:18:30.558507 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:30.557905    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-f06af6-77784df88-qg6p6" event={"ID":"0451a825-071c-4214-b5ea-800baa964cde","Type":"ContainerDied","Data":"62f624a03dbac977429655b69f9fbbc0a4a2109e23358cf2e6218183c50c8194"}
Oct 30 20:18:30.558507 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:30.557945    6996 scope.go:122] "RemoveContainer" containerID="03553bdb3ed85902eb05ef420ac75c54d97c5785e6f2bc3a1bf95eafccd8e97b"
Oct 30 20:18:30.558507 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:30.558130    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-f06af6-77784df88-qg6p6"
Oct 30 20:18:30.589757 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:30.589729    6996 scope.go:122] "RemoveContainer" containerID="03553bdb3ed85902eb05ef420ac75c54d97c5785e6f2bc3a1bf95eafccd8e97b"
Oct 30 20:18:30.590130 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:18:30.590097    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03553bdb3ed85902eb05ef420ac75c54d97c5785e6f2bc3a1bf95eafccd8e97b\": container with ID starting with 03553bdb3ed85902eb05ef420ac75c54d97c5785e6f2bc3a1bf95eafccd8e97b not found: ID does not exist" containerID="03553bdb3ed85902eb05ef420ac75c54d97c5785e6f2bc3a1bf95eafccd8e97b"
Oct 30 20:18:30.590211 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:30.590145    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03553bdb3ed85902eb05ef420ac75c54d97c5785e6f2bc3a1bf95eafccd8e97b"} err="failed to get container status \"03553bdb3ed85902eb05ef420ac75c54d97c5785e6f2bc3a1bf95eafccd8e97b\": rpc error: code = NotFound desc = could not find container \"03553bdb3ed85902eb05ef420ac75c54d97c5785e6f2bc3a1bf95eafccd8e97b\": container with ID starting with 03553bdb3ed85902eb05ef420ac75c54d97c5785e6f2bc3a1bf95eafccd8e97b not found: ID does not exist"
Oct 30 20:18:30.613882 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:30.613846    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-f06af6-77784df88-qg6p6"]
Oct 30 20:18:30.626673 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:30.626646    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-f06af6-77784df88-qg6p6"]
Oct 30 20:18:30.793031 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:30.792991    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0451a825-071c-4214-b5ea-800baa964cde" path="/var/lib/kubelet/pods/0451a825-071c-4214-b5ea-800baa964cde/volumes"
Oct 30 20:18:30.864629 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:30.864594    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-2a3e1b-6db86c8568-g56mm"]
Oct 30 20:18:30.864916 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:30.864890    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-2a3e1b-6db86c8568-g56mm" podUID="f32d849b-5292-444c-8369-87385b8d5348" containerName="maas-api" containerID="cri-o://d25ef271cdd63ee1bc4fa885ca1914ad20d1a36152441ec8529246a8239921ec" gracePeriod=30
Oct 30 20:18:31.268556 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:31.267313    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-2a3e1b-6db86c8568-g56mm"
Oct 30 20:18:31.270069 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:18:31.270039    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 20:18:31.316403 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:31.316351    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f32d849b-5292-444c-8369-87385b8d5348-kube-api-access-2kxkf\" (UniqueName: \"kubernetes.io/projected/f32d849b-5292-444c-8369-87385b8d5348-kube-api-access-2kxkf\") pod \"f32d849b-5292-444c-8369-87385b8d5348\" (UID: \"f32d849b-5292-444c-8369-87385b8d5348\") "
Oct 30 20:18:31.316623 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:31.316556    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f32d849b-5292-444c-8369-87385b8d5348-metrics-certs\" (UniqueName: \"kubernetes.io/secret/f32d849b-5292-444c-8369-87385b8d5348-metrics-certs\") pod \"f32d849b-5292-444c-8369-87385b8d5348\" (UID: \"f32d849b-5292-444c-8369-87385b8d5348\") "
Oct 30 20:18:31.316623 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:31.316599    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f32d849b-5292-444c-8369-87385b8d5348-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f32d849b-5292-444c-8369-87385b8d5348-openshift-service-ca\") pod \"f32d849b-5292-444c-8369-87385b8d5348\" (UID: \"f32d849b-5292-444c-8369-87385b8d5348\") "
Oct 30 20:18:31.316763 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:31.316739    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f32d849b-5292-444c-8369-87385b8d5348-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f32d849b-5292-444c-8369-87385b8d5348-maas-api-tls\") pod \"f32d849b-5292-444c-8369-87385b8d5348\" (UID: \"f32d849b-5292-444c-8369-87385b8d5348\") "
Oct 30 20:18:31.317522 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:31.317454    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f32d849b-5292-444c-8369-87385b8d5348-openshift-service-ca" pod "f32d849b-5292-444c-8369-87385b8d5348" (UID: "f32d849b-5292-444c-8369-87385b8d5348"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:18:31.321527 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:31.320375    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f32d849b-5292-444c-8369-87385b8d5348-kube-api-access-2kxkf" pod "f32d849b-5292-444c-8369-87385b8d5348" (UID: "f32d849b-5292-444c-8369-87385b8d5348"). InnerVolumeSpecName "kube-api-access-2kxkf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:18:31.321527 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:31.320670    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f32d849b-5292-444c-8369-87385b8d5348-metrics-certs" pod "f32d849b-5292-444c-8369-87385b8d5348" (UID: "f32d849b-5292-444c-8369-87385b8d5348"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:18:31.321527 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:31.320856    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f32d849b-5292-444c-8369-87385b8d5348-maas-api-tls" pod "f32d849b-5292-444c-8369-87385b8d5348" (UID: "f32d849b-5292-444c-8369-87385b8d5348"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:18:31.418727 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:31.418686    6996 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f32d849b-5292-444c-8369-87385b8d5348-metrics-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:31.418923 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:31.418727    6996 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f32d849b-5292-444c-8369-87385b8d5348-openshift-service-ca\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:31.418923 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:31.418751    6996 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f32d849b-5292-444c-8369-87385b8d5348-maas-api-tls\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:31.418923 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:31.418763    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2kxkf\" (UniqueName: \"kubernetes.io/projected/f32d849b-5292-444c-8369-87385b8d5348-kube-api-access-2kxkf\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:31.567936 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:31.567900    6996 generic.go:356] "Generic (PLEG): container finished" podID="f32d849b-5292-444c-8369-87385b8d5348" containerID="d25ef271cdd63ee1bc4fa885ca1914ad20d1a36152441ec8529246a8239921ec" exitCode=0
Oct 30 20:18:31.568136 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:31.567955    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-2a3e1b-6db86c8568-g56mm"
Oct 30 20:18:31.568136 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:31.567975    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-2a3e1b-6db86c8568-g56mm" event={"ID":"f32d849b-5292-444c-8369-87385b8d5348","Type":"ContainerDied","Data":"d25ef271cdd63ee1bc4fa885ca1914ad20d1a36152441ec8529246a8239921ec"}
Oct 30 20:18:31.568136 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:31.568031    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-2a3e1b-6db86c8568-g56mm" event={"ID":"f32d849b-5292-444c-8369-87385b8d5348","Type":"ContainerDied","Data":"417812bf77ba1cd84d4f356b0b74731e7ccc617891081515358277e82e0ce4f9"}
Oct 30 20:18:31.568136 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:31.568055    6996 scope.go:122] "RemoveContainer" containerID="d25ef271cdd63ee1bc4fa885ca1914ad20d1a36152441ec8529246a8239921ec"
Oct 30 20:18:31.580887 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:31.580861    6996 scope.go:122] "RemoveContainer" containerID="d25ef271cdd63ee1bc4fa885ca1914ad20d1a36152441ec8529246a8239921ec"
Oct 30 20:18:31.581286 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:18:31.581258    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d25ef271cdd63ee1bc4fa885ca1914ad20d1a36152441ec8529246a8239921ec\": container with ID starting with d25ef271cdd63ee1bc4fa885ca1914ad20d1a36152441ec8529246a8239921ec not found: ID does not exist" containerID="d25ef271cdd63ee1bc4fa885ca1914ad20d1a36152441ec8529246a8239921ec"
Oct 30 20:18:31.581380 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:31.581299    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d25ef271cdd63ee1bc4fa885ca1914ad20d1a36152441ec8529246a8239921ec"} err="failed to get container status \"d25ef271cdd63ee1bc4fa885ca1914ad20d1a36152441ec8529246a8239921ec\": rpc error: code = NotFound desc = could not find container \"d25ef271cdd63ee1bc4fa885ca1914ad20d1a36152441ec8529246a8239921ec\": container with ID starting with d25ef271cdd63ee1bc4fa885ca1914ad20d1a36152441ec8529246a8239921ec not found: ID does not exist"
Oct 30 20:18:31.596115 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:31.596080    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-2a3e1b-6db86c8568-g56mm"]
Oct 30 20:18:31.599541 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:31.599515    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-2a3e1b-6db86c8568-g56mm"]
Oct 30 20:18:32.604815 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:32.604778    6996 generic.go:356] "Generic (PLEG): container finished" podID="dc5d56e5-1eef-44d3-97c0-84e9af887161" containerID="0fe16b0c2359fc34bdf49b70910a721d291931704937bbade12905f4a23132d1" exitCode=0
Oct 30 20:18:32.605424 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:32.605393    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-c1fdd6-5d68c879bc-ccgjb" event={"ID":"dc5d56e5-1eef-44d3-97c0-84e9af887161","Type":"ContainerDied","Data":"0fe16b0c2359fc34bdf49b70910a721d291931704937bbade12905f4a23132d1"}
Oct 30 20:18:32.610351 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:32.610184    6996 generic.go:356] "Generic (PLEG): container finished" podID="71cedf55-9f63-4c56-b75f-ab67bb24d556" containerID="3c3409a29bf819acfdd659f6777163d4330cd0d2df4cc59d2965841b5a781cc7" exitCode=0
Oct 30 20:18:32.610351 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:32.610283    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-c1fdd6-76556ddfb-w9t9z" event={"ID":"71cedf55-9f63-4c56-b75f-ab67bb24d556","Type":"ContainerDied","Data":"3c3409a29bf819acfdd659f6777163d4330cd0d2df4cc59d2965841b5a781cc7"}
Oct 30 20:18:32.727662 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:32.727635    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-c1fdd6-76556ddfb-w9t9z"
Oct 30 20:18:32.750861 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:32.750839    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-c1fdd6-5d68c879bc-ccgjb"
Oct 30 20:18:32.814140 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:32.814096    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f32d849b-5292-444c-8369-87385b8d5348" path="/var/lib/kubelet/pods/f32d849b-5292-444c-8369-87385b8d5348/volumes"
Oct 30 20:18:32.839528 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:32.839495    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dc5d56e5-1eef-44d3-97c0-84e9af887161-kube-api-access-np75j\" (UniqueName: \"kubernetes.io/projected/dc5d56e5-1eef-44d3-97c0-84e9af887161-kube-api-access-np75j\") pod \"dc5d56e5-1eef-44d3-97c0-84e9af887161\" (UID: \"dc5d56e5-1eef-44d3-97c0-84e9af887161\") "
Oct 30 20:18:32.839721 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:32.839539    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dc5d56e5-1eef-44d3-97c0-84e9af887161-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc5d56e5-1eef-44d3-97c0-84e9af887161-plugins-config-volume\") pod \"dc5d56e5-1eef-44d3-97c0-84e9af887161\" (UID: \"dc5d56e5-1eef-44d3-97c0-84e9af887161\") "
Oct 30 20:18:32.839721 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:32.839664    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/71cedf55-9f63-4c56-b75f-ab67bb24d556-kube-api-access-v8d2w\" (UniqueName: \"kubernetes.io/projected/71cedf55-9f63-4c56-b75f-ab67bb24d556-kube-api-access-v8d2w\") pod \"71cedf55-9f63-4c56-b75f-ab67bb24d556\" (UID: \"71cedf55-9f63-4c56-b75f-ab67bb24d556\") "
Oct 30 20:18:32.839857 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:32.839790    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/71cedf55-9f63-4c56-b75f-ab67bb24d556-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/71cedf55-9f63-4c56-b75f-ab67bb24d556-plugins-config-volume\") pod \"71cedf55-9f63-4c56-b75f-ab67bb24d556\" (UID: \"71cedf55-9f63-4c56-b75f-ab67bb24d556\") "
Oct 30 20:18:32.840027 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:32.840001    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc5d56e5-1eef-44d3-97c0-84e9af887161-plugins-config-volume" pod "dc5d56e5-1eef-44d3-97c0-84e9af887161" (UID: "dc5d56e5-1eef-44d3-97c0-84e9af887161"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:18:32.840233 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:32.840205    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71cedf55-9f63-4c56-b75f-ab67bb24d556-plugins-config-volume" pod "71cedf55-9f63-4c56-b75f-ab67bb24d556" (UID: "71cedf55-9f63-4c56-b75f-ab67bb24d556"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:18:32.840440 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:32.840421    6996 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/71cedf55-9f63-4c56-b75f-ab67bb24d556-plugins-config-volume\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:32.840566 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:32.840444    6996 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc5d56e5-1eef-44d3-97c0-84e9af887161-plugins-config-volume\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:32.841889 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:32.841868    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc5d56e5-1eef-44d3-97c0-84e9af887161-kube-api-access-np75j" pod "dc5d56e5-1eef-44d3-97c0-84e9af887161" (UID: "dc5d56e5-1eef-44d3-97c0-84e9af887161"). InnerVolumeSpecName "kube-api-access-np75j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:18:32.841988 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:32.841896    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71cedf55-9f63-4c56-b75f-ab67bb24d556-kube-api-access-v8d2w" pod "71cedf55-9f63-4c56-b75f-ab67bb24d556" (UID: "71cedf55-9f63-4c56-b75f-ab67bb24d556"). InnerVolumeSpecName "kube-api-access-v8d2w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:18:32.941304 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:32.941223    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-np75j\" (UniqueName: \"kubernetes.io/projected/dc5d56e5-1eef-44d3-97c0-84e9af887161-kube-api-access-np75j\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:32.941304 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:32.941253    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v8d2w\" (UniqueName: \"kubernetes.io/projected/71cedf55-9f63-4c56-b75f-ab67bb24d556-kube-api-access-v8d2w\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:33.620268 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:33.620234    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-c1fdd6-5d68c879bc-ccgjb"
Oct 30 20:18:33.620268 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:33.620249    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-c1fdd6-5d68c879bc-ccgjb" event={"ID":"dc5d56e5-1eef-44d3-97c0-84e9af887161","Type":"ContainerDied","Data":"e94a528c11be2a42d16f1bfd4cb7ee1b2391a429a728a698da4c951360767817"}
Oct 30 20:18:33.620855 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:33.620333    6996 scope.go:122] "RemoveContainer" containerID="0fe16b0c2359fc34bdf49b70910a721d291931704937bbade12905f4a23132d1"
Oct 30 20:18:33.622075 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:33.622054    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-c1fdd6-76556ddfb-w9t9z"
Oct 30 20:18:33.622288 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:33.622075    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-c1fdd6-76556ddfb-w9t9z" event={"ID":"71cedf55-9f63-4c56-b75f-ab67bb24d556","Type":"ContainerDied","Data":"df391d0beb5fdcce9de92a7b4812e0f7986dd7404f3702d84bd34492e06741db"}
Oct 30 20:18:33.634164 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:33.634138    6996 scope.go:122] "RemoveContainer" containerID="3c3409a29bf819acfdd659f6777163d4330cd0d2df4cc59d2965841b5a781cc7"
Oct 30 20:18:33.655371 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:33.655334    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-c1fdd6-5d68c879bc-ccgjb"]
Oct 30 20:18:33.658555 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:33.658525    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-c1fdd6-5d68c879bc-ccgjb"]
Oct 30 20:18:33.672400 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:33.672369    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-c1fdd6-76556ddfb-w9t9z"]
Oct 30 20:18:33.680974 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:33.680952    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-c1fdd6-76556ddfb-w9t9z"]
Oct 30 20:18:34.798965 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:34.798925    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="71cedf55-9f63-4c56-b75f-ab67bb24d556" path="/var/lib/kubelet/pods/71cedf55-9f63-4c56-b75f-ab67bb24d556/volumes"
Oct 30 20:18:34.799396 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:34.799381    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dc5d56e5-1eef-44d3-97c0-84e9af887161" path="/var/lib/kubelet/pods/dc5d56e5-1eef-44d3-97c0-84e9af887161/volumes"
Oct 30 20:18:36.024602 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:36.024560    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q"]
Oct 30 20:18:36.025193 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:36.025157    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q" podUID="e4dc9cd9-2754-4e38-af83-046f8536a53b" containerName="istio-proxy" containerID="cri-o://4e76f852fdd7b74ff9a30e4a940491d1c806c247b7b6eb613489659148d1e143" gracePeriod=30
Oct 30 20:18:37.425872 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:37.425837    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw"]
Oct 30 20:18:37.426315 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:37.426119    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw" podUID="e41df835-fccd-45bb-bfca-ca1f81f6261a" containerName="istio-proxy" containerID="cri-o://e79c87eafd6a76089707fac19fee20748a19a2eb6ef2509076ec6e6be096c035" gracePeriod=30
Oct 30 20:18:39.173522 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:39.173469    6996 patch_prober.go:29] interesting pod/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 20:18:39.173941 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:39.173556    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw" podUID="e41df835-fccd-45bb-bfca-ca1f81f6261a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 20:18:40.289096 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.289059    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-bdf9e6-6b949fc948-zgdg6"]
Oct 30 20:18:40.290123 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.289804    6996 cpu_manager.go:414] "containerMap: removing container" podUID="06a16fac-9e8d-4724-a949-a8d31f13002d" containerName="istio-proxy"
Oct 30 20:18:40.290123 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.289826    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="06a16fac-9e8d-4724-a949-a8d31f13002d" containerName="istio-proxy"
Oct 30 20:18:40.290123 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.289851    6996 cpu_manager.go:414] "containerMap: removing container" podUID="3bc8f341-bbf4-42d4-941a-0e636de353fe" containerName="payload-processing"
Oct 30 20:18:40.290123 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.289860    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3bc8f341-bbf4-42d4-941a-0e636de353fe" containerName="payload-processing"
Oct 30 20:18:40.290123 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.289875    6996 cpu_manager.go:414] "containerMap: removing container" podUID="89a06251-4c13-4b46-99fb-6b3db0c4ddf6" containerName="maas-api"
Oct 30 20:18:40.290123 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.289885    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="89a06251-4c13-4b46-99fb-6b3db0c4ddf6" containerName="maas-api"
Oct 30 20:18:40.290123 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.289902    6996 cpu_manager.go:414] "containerMap: removing container" podUID="0451a825-071c-4214-b5ea-800baa964cde" containerName="payload-processing"
Oct 30 20:18:40.290123 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.289911    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0451a825-071c-4214-b5ea-800baa964cde" containerName="payload-processing"
Oct 30 20:18:40.290123 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.289929    6996 cpu_manager.go:414] "containerMap: removing container" podUID="dc5d56e5-1eef-44d3-97c0-84e9af887161" containerName="payload-pre-processing"
Oct 30 20:18:40.290123 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.289936    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dc5d56e5-1eef-44d3-97c0-84e9af887161" containerName="payload-pre-processing"
Oct 30 20:18:40.290123 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.289958    6996 cpu_manager.go:414] "containerMap: removing container" podUID="0e470010-625d-443f-81ec-9bf965988212" containerName="main"
Oct 30 20:18:40.290123 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.289965    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0e470010-625d-443f-81ec-9bf965988212" containerName="main"
Oct 30 20:18:40.290123 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.289980    6996 cpu_manager.go:414] "containerMap: removing container" podUID="f32d849b-5292-444c-8369-87385b8d5348" containerName="maas-api"
Oct 30 20:18:40.290123 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.289987    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f32d849b-5292-444c-8369-87385b8d5348" containerName="maas-api"
Oct 30 20:18:40.290123 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.290000    6996 cpu_manager.go:414] "containerMap: removing container" podUID="3bdb062c-f6d3-4d06-9c69-b9d0b8e11726" containerName="maas-api"
Oct 30 20:18:40.290123 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.290007    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3bdb062c-f6d3-4d06-9c69-b9d0b8e11726" containerName="maas-api"
Oct 30 20:18:40.290123 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.290016    6996 cpu_manager.go:414] "containerMap: removing container" podUID="71cedf55-9f63-4c56-b75f-ab67bb24d556" containerName="payload-processing"
Oct 30 20:18:40.290123 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.290020    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="71cedf55-9f63-4c56-b75f-ab67bb24d556" containerName="payload-processing"
Oct 30 20:18:40.290123 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.290026    6996 cpu_manager.go:414] "containerMap: removing container" podUID="33ca6985-3896-45c4-bb2f-b58fb5fb40b4" containerName="maas-api"
Oct 30 20:18:40.290123 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.290031    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="33ca6985-3896-45c4-bb2f-b58fb5fb40b4" containerName="maas-api"
Oct 30 20:18:40.290123 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.290039    6996 cpu_manager.go:414] "containerMap: removing container" podUID="8d38b232-9550-4272-8174-7c457b1f4b38" containerName="maas-api"
Oct 30 20:18:40.290123 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.290043    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8d38b232-9550-4272-8174-7c457b1f4b38" containerName="maas-api"
Oct 30 20:18:40.290123 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.290053    6996 cpu_manager.go:414] "containerMap: removing container" podUID="34320df2-abbd-4508-91e0-bad01e9f5c25" containerName="maas-api"
Oct 30 20:18:40.290123 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.290058    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="34320df2-abbd-4508-91e0-bad01e9f5c25" containerName="maas-api"
Oct 30 20:18:40.290123 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.290072    6996 cpu_manager.go:414] "containerMap: removing container" podUID="89009638-003d-4a5d-ab13-1f8c8f64d3b7" containerName="main"
Oct 30 20:18:40.290123 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.290076    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="89009638-003d-4a5d-ab13-1f8c8f64d3b7" containerName="main"
Oct 30 20:18:40.290123 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.290083    6996 cpu_manager.go:414] "containerMap: removing container" podUID="a1857d2d-3b40-4e63-be33-be4ba476f926" containerName="main"
Oct 30 20:18:40.290123 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.290087    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a1857d2d-3b40-4e63-be33-be4ba476f926" containerName="main"
Oct 30 20:18:40.290123 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.290100    6996 cpu_manager.go:414] "containerMap: removing container" podUID="19fe4fba-09ae-4620-b4d2-0ef8c2463d71" containerName="main"
Oct 30 20:18:40.290123 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.290106    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="19fe4fba-09ae-4620-b4d2-0ef8c2463d71" containerName="main"
Oct 30 20:18:40.290123 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.290115    6996 cpu_manager.go:414] "containerMap: removing container" podUID="6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a" containerName="maas-api"
Oct 30 20:18:40.290123 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.290119    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a" containerName="maas-api"
Oct 30 20:18:40.290123 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.290133    6996 cpu_manager.go:414] "containerMap: removing container" podUID="80f2d362-3093-42b0-a3c0-c8bf25c43a84" containerName="maas-api"
Oct 30 20:18:40.290123 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.290138    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="80f2d362-3093-42b0-a3c0-c8bf25c43a84" containerName="maas-api"
Oct 30 20:18:40.290123 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.290145    6996 cpu_manager.go:414] "containerMap: removing container" podUID="4b366fcc-1688-4082-bb67-b2f05afdd019" containerName="maas-api"
Oct 30 20:18:40.290123 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.290150    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4b366fcc-1688-4082-bb67-b2f05afdd019" containerName="maas-api"
Oct 30 20:18:40.291668 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.290258    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="3bc8f341-bbf4-42d4-941a-0e636de353fe" containerName="payload-processing"
Oct 30 20:18:40.291668 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.290266    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="89a06251-4c13-4b46-99fb-6b3db0c4ddf6" containerName="maas-api"
Oct 30 20:18:40.291668 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.290274    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="0451a825-071c-4214-b5ea-800baa964cde" containerName="payload-processing"
Oct 30 20:18:40.291668 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.290282    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="dc5d56e5-1eef-44d3-97c0-84e9af887161" containerName="payload-pre-processing"
Oct 30 20:18:40.291668 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.290289    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="0e470010-625d-443f-81ec-9bf965988212" containerName="main"
Oct 30 20:18:40.291668 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.290296    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="f32d849b-5292-444c-8369-87385b8d5348" containerName="maas-api"
Oct 30 20:18:40.291668 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.290301    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="3bdb062c-f6d3-4d06-9c69-b9d0b8e11726" containerName="maas-api"
Oct 30 20:18:40.291668 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.290307    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="71cedf55-9f63-4c56-b75f-ab67bb24d556" containerName="payload-processing"
Oct 30 20:18:40.291668 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.290312    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="33ca6985-3896-45c4-bb2f-b58fb5fb40b4" containerName="maas-api"
Oct 30 20:18:40.291668 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.290319    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="8d38b232-9550-4272-8174-7c457b1f4b38" containerName="maas-api"
Oct 30 20:18:40.291668 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.290324    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="34320df2-abbd-4508-91e0-bad01e9f5c25" containerName="maas-api"
Oct 30 20:18:40.291668 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.290330    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="89009638-003d-4a5d-ab13-1f8c8f64d3b7" containerName="main"
Oct 30 20:18:40.291668 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.290335    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="a1857d2d-3b40-4e63-be33-be4ba476f926" containerName="main"
Oct 30 20:18:40.291668 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.290340    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="19fe4fba-09ae-4620-b4d2-0ef8c2463d71" containerName="main"
Oct 30 20:18:40.291668 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.290346    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="7a68d2bd-f65c-4013-b700-3a0da95b301f" containerName="cleanup"
Oct 30 20:18:40.291668 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.290351    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="6a0eb5d6-3bc0-4c49-9fd1-50fbc70d716a" containerName="maas-api"
Oct 30 20:18:40.291668 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.290357    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="80f2d362-3093-42b0-a3c0-c8bf25c43a84" containerName="maas-api"
Oct 30 20:18:40.291668 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.290362    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="4b366fcc-1688-4082-bb67-b2f05afdd019" containerName="maas-api"
Oct 30 20:18:40.291668 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.290367    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="06a16fac-9e8d-4724-a949-a8d31f13002d" containerName="istio-proxy"
Oct 30 20:18:40.295232 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.295209    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bdf9e6-6b949fc948-zgdg6"
Oct 30 20:18:40.300723 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.300697    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-bdf9e6-serving-cert\""
Oct 30 20:18:40.309989 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.309961    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-bdf9e6-6b949fc948-zgdg6"]
Oct 30 20:18:40.392240 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.392201    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/21af0b9f-44b8-43fc-bcda-751e196384ff-openshift-service-ca\") pod \"maas-api-e2e-inf-b-bdf9e6-6b949fc948-zgdg6\" (UID: \"21af0b9f-44b8-43fc-bcda-751e196384ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bdf9e6-6b949fc948-zgdg6"
Oct 30 20:18:40.392240 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.392241    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/21af0b9f-44b8-43fc-bcda-751e196384ff-maas-api-tls\") pod \"maas-api-e2e-inf-b-bdf9e6-6b949fc948-zgdg6\" (UID: \"21af0b9f-44b8-43fc-bcda-751e196384ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bdf9e6-6b949fc948-zgdg6"
Oct 30 20:18:40.392436 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.392257    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/21af0b9f-44b8-43fc-bcda-751e196384ff-metrics-certs\") pod \"maas-api-e2e-inf-b-bdf9e6-6b949fc948-zgdg6\" (UID: \"21af0b9f-44b8-43fc-bcda-751e196384ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bdf9e6-6b949fc948-zgdg6"
Oct 30 20:18:40.392492 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.392431    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lqf7\" (UniqueName: \"kubernetes.io/projected/21af0b9f-44b8-43fc-bcda-751e196384ff-kube-api-access-6lqf7\") pod \"maas-api-e2e-inf-b-bdf9e6-6b949fc948-zgdg6\" (UID: \"21af0b9f-44b8-43fc-bcda-751e196384ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bdf9e6-6b949fc948-zgdg6"
Oct 30 20:18:40.493918 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.493872    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lqf7\" (UniqueName: \"kubernetes.io/projected/21af0b9f-44b8-43fc-bcda-751e196384ff-kube-api-access-6lqf7\") pod \"maas-api-e2e-inf-b-bdf9e6-6b949fc948-zgdg6\" (UID: \"21af0b9f-44b8-43fc-bcda-751e196384ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bdf9e6-6b949fc948-zgdg6"
Oct 30 20:18:40.494363 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.494169    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/21af0b9f-44b8-43fc-bcda-751e196384ff-openshift-service-ca\") pod \"maas-api-e2e-inf-b-bdf9e6-6b949fc948-zgdg6\" (UID: \"21af0b9f-44b8-43fc-bcda-751e196384ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bdf9e6-6b949fc948-zgdg6"
Oct 30 20:18:40.494363 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.494211    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/21af0b9f-44b8-43fc-bcda-751e196384ff-maas-api-tls\") pod \"maas-api-e2e-inf-b-bdf9e6-6b949fc948-zgdg6\" (UID: \"21af0b9f-44b8-43fc-bcda-751e196384ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bdf9e6-6b949fc948-zgdg6"
Oct 30 20:18:40.494363 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.494237    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/21af0b9f-44b8-43fc-bcda-751e196384ff-metrics-certs\") pod \"maas-api-e2e-inf-b-bdf9e6-6b949fc948-zgdg6\" (UID: \"21af0b9f-44b8-43fc-bcda-751e196384ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bdf9e6-6b949fc948-zgdg6"
Oct 30 20:18:40.497510 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.495670    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/21af0b9f-44b8-43fc-bcda-751e196384ff-openshift-service-ca\") pod \"maas-api-e2e-inf-b-bdf9e6-6b949fc948-zgdg6\" (UID: \"21af0b9f-44b8-43fc-bcda-751e196384ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bdf9e6-6b949fc948-zgdg6"
Oct 30 20:18:40.501428 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.501398    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/21af0b9f-44b8-43fc-bcda-751e196384ff-maas-api-tls\") pod \"maas-api-e2e-inf-b-bdf9e6-6b949fc948-zgdg6\" (UID: \"21af0b9f-44b8-43fc-bcda-751e196384ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bdf9e6-6b949fc948-zgdg6"
Oct 30 20:18:40.503988 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.503963    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/21af0b9f-44b8-43fc-bcda-751e196384ff-metrics-certs\") pod \"maas-api-e2e-inf-b-bdf9e6-6b949fc948-zgdg6\" (UID: \"21af0b9f-44b8-43fc-bcda-751e196384ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bdf9e6-6b949fc948-zgdg6"
Oct 30 20:18:40.507993 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.507967    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lqf7\" (UniqueName: \"kubernetes.io/projected/21af0b9f-44b8-43fc-bcda-751e196384ff-kube-api-access-6lqf7\") pod \"maas-api-e2e-inf-b-bdf9e6-6b949fc948-zgdg6\" (UID: \"21af0b9f-44b8-43fc-bcda-751e196384ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bdf9e6-6b949fc948-zgdg6"
Oct 30 20:18:40.611161 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.611131    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bdf9e6-6b949fc948-zgdg6"
Oct 30 20:18:40.911006 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:40.910975    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-bdf9e6-6b949fc948-zgdg6"]
Oct 30 20:18:40.913562 ip-10-0-141-42 kubenswrapper[6996]: W1030 20:18:40.913533    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21af0b9f_44b8_43fc_bcda_751e196384ff.slice/crio-c306bbd1a3170d9da9e586d81fd7fdc4ce8d51ddd7dab636e142d2466a97895f WatchSource:0}: Error finding container c306bbd1a3170d9da9e586d81fd7fdc4ce8d51ddd7dab636e142d2466a97895f: Status 404 returned error can't find the container with id c306bbd1a3170d9da9e586d81fd7fdc4ce8d51ddd7dab636e142d2466a97895f
Oct 30 20:18:41.330984 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:41.330958    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q"
Oct 30 20:18:41.407961 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:41.407923    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e4dc9cd9-2754-4e38-af83-046f8536a53b-kube-api-access-75mqr\" (UniqueName: \"kubernetes.io/projected/e4dc9cd9-2754-4e38-af83-046f8536a53b-kube-api-access-75mqr\") pod \"e4dc9cd9-2754-4e38-af83-046f8536a53b\" (UID: \"e4dc9cd9-2754-4e38-af83-046f8536a53b\") "
Oct 30 20:18:41.408162 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:41.408020    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e4dc9cd9-2754-4e38-af83-046f8536a53b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4dc9cd9-2754-4e38-af83-046f8536a53b-istio-podinfo\") pod \"e4dc9cd9-2754-4e38-af83-046f8536a53b\" (UID: \"e4dc9cd9-2754-4e38-af83-046f8536a53b\") "
Oct 30 20:18:41.408162 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:41.408059    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e4dc9cd9-2754-4e38-af83-046f8536a53b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4dc9cd9-2754-4e38-af83-046f8536a53b-workload-certs\") pod \"e4dc9cd9-2754-4e38-af83-046f8536a53b\" (UID: \"e4dc9cd9-2754-4e38-af83-046f8536a53b\") "
Oct 30 20:18:41.408162 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:41.408096    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e4dc9cd9-2754-4e38-af83-046f8536a53b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4dc9cd9-2754-4e38-af83-046f8536a53b-istio-data\") pod \"e4dc9cd9-2754-4e38-af83-046f8536a53b\" (UID: \"e4dc9cd9-2754-4e38-af83-046f8536a53b\") "
Oct 30 20:18:41.408162 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:41.408155    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e4dc9cd9-2754-4e38-af83-046f8536a53b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4dc9cd9-2754-4e38-af83-046f8536a53b-credential-socket\") pod \"e4dc9cd9-2754-4e38-af83-046f8536a53b\" (UID: \"e4dc9cd9-2754-4e38-af83-046f8536a53b\") "
Oct 30 20:18:41.408592 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:41.408204    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e4dc9cd9-2754-4e38-af83-046f8536a53b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4dc9cd9-2754-4e38-af83-046f8536a53b-istiod-ca-cert\") pod \"e4dc9cd9-2754-4e38-af83-046f8536a53b\" (UID: \"e4dc9cd9-2754-4e38-af83-046f8536a53b\") "
Oct 30 20:18:41.408592 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:41.408357    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e4dc9cd9-2754-4e38-af83-046f8536a53b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4dc9cd9-2754-4e38-af83-046f8536a53b-istio-envoy\") pod \"e4dc9cd9-2754-4e38-af83-046f8536a53b\" (UID: \"e4dc9cd9-2754-4e38-af83-046f8536a53b\") "
Oct 30 20:18:41.408592 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:41.408392    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e4dc9cd9-2754-4e38-af83-046f8536a53b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4dc9cd9-2754-4e38-af83-046f8536a53b-workload-socket\") pod \"e4dc9cd9-2754-4e38-af83-046f8536a53b\" (UID: \"e4dc9cd9-2754-4e38-af83-046f8536a53b\") "
Oct 30 20:18:41.408592 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:41.408511    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e4dc9cd9-2754-4e38-af83-046f8536a53b-istio-token\" (UniqueName: \"kubernetes.io/projected/e4dc9cd9-2754-4e38-af83-046f8536a53b-istio-token\") pod \"e4dc9cd9-2754-4e38-af83-046f8536a53b\" (UID: \"e4dc9cd9-2754-4e38-af83-046f8536a53b\") "
Oct 30 20:18:41.409928 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:41.409850    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4dc9cd9-2754-4e38-af83-046f8536a53b-istio-data" pod "e4dc9cd9-2754-4e38-af83-046f8536a53b" (UID: "e4dc9cd9-2754-4e38-af83-046f8536a53b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:18:41.410169 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:41.409988    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4dc9cd9-2754-4e38-af83-046f8536a53b-istiod-ca-cert" pod "e4dc9cd9-2754-4e38-af83-046f8536a53b" (UID: "e4dc9cd9-2754-4e38-af83-046f8536a53b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:18:41.410315 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:41.410183    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4dc9cd9-2754-4e38-af83-046f8536a53b-credential-socket" pod "e4dc9cd9-2754-4e38-af83-046f8536a53b" (UID: "e4dc9cd9-2754-4e38-af83-046f8536a53b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:18:41.410315 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:41.410208    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4dc9cd9-2754-4e38-af83-046f8536a53b-workload-socket" pod "e4dc9cd9-2754-4e38-af83-046f8536a53b" (UID: "e4dc9cd9-2754-4e38-af83-046f8536a53b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:18:41.410315 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:41.410281    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4dc9cd9-2754-4e38-af83-046f8536a53b-workload-certs" pod "e4dc9cd9-2754-4e38-af83-046f8536a53b" (UID: "e4dc9cd9-2754-4e38-af83-046f8536a53b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:18:41.413519 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:41.413446    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4dc9cd9-2754-4e38-af83-046f8536a53b-istio-envoy" pod "e4dc9cd9-2754-4e38-af83-046f8536a53b" (UID: "e4dc9cd9-2754-4e38-af83-046f8536a53b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:18:41.414710 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:41.414683    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e4dc9cd9-2754-4e38-af83-046f8536a53b-istio-podinfo" pod "e4dc9cd9-2754-4e38-af83-046f8536a53b" (UID: "e4dc9cd9-2754-4e38-af83-046f8536a53b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 20:18:41.414885 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:41.414800    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4dc9cd9-2754-4e38-af83-046f8536a53b-istio-token" pod "e4dc9cd9-2754-4e38-af83-046f8536a53b" (UID: "e4dc9cd9-2754-4e38-af83-046f8536a53b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:18:41.416606 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:41.416498    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4dc9cd9-2754-4e38-af83-046f8536a53b-kube-api-access-75mqr" pod "e4dc9cd9-2754-4e38-af83-046f8536a53b" (UID: "e4dc9cd9-2754-4e38-af83-046f8536a53b"). InnerVolumeSpecName "kube-api-access-75mqr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:18:41.511088 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:41.511055    6996 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4dc9cd9-2754-4e38-af83-046f8536a53b-credential-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:41.511222 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:41.511091    6996 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4dc9cd9-2754-4e38-af83-046f8536a53b-istiod-ca-cert\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:41.511222 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:41.511102    6996 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4dc9cd9-2754-4e38-af83-046f8536a53b-istio-envoy\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:41.511222 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:41.511115    6996 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4dc9cd9-2754-4e38-af83-046f8536a53b-workload-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:41.511222 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:41.511126    6996 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4dc9cd9-2754-4e38-af83-046f8536a53b-istio-token\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:41.511222 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:41.511136    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-75mqr\" (UniqueName: \"kubernetes.io/projected/e4dc9cd9-2754-4e38-af83-046f8536a53b-kube-api-access-75mqr\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:41.511222 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:41.511151    6996 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4dc9cd9-2754-4e38-af83-046f8536a53b-istio-podinfo\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:41.511222 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:41.511165    6996 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4dc9cd9-2754-4e38-af83-046f8536a53b-workload-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:41.511222 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:41.511180    6996 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4dc9cd9-2754-4e38-af83-046f8536a53b-istio-data\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:41.700905 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:41.700828    6996 generic.go:356] "Generic (PLEG): container finished" podID="e4dc9cd9-2754-4e38-af83-046f8536a53b" containerID="4e76f852fdd7b74ff9a30e4a940491d1c806c247b7b6eb613489659148d1e143" exitCode=0
Oct 30 20:18:41.700905 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:41.700883    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q"
Oct 30 20:18:41.701100 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:41.700910    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q" event={"ID":"e4dc9cd9-2754-4e38-af83-046f8536a53b","Type":"ContainerDied","Data":"4e76f852fdd7b74ff9a30e4a940491d1c806c247b7b6eb613489659148d1e143"}
Oct 30 20:18:41.701100 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:41.700971    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q" event={"ID":"e4dc9cd9-2754-4e38-af83-046f8536a53b","Type":"ContainerDied","Data":"fadeab9ec2914c51273ab05c0c67e3ff3307a6fea2b3a156bf6a6d7d6da95fd8"}
Oct 30 20:18:41.701100 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:41.701002    6996 scope.go:122] "RemoveContainer" containerID="4e76f852fdd7b74ff9a30e4a940491d1c806c247b7b6eb613489659148d1e143"
Oct 30 20:18:41.702836 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:41.702793    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bdf9e6-6b949fc948-zgdg6" event={"ID":"21af0b9f-44b8-43fc-bcda-751e196384ff","Type":"ContainerStarted","Data":"905d3b472d44ebce176ef0717cba463c5eb8db1718696af65c82fe7b75d32f83"}
Oct 30 20:18:41.702980 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:41.702845    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bdf9e6-6b949fc948-zgdg6" event={"ID":"21af0b9f-44b8-43fc-bcda-751e196384ff","Type":"ContainerStarted","Data":"c306bbd1a3170d9da9e586d81fd7fdc4ce8d51ddd7dab636e142d2466a97895f"}
Oct 30 20:18:41.702980 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:41.702897    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bdf9e6-6b949fc948-zgdg6"
Oct 30 20:18:41.714153 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:41.714126    6996 scope.go:122] "RemoveContainer" containerID="4e76f852fdd7b74ff9a30e4a940491d1c806c247b7b6eb613489659148d1e143"
Oct 30 20:18:41.714551 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:18:41.714515    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e76f852fdd7b74ff9a30e4a940491d1c806c247b7b6eb613489659148d1e143\": container with ID starting with 4e76f852fdd7b74ff9a30e4a940491d1c806c247b7b6eb613489659148d1e143 not found: ID does not exist" containerID="4e76f852fdd7b74ff9a30e4a940491d1c806c247b7b6eb613489659148d1e143"
Oct 30 20:18:41.714722 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:41.714560    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e76f852fdd7b74ff9a30e4a940491d1c806c247b7b6eb613489659148d1e143"} err="failed to get container status \"4e76f852fdd7b74ff9a30e4a940491d1c806c247b7b6eb613489659148d1e143\": rpc error: code = NotFound desc = could not find container \"4e76f852fdd7b74ff9a30e4a940491d1c806c247b7b6eb613489659148d1e143\": container with ID starting with 4e76f852fdd7b74ff9a30e4a940491d1c806c247b7b6eb613489659148d1e143 not found: ID does not exist"
Oct 30 20:18:41.742466 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:41.742411    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bdf9e6-6b949fc948-zgdg6" podStartSLOduration=1.266072091 podStartE2EDuration="1.742392865s" totalImagesPullingTime="476.320774ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:18:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:18:41.736448339 +0000 UTC m=+2797.064566494" watchObservedRunningTime="2026-10-30 20:18:41.742392865 +0000 UTC m=+2797.070511023"
Oct 30 20:18:41.758640 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:41.758612    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q"]
Oct 30 20:18:41.766561 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:41.766524    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-f06af6-openshift-default-f48f877dc-8c68q"]
Oct 30 20:18:42.739378 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:42.739337    6996 generic.go:356] "Generic (PLEG): container finished" podID="e41df835-fccd-45bb-bfca-ca1f81f6261a" containerID="e79c87eafd6a76089707fac19fee20748a19a2eb6ef2509076ec6e6be096c035" exitCode=0
Oct 30 20:18:42.740131 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:42.739652    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw" event={"ID":"e41df835-fccd-45bb-bfca-ca1f81f6261a","Type":"ContainerDied","Data":"e79c87eafd6a76089707fac19fee20748a19a2eb6ef2509076ec6e6be096c035"}
Oct 30 20:18:42.761288 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:42.761263    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw"
Oct 30 20:18:42.797051 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:42.797011    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e4dc9cd9-2754-4e38-af83-046f8536a53b" path="/var/lib/kubelet/pods/e4dc9cd9-2754-4e38-af83-046f8536a53b/volumes"
Oct 30 20:18:42.835190 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:42.835150    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e41df835-fccd-45bb-bfca-ca1f81f6261a-istio-token\" (UniqueName: \"kubernetes.io/projected/e41df835-fccd-45bb-bfca-ca1f81f6261a-istio-token\") pod \"e41df835-fccd-45bb-bfca-ca1f81f6261a\" (UID: \"e41df835-fccd-45bb-bfca-ca1f81f6261a\") "
Oct 30 20:18:42.835190 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:42.835187    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e41df835-fccd-45bb-bfca-ca1f81f6261a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e41df835-fccd-45bb-bfca-ca1f81f6261a-istio-podinfo\") pod \"e41df835-fccd-45bb-bfca-ca1f81f6261a\" (UID: \"e41df835-fccd-45bb-bfca-ca1f81f6261a\") "
Oct 30 20:18:42.835458 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:42.835316    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e41df835-fccd-45bb-bfca-ca1f81f6261a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e41df835-fccd-45bb-bfca-ca1f81f6261a-istiod-ca-cert\") pod \"e41df835-fccd-45bb-bfca-ca1f81f6261a\" (UID: \"e41df835-fccd-45bb-bfca-ca1f81f6261a\") "
Oct 30 20:18:42.835458 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:42.835347    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e41df835-fccd-45bb-bfca-ca1f81f6261a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e41df835-fccd-45bb-bfca-ca1f81f6261a-istio-envoy\") pod \"e41df835-fccd-45bb-bfca-ca1f81f6261a\" (UID: \"e41df835-fccd-45bb-bfca-ca1f81f6261a\") "
Oct 30 20:18:42.835458 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:42.835377    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e41df835-fccd-45bb-bfca-ca1f81f6261a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e41df835-fccd-45bb-bfca-ca1f81f6261a-credential-socket\") pod \"e41df835-fccd-45bb-bfca-ca1f81f6261a\" (UID: \"e41df835-fccd-45bb-bfca-ca1f81f6261a\") "
Oct 30 20:18:42.835458 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:42.835403    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e41df835-fccd-45bb-bfca-ca1f81f6261a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e41df835-fccd-45bb-bfca-ca1f81f6261a-istio-data\") pod \"e41df835-fccd-45bb-bfca-ca1f81f6261a\" (UID: \"e41df835-fccd-45bb-bfca-ca1f81f6261a\") "
Oct 30 20:18:42.835458 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:42.835448    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e41df835-fccd-45bb-bfca-ca1f81f6261a-kube-api-access-9qq6t\" (UniqueName: \"kubernetes.io/projected/e41df835-fccd-45bb-bfca-ca1f81f6261a-kube-api-access-9qq6t\") pod \"e41df835-fccd-45bb-bfca-ca1f81f6261a\" (UID: \"e41df835-fccd-45bb-bfca-ca1f81f6261a\") "
Oct 30 20:18:42.835772 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:42.835596    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e41df835-fccd-45bb-bfca-ca1f81f6261a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e41df835-fccd-45bb-bfca-ca1f81f6261a-workload-certs\") pod \"e41df835-fccd-45bb-bfca-ca1f81f6261a\" (UID: \"e41df835-fccd-45bb-bfca-ca1f81f6261a\") "
Oct 30 20:18:42.835772 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:42.835665    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e41df835-fccd-45bb-bfca-ca1f81f6261a-credential-socket" pod "e41df835-fccd-45bb-bfca-ca1f81f6261a" (UID: "e41df835-fccd-45bb-bfca-ca1f81f6261a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:18:42.835887 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:42.835833    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e41df835-fccd-45bb-bfca-ca1f81f6261a-workload-certs" pod "e41df835-fccd-45bb-bfca-ca1f81f6261a" (UID: "e41df835-fccd-45bb-bfca-ca1f81f6261a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:18:42.835887 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:42.835843    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e41df835-fccd-45bb-bfca-ca1f81f6261a-istiod-ca-cert" pod "e41df835-fccd-45bb-bfca-ca1f81f6261a" (UID: "e41df835-fccd-45bb-bfca-ca1f81f6261a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:18:42.836008 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:42.835879    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e41df835-fccd-45bb-bfca-ca1f81f6261a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e41df835-fccd-45bb-bfca-ca1f81f6261a-workload-socket\") pod \"e41df835-fccd-45bb-bfca-ca1f81f6261a\" (UID: \"e41df835-fccd-45bb-bfca-ca1f81f6261a\") "
Oct 30 20:18:42.836008 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:42.835891    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e41df835-fccd-45bb-bfca-ca1f81f6261a-istio-data" pod "e41df835-fccd-45bb-bfca-ca1f81f6261a" (UID: "e41df835-fccd-45bb-bfca-ca1f81f6261a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:18:42.836116 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:42.836101    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e41df835-fccd-45bb-bfca-ca1f81f6261a-workload-socket" pod "e41df835-fccd-45bb-bfca-ca1f81f6261a" (UID: "e41df835-fccd-45bb-bfca-ca1f81f6261a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:18:42.836856 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:42.836559    6996 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e41df835-fccd-45bb-bfca-ca1f81f6261a-workload-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:42.836856 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:42.836585    6996 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e41df835-fccd-45bb-bfca-ca1f81f6261a-workload-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:42.836856 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:42.836600    6996 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e41df835-fccd-45bb-bfca-ca1f81f6261a-istiod-ca-cert\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:42.836856 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:42.836614    6996 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e41df835-fccd-45bb-bfca-ca1f81f6261a-credential-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:42.836856 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:42.836630    6996 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e41df835-fccd-45bb-bfca-ca1f81f6261a-istio-data\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:42.838071 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:42.838045    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e41df835-fccd-45bb-bfca-ca1f81f6261a-istio-podinfo" pod "e41df835-fccd-45bb-bfca-ca1f81f6261a" (UID: "e41df835-fccd-45bb-bfca-ca1f81f6261a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 20:18:42.838162 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:42.838043    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e41df835-fccd-45bb-bfca-ca1f81f6261a-istio-envoy" pod "e41df835-fccd-45bb-bfca-ca1f81f6261a" (UID: "e41df835-fccd-45bb-bfca-ca1f81f6261a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:18:42.838162 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:42.838150    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e41df835-fccd-45bb-bfca-ca1f81f6261a-istio-token" pod "e41df835-fccd-45bb-bfca-ca1f81f6261a" (UID: "e41df835-fccd-45bb-bfca-ca1f81f6261a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:18:42.838246 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:42.838177    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e41df835-fccd-45bb-bfca-ca1f81f6261a-kube-api-access-9qq6t" pod "e41df835-fccd-45bb-bfca-ca1f81f6261a" (UID: "e41df835-fccd-45bb-bfca-ca1f81f6261a"). InnerVolumeSpecName "kube-api-access-9qq6t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:18:42.908808 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:42.908766    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc"]
Oct 30 20:18:42.909779 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:42.909548    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc" podUID="7180ebbf-2671-4778-81a7-57a8aae4a783" containerName="istio-proxy" containerID="cri-o://bff4c9621e743d156397f4765bc0b85e01dd4b592bc26e11ec2f83a706173f90" gracePeriod=30
Oct 30 20:18:42.937947 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:42.937882    6996 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e41df835-fccd-45bb-bfca-ca1f81f6261a-istio-envoy\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:42.937947 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:42.937914    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9qq6t\" (UniqueName: \"kubernetes.io/projected/e41df835-fccd-45bb-bfca-ca1f81f6261a-kube-api-access-9qq6t\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:42.937947 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:42.937925    6996 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e41df835-fccd-45bb-bfca-ca1f81f6261a-istio-token\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:42.938181 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:42.937953    6996 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e41df835-fccd-45bb-bfca-ca1f81f6261a-istio-podinfo\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:43.760221 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:43.760176    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw" event={"ID":"e41df835-fccd-45bb-bfca-ca1f81f6261a","Type":"ContainerDied","Data":"fc82f523aecfe9c5effea5085c752db1f0891affe4674a3e6b2c298a2f596d06"}
Oct 30 20:18:43.760721 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:43.760282    6996 scope.go:122] "RemoveContainer" containerID="e79c87eafd6a76089707fac19fee20748a19a2eb6ef2509076ec6e6be096c035"
Oct 30 20:18:43.760721 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:43.760197    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw"
Oct 30 20:18:43.794717 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:43.794684    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw"]
Oct 30 20:18:43.803830 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:43.803798    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-c1fdd6-openshift-default-5845b6b8cf-5hvzw"]
Oct 30 20:18:44.808004 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:44.807964    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e41df835-fccd-45bb-bfca-ca1f81f6261a" path="/var/lib/kubelet/pods/e41df835-fccd-45bb-bfca-ca1f81f6261a/volumes"
Oct 30 20:18:45.789262 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:45.788993    6996 patch_prober.go:29] interesting pod/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 20:18:45.789456 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:45.789089    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc" podUID="7180ebbf-2671-4778-81a7-57a8aae4a783" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 20:18:47.751324 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:47.751291    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bdf9e6-6b949fc948-zgdg6"
Oct 30 20:18:48.214631 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:48.214606    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc"
Oct 30 20:18:48.304135 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:48.304058    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7180ebbf-2671-4778-81a7-57a8aae4a783-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7180ebbf-2671-4778-81a7-57a8aae4a783-istio-podinfo\") pod \"7180ebbf-2671-4778-81a7-57a8aae4a783\" (UID: \"7180ebbf-2671-4778-81a7-57a8aae4a783\") "
Oct 30 20:18:48.304310 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:48.304157    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7180ebbf-2671-4778-81a7-57a8aae4a783-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7180ebbf-2671-4778-81a7-57a8aae4a783-istio-envoy\") pod \"7180ebbf-2671-4778-81a7-57a8aae4a783\" (UID: \"7180ebbf-2671-4778-81a7-57a8aae4a783\") "
Oct 30 20:18:48.304310 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:48.304193    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7180ebbf-2671-4778-81a7-57a8aae4a783-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7180ebbf-2671-4778-81a7-57a8aae4a783-workload-certs\") pod \"7180ebbf-2671-4778-81a7-57a8aae4a783\" (UID: \"7180ebbf-2671-4778-81a7-57a8aae4a783\") "
Oct 30 20:18:48.304310 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:48.304245    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7180ebbf-2671-4778-81a7-57a8aae4a783-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7180ebbf-2671-4778-81a7-57a8aae4a783-workload-socket\") pod \"7180ebbf-2671-4778-81a7-57a8aae4a783\" (UID: \"7180ebbf-2671-4778-81a7-57a8aae4a783\") "
Oct 30 20:18:48.304310 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:48.304300    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7180ebbf-2671-4778-81a7-57a8aae4a783-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7180ebbf-2671-4778-81a7-57a8aae4a783-istiod-ca-cert\") pod \"7180ebbf-2671-4778-81a7-57a8aae4a783\" (UID: \"7180ebbf-2671-4778-81a7-57a8aae4a783\") "
Oct 30 20:18:48.304597 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:48.304424    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7180ebbf-2671-4778-81a7-57a8aae4a783-kube-api-access-cpkjg\" (UniqueName: \"kubernetes.io/projected/7180ebbf-2671-4778-81a7-57a8aae4a783-kube-api-access-cpkjg\") pod \"7180ebbf-2671-4778-81a7-57a8aae4a783\" (UID: \"7180ebbf-2671-4778-81a7-57a8aae4a783\") "
Oct 30 20:18:48.304597 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:48.304455    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7180ebbf-2671-4778-81a7-57a8aae4a783-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7180ebbf-2671-4778-81a7-57a8aae4a783-istio-data\") pod \"7180ebbf-2671-4778-81a7-57a8aae4a783\" (UID: \"7180ebbf-2671-4778-81a7-57a8aae4a783\") "
Oct 30 20:18:48.304597 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:48.304547    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7180ebbf-2671-4778-81a7-57a8aae4a783-istio-token\" (UniqueName: \"kubernetes.io/projected/7180ebbf-2671-4778-81a7-57a8aae4a783-istio-token\") pod \"7180ebbf-2671-4778-81a7-57a8aae4a783\" (UID: \"7180ebbf-2671-4778-81a7-57a8aae4a783\") "
Oct 30 20:18:48.304597 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:48.304553    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7180ebbf-2671-4778-81a7-57a8aae4a783-workload-certs" pod "7180ebbf-2671-4778-81a7-57a8aae4a783" (UID: "7180ebbf-2671-4778-81a7-57a8aae4a783"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:18:48.304597 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:48.304588    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7180ebbf-2671-4778-81a7-57a8aae4a783-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7180ebbf-2671-4778-81a7-57a8aae4a783-credential-socket\") pod \"7180ebbf-2671-4778-81a7-57a8aae4a783\" (UID: \"7180ebbf-2671-4778-81a7-57a8aae4a783\") "
Oct 30 20:18:48.304597 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:48.304560    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7180ebbf-2671-4778-81a7-57a8aae4a783-workload-socket" pod "7180ebbf-2671-4778-81a7-57a8aae4a783" (UID: "7180ebbf-2671-4778-81a7-57a8aae4a783"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:18:48.304932 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:48.304773    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7180ebbf-2671-4778-81a7-57a8aae4a783-istio-data" pod "7180ebbf-2671-4778-81a7-57a8aae4a783" (UID: "7180ebbf-2671-4778-81a7-57a8aae4a783"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:18:48.304932 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:48.304794    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7180ebbf-2671-4778-81a7-57a8aae4a783-istiod-ca-cert" pod "7180ebbf-2671-4778-81a7-57a8aae4a783" (UID: "7180ebbf-2671-4778-81a7-57a8aae4a783"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:18:48.305049 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:48.304948    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7180ebbf-2671-4778-81a7-57a8aae4a783-credential-socket" pod "7180ebbf-2671-4778-81a7-57a8aae4a783" (UID: "7180ebbf-2671-4778-81a7-57a8aae4a783"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:18:48.305153 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:48.305134    6996 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7180ebbf-2671-4778-81a7-57a8aae4a783-workload-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:48.305221 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:48.305153    6996 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7180ebbf-2671-4778-81a7-57a8aae4a783-workload-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:48.305221 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:48.305163    6996 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7180ebbf-2671-4778-81a7-57a8aae4a783-istiod-ca-cert\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:48.305221 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:48.305172    6996 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7180ebbf-2671-4778-81a7-57a8aae4a783-istio-data\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:48.305221 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:48.305181    6996 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7180ebbf-2671-4778-81a7-57a8aae4a783-credential-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:48.306610 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:48.306591    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7180ebbf-2671-4778-81a7-57a8aae4a783-istio-envoy" pod "7180ebbf-2671-4778-81a7-57a8aae4a783" (UID: "7180ebbf-2671-4778-81a7-57a8aae4a783"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:18:48.306791 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:48.306772    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7180ebbf-2671-4778-81a7-57a8aae4a783-istio-podinfo" pod "7180ebbf-2671-4778-81a7-57a8aae4a783" (UID: "7180ebbf-2671-4778-81a7-57a8aae4a783"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 20:18:48.306855 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:48.306784    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7180ebbf-2671-4778-81a7-57a8aae4a783-istio-token" pod "7180ebbf-2671-4778-81a7-57a8aae4a783" (UID: "7180ebbf-2671-4778-81a7-57a8aae4a783"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:18:48.306855 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:48.306818    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7180ebbf-2671-4778-81a7-57a8aae4a783-kube-api-access-cpkjg" pod "7180ebbf-2671-4778-81a7-57a8aae4a783" (UID: "7180ebbf-2671-4778-81a7-57a8aae4a783"). InnerVolumeSpecName "kube-api-access-cpkjg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:18:48.406950 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:48.406917    6996 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7180ebbf-2671-4778-81a7-57a8aae4a783-istio-token\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:48.406950 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:48.406946    6996 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7180ebbf-2671-4778-81a7-57a8aae4a783-istio-podinfo\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:48.407142 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:48.406959    6996 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7180ebbf-2671-4778-81a7-57a8aae4a783-istio-envoy\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:48.407142 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:48.406969    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cpkjg\" (UniqueName: \"kubernetes.io/projected/7180ebbf-2671-4778-81a7-57a8aae4a783-kube-api-access-cpkjg\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:18:48.795630 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:48.795594    6996 generic.go:356] "Generic (PLEG): container finished" podID="7180ebbf-2671-4778-81a7-57a8aae4a783" containerID="bff4c9621e743d156397f4765bc0b85e01dd4b592bc26e11ec2f83a706173f90" exitCode=0
Oct 30 20:18:48.796098 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:48.795639    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc"
Oct 30 20:18:48.796098 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:48.795668    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc" event={"ID":"7180ebbf-2671-4778-81a7-57a8aae4a783","Type":"ContainerDied","Data":"bff4c9621e743d156397f4765bc0b85e01dd4b592bc26e11ec2f83a706173f90"}
Oct 30 20:18:48.796098 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:48.795716    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc" event={"ID":"7180ebbf-2671-4778-81a7-57a8aae4a783","Type":"ContainerDied","Data":"44f22fc183cd425c089f4c00fb7f833b527d0c4a8f8cafa333f540bd584c6d70"}
Oct 30 20:18:48.796098 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:48.795739    6996 scope.go:122] "RemoveContainer" containerID="bff4c9621e743d156397f4765bc0b85e01dd4b592bc26e11ec2f83a706173f90"
Oct 30 20:18:48.806828 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:48.806811    6996 scope.go:122] "RemoveContainer" containerID="bff4c9621e743d156397f4765bc0b85e01dd4b592bc26e11ec2f83a706173f90"
Oct 30 20:18:48.807090 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:18:48.807071    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bff4c9621e743d156397f4765bc0b85e01dd4b592bc26e11ec2f83a706173f90\": container with ID starting with bff4c9621e743d156397f4765bc0b85e01dd4b592bc26e11ec2f83a706173f90 not found: ID does not exist" containerID="bff4c9621e743d156397f4765bc0b85e01dd4b592bc26e11ec2f83a706173f90"
Oct 30 20:18:48.807145 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:48.807098    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bff4c9621e743d156397f4765bc0b85e01dd4b592bc26e11ec2f83a706173f90"} err="failed to get container status \"bff4c9621e743d156397f4765bc0b85e01dd4b592bc26e11ec2f83a706173f90\": rpc error: code = NotFound desc = could not find container \"bff4c9621e743d156397f4765bc0b85e01dd4b592bc26e11ec2f83a706173f90\": container with ID starting with bff4c9621e743d156397f4765bc0b85e01dd4b592bc26e11ec2f83a706173f90 not found: ID does not exist"
Oct 30 20:18:48.821736 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:48.821710    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc"]
Oct 30 20:18:48.826805 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:48.826782    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-2a3e1b-openshift-default-679cc6f7f7-kpkwc"]
Oct 30 20:18:50.793386 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:18:50.793353    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7180ebbf-2671-4778-81a7-57a8aae4a783" path="/var/lib/kubelet/pods/7180ebbf-2671-4778-81a7-57a8aae4a783/volumes"
Oct 30 20:19:01.191592 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:19:01.191547    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 20:19:08.339455 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:19:08.339410    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 20:19:09.234239 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:19:09.234198    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-bdf9e6/test-model-bdf9e6-kserve-6bbdfd7fb7-szfjr"]
Oct 30 20:19:09.235162 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:19:09.235133    6996 cpu_manager.go:414] "containerMap: removing container" podUID="7180ebbf-2671-4778-81a7-57a8aae4a783" containerName="istio-proxy"
Oct 30 20:19:09.235162 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:19:09.235158    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7180ebbf-2671-4778-81a7-57a8aae4a783" containerName="istio-proxy"
Oct 30 20:19:09.235330 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:19:09.235201    6996 cpu_manager.go:414] "containerMap: removing container" podUID="e4dc9cd9-2754-4e38-af83-046f8536a53b" containerName="istio-proxy"
Oct 30 20:19:09.235330 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:19:09.235207    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e4dc9cd9-2754-4e38-af83-046f8536a53b" containerName="istio-proxy"
Oct 30 20:19:09.235330 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:19:09.235223    6996 cpu_manager.go:414] "containerMap: removing container" podUID="e41df835-fccd-45bb-bfca-ca1f81f6261a" containerName="istio-proxy"
Oct 30 20:19:09.235330 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:19:09.235228    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e41df835-fccd-45bb-bfca-ca1f81f6261a" containerName="istio-proxy"
Oct 30 20:19:09.235330 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:19:09.235303    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="e4dc9cd9-2754-4e38-af83-046f8536a53b" containerName="istio-proxy"
Oct 30 20:19:09.235330 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:19:09.235314    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="e41df835-fccd-45bb-bfca-ca1f81f6261a" containerName="istio-proxy"
Oct 30 20:19:09.235330 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:19:09.235321    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="7180ebbf-2671-4778-81a7-57a8aae4a783" containerName="istio-proxy"
Oct 30 20:19:09.240471 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:19:09.240443    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-bdf9e6/test-model-bdf9e6-kserve-6bbdfd7fb7-szfjr"
Oct 30 20:19:09.250661 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:19:09.250634    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-bdf9e6\"/\"kube-root-ca.crt\""
Oct 30 20:19:09.250789 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:19:09.250633    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-bdf9e6\"/\"openshift-service-ca.crt\""
Oct 30 20:19:09.250789 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:19:09.250739    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-bdf9e6\"/\"default-dockercfg-dc8hw\""
Oct 30 20:19:09.250896 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:19:09.250795    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-bdf9e6\"/\"test-model-bdf9e6-kserve-self-signed-certs\""
Oct 30 20:19:09.253335 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:19:09.253315    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-bdf9e6/test-model-bdf9e6-kserve-6bbdfd7fb7-szfjr"]
Oct 30 20:19:09.384526 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:19:09.384469    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxw7q\" (UniqueName: \"kubernetes.io/projected/2eb68260-155c-4f10-84ec-0411e8fd0adb-kube-api-access-pxw7q\") pod \"test-model-bdf9e6-kserve-6bbdfd7fb7-szfjr\" (UID: \"2eb68260-155c-4f10-84ec-0411e8fd0adb\") " pod="ai-tenant-e2e-inf-b-bdf9e6/test-model-bdf9e6-kserve-6bbdfd7fb7-szfjr"
Oct 30 20:19:09.384980 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:19:09.384676    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2eb68260-155c-4f10-84ec-0411e8fd0adb-home\") pod \"test-model-bdf9e6-kserve-6bbdfd7fb7-szfjr\" (UID: \"2eb68260-155c-4f10-84ec-0411e8fd0adb\") " pod="ai-tenant-e2e-inf-b-bdf9e6/test-model-bdf9e6-kserve-6bbdfd7fb7-szfjr"
Oct 30 20:19:09.384980 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:19:09.384719    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2eb68260-155c-4f10-84ec-0411e8fd0adb-tls-certs\") pod \"test-model-bdf9e6-kserve-6bbdfd7fb7-szfjr\" (UID: \"2eb68260-155c-4f10-84ec-0411e8fd0adb\") " pod="ai-tenant-e2e-inf-b-bdf9e6/test-model-bdf9e6-kserve-6bbdfd7fb7-szfjr"
Oct 30 20:19:09.384980 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:19:09.384910    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2eb68260-155c-4f10-84ec-0411e8fd0adb-model-cache\") pod \"test-model-bdf9e6-kserve-6bbdfd7fb7-szfjr\" (UID: \"2eb68260-155c-4f10-84ec-0411e8fd0adb\") " pod="ai-tenant-e2e-inf-b-bdf9e6/test-model-bdf9e6-kserve-6bbdfd7fb7-szfjr"
Oct 30 20:19:09.384980 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:19:09.384968    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2eb68260-155c-4f10-84ec-0411e8fd0adb-dshm\") pod \"test-model-bdf9e6-kserve-6bbdfd7fb7-szfjr\" (UID: \"2eb68260-155c-4f10-84ec-0411e8fd0adb\") " pod="ai-tenant-e2e-inf-b-bdf9e6/test-model-bdf9e6-kserve-6bbdfd7fb7-szfjr"
Oct 30 20:19:09.385198 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:19:09.385038    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2eb68260-155c-4f10-84ec-0411e8fd0adb-tmp-dir\") pod \"test-model-bdf9e6-kserve-6bbdfd7fb7-szfjr\" (UID: \"2eb68260-155c-4f10-84ec-0411e8fd0adb\") " pod="ai-tenant-e2e-inf-b-bdf9e6/test-model-bdf9e6-kserve-6bbdfd7fb7-szfjr"
Oct 30 20:19:09.488548 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:19:09.487521    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2eb68260-155c-4f10-84ec-0411e8fd0adb-home\") pod \"test-model-bdf9e6-kserve-6bbdfd7fb7-szfjr\" (UID: \"2eb68260-155c-4f10-84ec-0411e8fd0adb\") " pod="ai-tenant-e2e-inf-b-bdf9e6/test-model-bdf9e6-kserve-6bbdfd7fb7-szfjr"
Oct 30 20:19:09.488548 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:19:09.487572    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2eb68260-155c-4f10-84ec-0411e8fd0adb-tls-certs\") pod \"test-model-bdf9e6-kserve-6bbdfd7fb7-szfjr\" (UID: \"2eb68260-155c-4f10-84ec-0411e8fd0adb\") " pod="ai-tenant-e2e-inf-b-bdf9e6/test-model-bdf9e6-kserve-6bbdfd7fb7-szfjr"
Oct 30 20:19:09.488548 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:19:09.487695    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2eb68260-155c-4f10-84ec-0411e8fd0adb-model-cache\") pod \"test-model-bdf9e6-kserve-6bbdfd7fb7-szfjr\" (UID: \"2eb68260-155c-4f10-84ec-0411e8fd0adb\") " pod="ai-tenant-e2e-inf-b-bdf9e6/test-model-bdf9e6-kserve-6bbdfd7fb7-szfjr"
Oct 30 20:19:09.488548 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:19:09.487741    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2eb68260-155c-4f10-84ec-0411e8fd0adb-dshm\") pod \"test-model-bdf9e6-kserve-6bbdfd7fb7-szfjr\" (UID: \"2eb68260-155c-4f10-84ec-0411e8fd0adb\") " pod="ai-tenant-e2e-inf-b-bdf9e6/test-model-bdf9e6-kserve-6bbdfd7fb7-szfjr"
Oct 30 20:19:09.488548 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:19:09.487781    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2eb68260-155c-4f10-84ec-0411e8fd0adb-tmp-dir\") pod \"test-model-bdf9e6-kserve-6bbdfd7fb7-szfjr\" (UID: \"2eb68260-155c-4f10-84ec-0411e8fd0adb\") " pod="ai-tenant-e2e-inf-b-bdf9e6/test-model-bdf9e6-kserve-6bbdfd7fb7-szfjr"
Oct 30 20:19:09.488548 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:19:09.487937    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxw7q\" (UniqueName: \"kubernetes.io/projected/2eb68260-155c-4f10-84ec-0411e8fd0adb-kube-api-access-pxw7q\") pod \"test-model-bdf9e6-kserve-6bbdfd7fb7-szfjr\" (UID: \"2eb68260-155c-4f10-84ec-0411e8fd0adb\") " pod="ai-tenant-e2e-inf-b-bdf9e6/test-model-bdf9e6-kserve-6bbdfd7fb7-szfjr"
Oct 30 20:19:09.488548 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:19:09.488539    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2eb68260-155c-4f10-84ec-0411e8fd0adb-model-cache\") pod \"test-model-bdf9e6-kserve-6bbdfd7fb7-szfjr\" (UID: \"2eb68260-155c-4f10-84ec-0411e8fd0adb\") " pod="ai-tenant-e2e-inf-b-bdf9e6/test-model-bdf9e6-kserve-6bbdfd7fb7-szfjr"
Oct 30 20:19:09.489130 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:19:09.489099    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2eb68260-155c-4f10-84ec-0411e8fd0adb-tmp-dir\") pod \"test-model-bdf9e6-kserve-6bbdfd7fb7-szfjr\" (UID: \"2eb68260-155c-4f10-84ec-0411e8fd0adb\") " pod="ai-tenant-e2e-inf-b-bdf9e6/test-model-bdf9e6-kserve-6bbdfd7fb7-szfjr"
Oct 30 20:19:09.496873 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:19:09.496843    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2eb68260-155c-4f10-84ec-0411e8fd0adb-home\") pod \"test-model-bdf9e6-kserve-6bbdfd7fb7-szfjr\" (UID: \"2eb68260-155c-4f10-84ec-0411e8fd0adb\") " pod="ai-tenant-e2e-inf-b-bdf9e6/test-model-bdf9e6-kserve-6bbdfd7fb7-szfjr"
Oct 30 20:19:09.514533 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:19:09.500180    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2eb68260-155c-4f10-84ec-0411e8fd0adb-tls-certs\") pod \"test-model-bdf9e6-kserve-6bbdfd7fb7-szfjr\" (UID: \"2eb68260-155c-4f10-84ec-0411e8fd0adb\") " pod="ai-tenant-e2e-inf-b-bdf9e6/test-model-bdf9e6-kserve-6bbdfd7fb7-szfjr"
Oct 30 20:19:09.514533 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:19:09.500460    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2eb68260-155c-4f10-84ec-0411e8fd0adb-dshm\") pod \"test-model-bdf9e6-kserve-6bbdfd7fb7-szfjr\" (UID: \"2eb68260-155c-4f10-84ec-0411e8fd0adb\") " pod="ai-tenant-e2e-inf-b-bdf9e6/test-model-bdf9e6-kserve-6bbdfd7fb7-szfjr"
Oct 30 20:19:09.514533 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:19:09.508092    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxw7q\" (UniqueName: \"kubernetes.io/projected/2eb68260-155c-4f10-84ec-0411e8fd0adb-kube-api-access-pxw7q\") pod \"test-model-bdf9e6-kserve-6bbdfd7fb7-szfjr\" (UID: \"2eb68260-155c-4f10-84ec-0411e8fd0adb\") " pod="ai-tenant-e2e-inf-b-bdf9e6/test-model-bdf9e6-kserve-6bbdfd7fb7-szfjr"
Oct 30 20:19:09.556074 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:19:09.556033    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-bdf9e6/test-model-bdf9e6-kserve-6bbdfd7fb7-szfjr"
Oct 30 20:19:09.892150 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:19:09.892119    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-bdf9e6/test-model-bdf9e6-kserve-6bbdfd7fb7-szfjr"]
Oct 30 20:19:09.893949 ip-10-0-141-42 kubenswrapper[6996]: W1030 20:19:09.893919    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2eb68260_155c_4f10_84ec_0411e8fd0adb.slice/crio-0fd3568e5535766cbe27b537b74a1b5c27e1b5a4ea191338bf9f59550b4a1e7a WatchSource:0}: Error finding container 0fd3568e5535766cbe27b537b74a1b5c27e1b5a4ea191338bf9f59550b4a1e7a: Status 404 returned error can't find the container with id 0fd3568e5535766cbe27b537b74a1b5c27e1b5a4ea191338bf9f59550b4a1e7a
Oct 30 20:19:09.951054 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:19:09.951021    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-bdf9e6/test-model-bdf9e6-kserve-6bbdfd7fb7-szfjr" event={"ID":"2eb68260-155c-4f10-84ec-0411e8fd0adb","Type":"ContainerStarted","Data":"0fd3568e5535766cbe27b537b74a1b5c27e1b5a4ea191338bf9f59550b4a1e7a"}
Oct 30 20:19:10.959464 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:19:10.959428    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-bdf9e6/test-model-bdf9e6-kserve-6bbdfd7fb7-szfjr" event={"ID":"2eb68260-155c-4f10-84ec-0411e8fd0adb","Type":"ContainerStarted","Data":"fa0da3fe08a83dab5b9f400da642de792b60ba9971ce98857c33c799ec279e29"}
Oct 30 20:19:10.982327 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:19:10.982281    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-bdf9e6/test-model-bdf9e6-kserve-6bbdfd7fb7-szfjr" podStartSLOduration=1.982268628 podStartE2EDuration="1.982268628s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:19:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:19:10.980903113 +0000 UTC m=+2826.309021268" watchObservedRunningTime="2026-10-30 20:19:10.982268628 +0000 UTC m=+2826.310386819"
Oct 30 20:19:11.556205 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:19:11.556168    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-bdf9e6/test-model-bdf9e6-kserve-6bbdfd7fb7-szfjr"
Oct 30 20:19:19.558055 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:19:19.558017    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-bdf9e6/test-model-bdf9e6-kserve-6bbdfd7fb7-szfjr"
Oct 30 20:19:19.574556 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:19:19.574525    6996 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-bdf9e6/test-model-bdf9e6-kserve-6bbdfd7fb7-szfjr"
Oct 30 20:19:20.031457 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:19:20.031417    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-bdf9e6/test-model-bdf9e6-kserve-6bbdfd7fb7-szfjr"
Oct 30 20:19:31.191741 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:19:31.191704    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 20:19:37.551507 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:19:37.551436    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 20:19:55.593442 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:19:55.593400    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 20:20:01.213231 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:20:01.213184    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 20:20:18.684952 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:20:18.684817    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 20:20:31.193106 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:20:31.193066    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 20:20:39.576396 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:20:39.576360    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 20:20:46.121995 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:20:46.121956    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-f63d20-7bdf8df68b-clm5w"]
Oct 30 20:20:46.122614 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:20:46.122241    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f63d20-7bdf8df68b-clm5w" podUID="d1d1de99-5578-4b8a-b94d-c8c4a095d760" containerName="maas-api" containerID="cri-o://40041b3b73f16d71ce6ee630d3555932a30bdd2c62fc5b8dfac6c75fae446034" gracePeriod=30
Oct 30 20:20:46.602652 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:20:46.602614    6996 generic.go:356] "Generic (PLEG): container finished" podID="d1d1de99-5578-4b8a-b94d-c8c4a095d760" containerID="40041b3b73f16d71ce6ee630d3555932a30bdd2c62fc5b8dfac6c75fae446034" exitCode=0
Oct 30 20:20:46.602894 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:20:46.602692    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f63d20-7bdf8df68b-clm5w" event={"ID":"d1d1de99-5578-4b8a-b94d-c8c4a095d760","Type":"ContainerDied","Data":"40041b3b73f16d71ce6ee630d3555932a30bdd2c62fc5b8dfac6c75fae446034"}
Oct 30 20:20:46.602894 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:20:46.602737    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f63d20-7bdf8df68b-clm5w" event={"ID":"d1d1de99-5578-4b8a-b94d-c8c4a095d760","Type":"ContainerDied","Data":"99c7173122f8241c8c41cfa82eab150d14f1b0716cccf73848210127bd7f96ea"}
Oct 30 20:20:46.602894 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:20:46.602762    6996 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="99c7173122f8241c8c41cfa82eab150d14f1b0716cccf73848210127bd7f96ea"
Oct 30 20:20:46.604080 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:20:46.604049    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f63d20-7bdf8df68b-clm5w"
Oct 30 20:20:46.757205 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:20:46.757122    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d1d1de99-5578-4b8a-b94d-c8c4a095d760-kube-api-access-vpt76\" (UniqueName: \"kubernetes.io/projected/d1d1de99-5578-4b8a-b94d-c8c4a095d760-kube-api-access-vpt76\") pod \"d1d1de99-5578-4b8a-b94d-c8c4a095d760\" (UID: \"d1d1de99-5578-4b8a-b94d-c8c4a095d760\") "
Oct 30 20:20:46.757205 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:20:46.757159    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d1d1de99-5578-4b8a-b94d-c8c4a095d760-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d1d1de99-5578-4b8a-b94d-c8c4a095d760-maas-api-tls\") pod \"d1d1de99-5578-4b8a-b94d-c8c4a095d760\" (UID: \"d1d1de99-5578-4b8a-b94d-c8c4a095d760\") "
Oct 30 20:20:46.757205 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:20:46.757190    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d1d1de99-5578-4b8a-b94d-c8c4a095d760-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1d1de99-5578-4b8a-b94d-c8c4a095d760-openshift-service-ca\") pod \"d1d1de99-5578-4b8a-b94d-c8c4a095d760\" (UID: \"d1d1de99-5578-4b8a-b94d-c8c4a095d760\") "
Oct 30 20:20:46.757463 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:20:46.757213    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d1d1de99-5578-4b8a-b94d-c8c4a095d760-metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1d1de99-5578-4b8a-b94d-c8c4a095d760-metrics-certs\") pod \"d1d1de99-5578-4b8a-b94d-c8c4a095d760\" (UID: \"d1d1de99-5578-4b8a-b94d-c8c4a095d760\") "
Oct 30 20:20:46.757680 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:20:46.757647    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1d1de99-5578-4b8a-b94d-c8c4a095d760-openshift-service-ca" pod "d1d1de99-5578-4b8a-b94d-c8c4a095d760" (UID: "d1d1de99-5578-4b8a-b94d-c8c4a095d760"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:20:46.757944 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:20:46.757925    6996 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1d1de99-5578-4b8a-b94d-c8c4a095d760-openshift-service-ca\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:20:46.759412 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:20:46.759385    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1d1de99-5578-4b8a-b94d-c8c4a095d760-maas-api-tls" pod "d1d1de99-5578-4b8a-b94d-c8c4a095d760" (UID: "d1d1de99-5578-4b8a-b94d-c8c4a095d760"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:20:46.759552 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:20:46.759417    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1d1de99-5578-4b8a-b94d-c8c4a095d760-kube-api-access-vpt76" pod "d1d1de99-5578-4b8a-b94d-c8c4a095d760" (UID: "d1d1de99-5578-4b8a-b94d-c8c4a095d760"). InnerVolumeSpecName "kube-api-access-vpt76". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:20:46.759552 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:20:46.759433    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1d1de99-5578-4b8a-b94d-c8c4a095d760-metrics-certs" pod "d1d1de99-5578-4b8a-b94d-c8c4a095d760" (UID: "d1d1de99-5578-4b8a-b94d-c8c4a095d760"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:20:46.859046 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:20:46.859009    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vpt76\" (UniqueName: \"kubernetes.io/projected/d1d1de99-5578-4b8a-b94d-c8c4a095d760-kube-api-access-vpt76\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:20:46.859046 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:20:46.859040    6996 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d1d1de99-5578-4b8a-b94d-c8c4a095d760-maas-api-tls\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:20:46.859046 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:20:46.859049    6996 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1d1de99-5578-4b8a-b94d-c8c4a095d760-metrics-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:20:47.610093 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:20:47.610057    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f63d20-7bdf8df68b-clm5w"
Oct 30 20:20:47.632568 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:20:47.632533    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-f63d20-7bdf8df68b-clm5w"]
Oct 30 20:20:47.635162 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:20:47.635133    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-f63d20-7bdf8df68b-clm5w"]
Oct 30 20:20:48.791248 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:20:48.791214    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d1d1de99-5578-4b8a-b94d-c8c4a095d760" path="/var/lib/kubelet/pods/d1d1de99-5578-4b8a-b94d-c8c4a095d760/volumes"
Oct 30 20:20:55.170931 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:20:55.170871    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp"]
Oct 30 20:20:55.171428 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:20:55.171188    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp" podUID="a21bbe95-547d-41b0-b7c8-e1a296545947" containerName="istio-proxy" containerID="cri-o://7d73432516dfe591caae8bf776dadb8a4926e3847b29ae549e7e6f47989e1edd" gracePeriod=30
Oct 30 20:20:56.316657 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:20:56.316615    6996 patch_prober.go:29] interesting pod/e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 20:20:56.317138 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:20:56.316693    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp" podUID="a21bbe95-547d-41b0-b7c8-e1a296545947" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 20:21:00.453791 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:00.453703    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp"
Oct 30 20:21:00.552121 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:00.552080    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a21bbe95-547d-41b0-b7c8-e1a296545947-kube-api-access-xg6w4\" (UniqueName: \"kubernetes.io/projected/a21bbe95-547d-41b0-b7c8-e1a296545947-kube-api-access-xg6w4\") pod \"a21bbe95-547d-41b0-b7c8-e1a296545947\" (UID: \"a21bbe95-547d-41b0-b7c8-e1a296545947\") "
Oct 30 20:21:00.552304 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:00.552162    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a21bbe95-547d-41b0-b7c8-e1a296545947-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a21bbe95-547d-41b0-b7c8-e1a296545947-istio-podinfo\") pod \"a21bbe95-547d-41b0-b7c8-e1a296545947\" (UID: \"a21bbe95-547d-41b0-b7c8-e1a296545947\") "
Oct 30 20:21:00.552304 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:00.552237    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a21bbe95-547d-41b0-b7c8-e1a296545947-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a21bbe95-547d-41b0-b7c8-e1a296545947-workload-certs\") pod \"a21bbe95-547d-41b0-b7c8-e1a296545947\" (UID: \"a21bbe95-547d-41b0-b7c8-e1a296545947\") "
Oct 30 20:21:00.552404 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:00.552325    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a21bbe95-547d-41b0-b7c8-e1a296545947-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a21bbe95-547d-41b0-b7c8-e1a296545947-istio-data\") pod \"a21bbe95-547d-41b0-b7c8-e1a296545947\" (UID: \"a21bbe95-547d-41b0-b7c8-e1a296545947\") "
Oct 30 20:21:00.552472 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:00.552406    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a21bbe95-547d-41b0-b7c8-e1a296545947-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a21bbe95-547d-41b0-b7c8-e1a296545947-istio-envoy\") pod \"a21bbe95-547d-41b0-b7c8-e1a296545947\" (UID: \"a21bbe95-547d-41b0-b7c8-e1a296545947\") "
Oct 30 20:21:00.552559 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:00.552524    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a21bbe95-547d-41b0-b7c8-e1a296545947-workload-certs" pod "a21bbe95-547d-41b0-b7c8-e1a296545947" (UID: "a21bbe95-547d-41b0-b7c8-e1a296545947"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:21:00.552620 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:00.552556    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a21bbe95-547d-41b0-b7c8-e1a296545947-istio-token\" (UniqueName: \"kubernetes.io/projected/a21bbe95-547d-41b0-b7c8-e1a296545947-istio-token\") pod \"a21bbe95-547d-41b0-b7c8-e1a296545947\" (UID: \"a21bbe95-547d-41b0-b7c8-e1a296545947\") "
Oct 30 20:21:00.552620 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:00.552587    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a21bbe95-547d-41b0-b7c8-e1a296545947-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a21bbe95-547d-41b0-b7c8-e1a296545947-credential-socket\") pod \"a21bbe95-547d-41b0-b7c8-e1a296545947\" (UID: \"a21bbe95-547d-41b0-b7c8-e1a296545947\") "
Oct 30 20:21:00.552736 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:00.552613    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a21bbe95-547d-41b0-b7c8-e1a296545947-istio-data" pod "a21bbe95-547d-41b0-b7c8-e1a296545947" (UID: "a21bbe95-547d-41b0-b7c8-e1a296545947"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:21:00.552736 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:00.552642    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a21bbe95-547d-41b0-b7c8-e1a296545947-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a21bbe95-547d-41b0-b7c8-e1a296545947-workload-socket\") pod \"a21bbe95-547d-41b0-b7c8-e1a296545947\" (UID: \"a21bbe95-547d-41b0-b7c8-e1a296545947\") "
Oct 30 20:21:00.552736 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:00.552671    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a21bbe95-547d-41b0-b7c8-e1a296545947-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a21bbe95-547d-41b0-b7c8-e1a296545947-istiod-ca-cert\") pod \"a21bbe95-547d-41b0-b7c8-e1a296545947\" (UID: \"a21bbe95-547d-41b0-b7c8-e1a296545947\") "
Oct 30 20:21:00.552896 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:00.552843    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a21bbe95-547d-41b0-b7c8-e1a296545947-credential-socket" pod "a21bbe95-547d-41b0-b7c8-e1a296545947" (UID: "a21bbe95-547d-41b0-b7c8-e1a296545947"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:21:00.553061 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:00.553030    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a21bbe95-547d-41b0-b7c8-e1a296545947-workload-socket" pod "a21bbe95-547d-41b0-b7c8-e1a296545947" (UID: "a21bbe95-547d-41b0-b7c8-e1a296545947"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:21:00.553165 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:00.553146    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a21bbe95-547d-41b0-b7c8-e1a296545947-istiod-ca-cert" pod "a21bbe95-547d-41b0-b7c8-e1a296545947" (UID: "a21bbe95-547d-41b0-b7c8-e1a296545947"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:21:00.553274 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:00.553256    6996 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a21bbe95-547d-41b0-b7c8-e1a296545947-credential-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:21:00.553336 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:00.553281    6996 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a21bbe95-547d-41b0-b7c8-e1a296545947-workload-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:21:00.553336 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:00.553295    6996 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a21bbe95-547d-41b0-b7c8-e1a296545947-workload-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:21:00.553336 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:00.553308    6996 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a21bbe95-547d-41b0-b7c8-e1a296545947-istio-data\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:21:00.554856 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:00.554830    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a21bbe95-547d-41b0-b7c8-e1a296545947-istio-podinfo" pod "a21bbe95-547d-41b0-b7c8-e1a296545947" (UID: "a21bbe95-547d-41b0-b7c8-e1a296545947"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 20:21:00.554983 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:00.554914    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a21bbe95-547d-41b0-b7c8-e1a296545947-istio-token" pod "a21bbe95-547d-41b0-b7c8-e1a296545947" (UID: "a21bbe95-547d-41b0-b7c8-e1a296545947"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:21:00.554983 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:00.554935    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a21bbe95-547d-41b0-b7c8-e1a296545947-kube-api-access-xg6w4" pod "a21bbe95-547d-41b0-b7c8-e1a296545947" (UID: "a21bbe95-547d-41b0-b7c8-e1a296545947"). InnerVolumeSpecName "kube-api-access-xg6w4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:21:00.555101 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:00.555078    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a21bbe95-547d-41b0-b7c8-e1a296545947-istio-envoy" pod "a21bbe95-547d-41b0-b7c8-e1a296545947" (UID: "a21bbe95-547d-41b0-b7c8-e1a296545947"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:21:00.604762 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:00.604718    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 20:21:00.654454 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:00.654400    6996 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a21bbe95-547d-41b0-b7c8-e1a296545947-istio-envoy\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:21:00.654454 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:00.654441    6996 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a21bbe95-547d-41b0-b7c8-e1a296545947-istio-token\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:21:00.654454 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:00.654455    6996 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a21bbe95-547d-41b0-b7c8-e1a296545947-istiod-ca-cert\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:21:00.654789 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:00.654470    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xg6w4\" (UniqueName: \"kubernetes.io/projected/a21bbe95-547d-41b0-b7c8-e1a296545947-kube-api-access-xg6w4\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:21:00.654789 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:00.654506    6996 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a21bbe95-547d-41b0-b7c8-e1a296545947-istio-podinfo\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:21:00.695672 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:00.695630    6996 generic.go:356] "Generic (PLEG): container finished" podID="a21bbe95-547d-41b0-b7c8-e1a296545947" containerID="7d73432516dfe591caae8bf776dadb8a4926e3847b29ae549e7e6f47989e1edd" exitCode=0
Oct 30 20:21:00.695884 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:00.695773    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp" event={"ID":"a21bbe95-547d-41b0-b7c8-e1a296545947","Type":"ContainerDied","Data":"7d73432516dfe591caae8bf776dadb8a4926e3847b29ae549e7e6f47989e1edd"}
Oct 30 20:21:00.695884 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:00.695826    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp" event={"ID":"a21bbe95-547d-41b0-b7c8-e1a296545947","Type":"ContainerDied","Data":"261c442766844bf6b4c185098e498042bd7dfdf9df1d30385c844ca8e0632885"}
Oct 30 20:21:00.695884 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:00.695854    6996 scope.go:122] "RemoveContainer" containerID="7d73432516dfe591caae8bf776dadb8a4926e3847b29ae549e7e6f47989e1edd"
Oct 30 20:21:00.696061 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:00.696047    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp"
Oct 30 20:21:00.718718 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:00.718691    6996 scope.go:122] "RemoveContainer" containerID="a482c0cafd0058dd322aed5b2a574316076d71cae879579d39f173e60ab077ef"
Oct 30 20:21:00.726326 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:00.726290    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp"]
Oct 30 20:21:00.730147 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:00.730125    6996 scope.go:122] "RemoveContainer" containerID="7d73432516dfe591caae8bf776dadb8a4926e3847b29ae549e7e6f47989e1edd"
Oct 30 20:21:00.730539 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:21:00.730509    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d73432516dfe591caae8bf776dadb8a4926e3847b29ae549e7e6f47989e1edd\": container with ID starting with 7d73432516dfe591caae8bf776dadb8a4926e3847b29ae549e7e6f47989e1edd not found: ID does not exist" containerID="7d73432516dfe591caae8bf776dadb8a4926e3847b29ae549e7e6f47989e1edd"
Oct 30 20:21:00.730627 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:00.730548    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d73432516dfe591caae8bf776dadb8a4926e3847b29ae549e7e6f47989e1edd"} err="failed to get container status \"7d73432516dfe591caae8bf776dadb8a4926e3847b29ae549e7e6f47989e1edd\": rpc error: code = NotFound desc = could not find container \"7d73432516dfe591caae8bf776dadb8a4926e3847b29ae549e7e6f47989e1edd\": container with ID starting with 7d73432516dfe591caae8bf776dadb8a4926e3847b29ae549e7e6f47989e1edd not found: ID does not exist"
Oct 30 20:21:00.730627 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:00.730575    6996 scope.go:122] "RemoveContainer" containerID="a482c0cafd0058dd322aed5b2a574316076d71cae879579d39f173e60ab077ef"
Oct 30 20:21:00.734707 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:21:00.733792    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a482c0cafd0058dd322aed5b2a574316076d71cae879579d39f173e60ab077ef\": container with ID starting with a482c0cafd0058dd322aed5b2a574316076d71cae879579d39f173e60ab077ef not found: ID does not exist" containerID="a482c0cafd0058dd322aed5b2a574316076d71cae879579d39f173e60ab077ef"
Oct 30 20:21:00.734707 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:00.733833    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a482c0cafd0058dd322aed5b2a574316076d71cae879579d39f173e60ab077ef"} err="failed to get container status \"a482c0cafd0058dd322aed5b2a574316076d71cae879579d39f173e60ab077ef\": rpc error: code = NotFound desc = could not find container \"a482c0cafd0058dd322aed5b2a574316076d71cae879579d39f173e60ab077ef\": container with ID starting with a482c0cafd0058dd322aed5b2a574316076d71cae879579d39f173e60ab077ef not found: ID does not exist"
Oct 30 20:21:00.734707 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:00.734423    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-f63d20-openshift-default-6c69599f5c-dccsp"]
Oct 30 20:21:00.793678 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:00.793646    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a21bbe95-547d-41b0-b7c8-e1a296545947" path="/var/lib/kubelet/pods/a21bbe95-547d-41b0-b7c8-e1a296545947/volumes"
Oct 30 20:21:01.220230 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:21:01.220193    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 20:21:04.475173 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:04.475136    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-bdf9e6/test-model-bdf9e6-kserve-6bbdfd7fb7-szfjr"]
Oct 30 20:21:16.257421 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:16.257380    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c257e7-54b9f7d969-54pbk"]
Oct 30 20:21:16.258475 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:16.257654    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c257e7-54b9f7d969-54pbk" podUID="60867c87-cc28-4053-92d6-7f3f6024edc2" containerName="maas-api" containerID="cri-o://7e887c2e07cf97d28b7c5a610abf9cf98dc9202883b4a9b66c0b128f7e0e93c8" gracePeriod=30
Oct 30 20:21:16.580976 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:16.580942    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c257e7-54b9f7d969-54pbk"
Oct 30 20:21:16.681069 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:16.681029    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/60867c87-cc28-4053-92d6-7f3f6024edc2-kube-api-access-4xxw6\" (UniqueName: \"kubernetes.io/projected/60867c87-cc28-4053-92d6-7f3f6024edc2-kube-api-access-4xxw6\") pod \"60867c87-cc28-4053-92d6-7f3f6024edc2\" (UID: \"60867c87-cc28-4053-92d6-7f3f6024edc2\") "
Oct 30 20:21:16.681256 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:16.681078    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/60867c87-cc28-4053-92d6-7f3f6024edc2-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60867c87-cc28-4053-92d6-7f3f6024edc2-maas-api-tls\") pod \"60867c87-cc28-4053-92d6-7f3f6024edc2\" (UID: \"60867c87-cc28-4053-92d6-7f3f6024edc2\") "
Oct 30 20:21:16.681256 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:16.681105    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/60867c87-cc28-4053-92d6-7f3f6024edc2-metrics-certs\" (UniqueName: \"kubernetes.io/secret/60867c87-cc28-4053-92d6-7f3f6024edc2-metrics-certs\") pod \"60867c87-cc28-4053-92d6-7f3f6024edc2\" (UID: \"60867c87-cc28-4053-92d6-7f3f6024edc2\") "
Oct 30 20:21:16.681366 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:16.681270    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/60867c87-cc28-4053-92d6-7f3f6024edc2-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60867c87-cc28-4053-92d6-7f3f6024edc2-openshift-service-ca\") pod \"60867c87-cc28-4053-92d6-7f3f6024edc2\" (UID: \"60867c87-cc28-4053-92d6-7f3f6024edc2\") "
Oct 30 20:21:16.681748 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:16.681711    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60867c87-cc28-4053-92d6-7f3f6024edc2-openshift-service-ca" pod "60867c87-cc28-4053-92d6-7f3f6024edc2" (UID: "60867c87-cc28-4053-92d6-7f3f6024edc2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:21:16.683387 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:16.683356    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60867c87-cc28-4053-92d6-7f3f6024edc2-kube-api-access-4xxw6" pod "60867c87-cc28-4053-92d6-7f3f6024edc2" (UID: "60867c87-cc28-4053-92d6-7f3f6024edc2"). InnerVolumeSpecName "kube-api-access-4xxw6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:21:16.683543 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:16.683409    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60867c87-cc28-4053-92d6-7f3f6024edc2-metrics-certs" pod "60867c87-cc28-4053-92d6-7f3f6024edc2" (UID: "60867c87-cc28-4053-92d6-7f3f6024edc2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:21:16.683543 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:16.683420    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60867c87-cc28-4053-92d6-7f3f6024edc2-maas-api-tls" pod "60867c87-cc28-4053-92d6-7f3f6024edc2" (UID: "60867c87-cc28-4053-92d6-7f3f6024edc2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:21:16.782910 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:16.782812    6996 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60867c87-cc28-4053-92d6-7f3f6024edc2-openshift-service-ca\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:21:16.782910 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:16.782850    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4xxw6\" (UniqueName: \"kubernetes.io/projected/60867c87-cc28-4053-92d6-7f3f6024edc2-kube-api-access-4xxw6\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:21:16.782910 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:16.782865    6996 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60867c87-cc28-4053-92d6-7f3f6024edc2-maas-api-tls\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:21:16.782910 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:16.782873    6996 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60867c87-cc28-4053-92d6-7f3f6024edc2-metrics-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:21:16.802777 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:16.802742    6996 generic.go:356] "Generic (PLEG): container finished" podID="60867c87-cc28-4053-92d6-7f3f6024edc2" containerID="7e887c2e07cf97d28b7c5a610abf9cf98dc9202883b4a9b66c0b128f7e0e93c8" exitCode=0
Oct 30 20:21:16.802920 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:16.802807    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c257e7-54b9f7d969-54pbk"
Oct 30 20:21:16.802920 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:16.802805    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c257e7-54b9f7d969-54pbk" event={"ID":"60867c87-cc28-4053-92d6-7f3f6024edc2","Type":"ContainerDied","Data":"7e887c2e07cf97d28b7c5a610abf9cf98dc9202883b4a9b66c0b128f7e0e93c8"}
Oct 30 20:21:16.802996 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:16.802930    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c257e7-54b9f7d969-54pbk" event={"ID":"60867c87-cc28-4053-92d6-7f3f6024edc2","Type":"ContainerDied","Data":"3d754c414dbf3eaa19cbbd5126e2600aae065c7b32f55a68b903f247a383ba98"}
Oct 30 20:21:16.802996 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:16.802958    6996 scope.go:122] "RemoveContainer" containerID="7e887c2e07cf97d28b7c5a610abf9cf98dc9202883b4a9b66c0b128f7e0e93c8"
Oct 30 20:21:16.813382 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:16.813364    6996 scope.go:122] "RemoveContainer" containerID="7e887c2e07cf97d28b7c5a610abf9cf98dc9202883b4a9b66c0b128f7e0e93c8"
Oct 30 20:21:16.813709 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:21:16.813689    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e887c2e07cf97d28b7c5a610abf9cf98dc9202883b4a9b66c0b128f7e0e93c8\": container with ID starting with 7e887c2e07cf97d28b7c5a610abf9cf98dc9202883b4a9b66c0b128f7e0e93c8 not found: ID does not exist" containerID="7e887c2e07cf97d28b7c5a610abf9cf98dc9202883b4a9b66c0b128f7e0e93c8"
Oct 30 20:21:16.813786 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:16.813716    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e887c2e07cf97d28b7c5a610abf9cf98dc9202883b4a9b66c0b128f7e0e93c8"} err="failed to get container status \"7e887c2e07cf97d28b7c5a610abf9cf98dc9202883b4a9b66c0b128f7e0e93c8\": rpc error: code = NotFound desc = could not find container \"7e887c2e07cf97d28b7c5a610abf9cf98dc9202883b4a9b66c0b128f7e0e93c8\": container with ID starting with 7e887c2e07cf97d28b7c5a610abf9cf98dc9202883b4a9b66c0b128f7e0e93c8 not found: ID does not exist"
Oct 30 20:21:16.820493 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:16.820453    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c257e7-54b9f7d969-54pbk"]
Oct 30 20:21:16.823452 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:16.823427    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c257e7-54b9f7d969-54pbk"]
Oct 30 20:21:18.792660 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:18.792620    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="60867c87-cc28-4053-92d6-7f3f6024edc2" path="/var/lib/kubelet/pods/60867c87-cc28-4053-92d6-7f3f6024edc2/volumes"
Oct 30 20:21:19.496732 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:19.496681    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-bdf9e6/test-model-bdf9e6-kserve-6bbdfd7fb7-szfjr" podUID="2eb68260-155c-4f10-84ec-0411e8fd0adb" containerName="main" containerID="cri-o://fa0da3fe08a83dab5b9f400da642de792b60ba9971ce98857c33c799ec279e29" gracePeriod=45
Oct 30 20:21:19.768229 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:19.768202    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-bdf9e6/test-model-bdf9e6-kserve-6bbdfd7fb7-szfjr"
Oct 30 20:21:19.825663 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:19.825629    6996 generic.go:356] "Generic (PLEG): container finished" podID="2eb68260-155c-4f10-84ec-0411e8fd0adb" containerID="fa0da3fe08a83dab5b9f400da642de792b60ba9971ce98857c33c799ec279e29" exitCode=0
Oct 30 20:21:19.826179 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:19.825687    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-bdf9e6/test-model-bdf9e6-kserve-6bbdfd7fb7-szfjr" event={"ID":"2eb68260-155c-4f10-84ec-0411e8fd0adb","Type":"ContainerDied","Data":"fa0da3fe08a83dab5b9f400da642de792b60ba9971ce98857c33c799ec279e29"}
Oct 30 20:21:19.826179 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:19.825701    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-bdf9e6/test-model-bdf9e6-kserve-6bbdfd7fb7-szfjr"
Oct 30 20:21:19.826179 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:19.825723    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-bdf9e6/test-model-bdf9e6-kserve-6bbdfd7fb7-szfjr" event={"ID":"2eb68260-155c-4f10-84ec-0411e8fd0adb","Type":"ContainerDied","Data":"0fd3568e5535766cbe27b537b74a1b5c27e1b5a4ea191338bf9f59550b4a1e7a"}
Oct 30 20:21:19.826179 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:19.825746    6996 scope.go:122] "RemoveContainer" containerID="fa0da3fe08a83dab5b9f400da642de792b60ba9971ce98857c33c799ec279e29"
Oct 30 20:21:19.826473 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:19.826453    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2eb68260-155c-4f10-84ec-0411e8fd0adb-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2eb68260-155c-4f10-84ec-0411e8fd0adb-model-cache\") pod \"2eb68260-155c-4f10-84ec-0411e8fd0adb\" (UID: \"2eb68260-155c-4f10-84ec-0411e8fd0adb\") "
Oct 30 20:21:19.826574 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:19.826561    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2eb68260-155c-4f10-84ec-0411e8fd0adb-tls-certs\" (UniqueName: \"kubernetes.io/secret/2eb68260-155c-4f10-84ec-0411e8fd0adb-tls-certs\") pod \"2eb68260-155c-4f10-84ec-0411e8fd0adb\" (UID: \"2eb68260-155c-4f10-84ec-0411e8fd0adb\") "
Oct 30 20:21:19.826639 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:19.826590    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2eb68260-155c-4f10-84ec-0411e8fd0adb-kube-api-access-pxw7q\" (UniqueName: \"kubernetes.io/projected/2eb68260-155c-4f10-84ec-0411e8fd0adb-kube-api-access-pxw7q\") pod \"2eb68260-155c-4f10-84ec-0411e8fd0adb\" (UID: \"2eb68260-155c-4f10-84ec-0411e8fd0adb\") "
Oct 30 20:21:19.826639 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:19.826613    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2eb68260-155c-4f10-84ec-0411e8fd0adb-home\" (UniqueName: \"kubernetes.io/empty-dir/2eb68260-155c-4f10-84ec-0411e8fd0adb-home\") pod \"2eb68260-155c-4f10-84ec-0411e8fd0adb\" (UID: \"2eb68260-155c-4f10-84ec-0411e8fd0adb\") "
Oct 30 20:21:19.826740 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:19.826643    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2eb68260-155c-4f10-84ec-0411e8fd0adb-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2eb68260-155c-4f10-84ec-0411e8fd0adb-tmp-dir\") pod \"2eb68260-155c-4f10-84ec-0411e8fd0adb\" (UID: \"2eb68260-155c-4f10-84ec-0411e8fd0adb\") "
Oct 30 20:21:19.826798 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:19.826764    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2eb68260-155c-4f10-84ec-0411e8fd0adb-model-cache" pod "2eb68260-155c-4f10-84ec-0411e8fd0adb" (UID: "2eb68260-155c-4f10-84ec-0411e8fd0adb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:21:19.826952 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:19.826813    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2eb68260-155c-4f10-84ec-0411e8fd0adb-dshm\" (UniqueName: \"kubernetes.io/empty-dir/2eb68260-155c-4f10-84ec-0411e8fd0adb-dshm\") pod \"2eb68260-155c-4f10-84ec-0411e8fd0adb\" (UID: \"2eb68260-155c-4f10-84ec-0411e8fd0adb\") "
Oct 30 20:21:19.827036 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:19.827005    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2eb68260-155c-4f10-84ec-0411e8fd0adb-home" pod "2eb68260-155c-4f10-84ec-0411e8fd0adb" (UID: "2eb68260-155c-4f10-84ec-0411e8fd0adb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:21:19.827244 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:19.827215    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2eb68260-155c-4f10-84ec-0411e8fd0adb-tmp-dir" pod "2eb68260-155c-4f10-84ec-0411e8fd0adb" (UID: "2eb68260-155c-4f10-84ec-0411e8fd0adb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:21:19.827921 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:19.827849    6996 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2eb68260-155c-4f10-84ec-0411e8fd0adb-model-cache\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:21:19.827921 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:19.827874    6996 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2eb68260-155c-4f10-84ec-0411e8fd0adb-home\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:21:19.827921 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:19.827889    6996 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2eb68260-155c-4f10-84ec-0411e8fd0adb-tmp-dir\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:21:19.829632 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:19.829608    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2eb68260-155c-4f10-84ec-0411e8fd0adb-dshm" pod "2eb68260-155c-4f10-84ec-0411e8fd0adb" (UID: "2eb68260-155c-4f10-84ec-0411e8fd0adb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:21:19.829737 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:19.829627    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2eb68260-155c-4f10-84ec-0411e8fd0adb-tls-certs" pod "2eb68260-155c-4f10-84ec-0411e8fd0adb" (UID: "2eb68260-155c-4f10-84ec-0411e8fd0adb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:21:19.829737 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:19.829715    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2eb68260-155c-4f10-84ec-0411e8fd0adb-kube-api-access-pxw7q" pod "2eb68260-155c-4f10-84ec-0411e8fd0adb" (UID: "2eb68260-155c-4f10-84ec-0411e8fd0adb"). InnerVolumeSpecName "kube-api-access-pxw7q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:21:19.854395 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:19.854369    6996 scope.go:122] "RemoveContainer" containerID="fa0da3fe08a83dab5b9f400da642de792b60ba9971ce98857c33c799ec279e29"
Oct 30 20:21:19.854786 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:21:19.854764    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa0da3fe08a83dab5b9f400da642de792b60ba9971ce98857c33c799ec279e29\": container with ID starting with fa0da3fe08a83dab5b9f400da642de792b60ba9971ce98857c33c799ec279e29 not found: ID does not exist" containerID="fa0da3fe08a83dab5b9f400da642de792b60ba9971ce98857c33c799ec279e29"
Oct 30 20:21:19.854867 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:19.854796    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa0da3fe08a83dab5b9f400da642de792b60ba9971ce98857c33c799ec279e29"} err="failed to get container status \"fa0da3fe08a83dab5b9f400da642de792b60ba9971ce98857c33c799ec279e29\": rpc error: code = NotFound desc = could not find container \"fa0da3fe08a83dab5b9f400da642de792b60ba9971ce98857c33c799ec279e29\": container with ID starting with fa0da3fe08a83dab5b9f400da642de792b60ba9971ce98857c33c799ec279e29 not found: ID does not exist"
Oct 30 20:21:19.929447 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:19.929400    6996 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2eb68260-155c-4f10-84ec-0411e8fd0adb-dshm\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:21:19.929447 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:19.929438    6996 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2eb68260-155c-4f10-84ec-0411e8fd0adb-tls-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:21:19.929447 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:19.929452    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pxw7q\" (UniqueName: \"kubernetes.io/projected/2eb68260-155c-4f10-84ec-0411e8fd0adb-kube-api-access-pxw7q\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:21:20.151396 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:20.151356    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-bdf9e6/test-model-bdf9e6-kserve-6bbdfd7fb7-szfjr"]
Oct 30 20:21:20.160454 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:20.160419    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-bdf9e6/test-model-bdf9e6-kserve-6bbdfd7fb7-szfjr"]
Oct 30 20:21:20.791540 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:20.791508    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2eb68260-155c-4f10-84ec-0411e8fd0adb" path="/var/lib/kubelet/pods/2eb68260-155c-4f10-84ec-0411e8fd0adb/volumes"
Oct 30 20:21:21.052189 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:21.052103    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 20:21:31.192101 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:21:31.192059    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 20:21:32.760892 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:32.760847    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 20:21:49.930427 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:49.930336    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 20:21:52.374865 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:52.374832    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-bdf9e6-6b949fc948-zgdg6"]
Oct 30 20:21:52.375395 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:52.375110    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bdf9e6-6b949fc948-zgdg6" podUID="21af0b9f-44b8-43fc-bcda-751e196384ff" containerName="maas-api" containerID="cri-o://905d3b472d44ebce176ef0717cba463c5eb8db1718696af65c82fe7b75d32f83" gracePeriod=30
Oct 30 20:21:52.686040 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:52.686009    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bdf9e6-6b949fc948-zgdg6"
Oct 30 20:21:52.762542 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:52.762444    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/21af0b9f-44b8-43fc-bcda-751e196384ff-metrics-certs\" (UniqueName: \"kubernetes.io/secret/21af0b9f-44b8-43fc-bcda-751e196384ff-metrics-certs\") pod \"21af0b9f-44b8-43fc-bcda-751e196384ff\" (UID: \"21af0b9f-44b8-43fc-bcda-751e196384ff\") "
Oct 30 20:21:52.762542 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:52.762537    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/21af0b9f-44b8-43fc-bcda-751e196384ff-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/21af0b9f-44b8-43fc-bcda-751e196384ff-maas-api-tls\") pod \"21af0b9f-44b8-43fc-bcda-751e196384ff\" (UID: \"21af0b9f-44b8-43fc-bcda-751e196384ff\") "
Oct 30 20:21:52.762770 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:52.762652    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/21af0b9f-44b8-43fc-bcda-751e196384ff-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/21af0b9f-44b8-43fc-bcda-751e196384ff-openshift-service-ca\") pod \"21af0b9f-44b8-43fc-bcda-751e196384ff\" (UID: \"21af0b9f-44b8-43fc-bcda-751e196384ff\") "
Oct 30 20:21:52.762770 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:52.762698    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/21af0b9f-44b8-43fc-bcda-751e196384ff-kube-api-access-6lqf7\" (UniqueName: \"kubernetes.io/projected/21af0b9f-44b8-43fc-bcda-751e196384ff-kube-api-access-6lqf7\") pod \"21af0b9f-44b8-43fc-bcda-751e196384ff\" (UID: \"21af0b9f-44b8-43fc-bcda-751e196384ff\") "
Oct 30 20:21:52.763111 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:52.763080    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21af0b9f-44b8-43fc-bcda-751e196384ff-openshift-service-ca" pod "21af0b9f-44b8-43fc-bcda-751e196384ff" (UID: "21af0b9f-44b8-43fc-bcda-751e196384ff"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:21:52.763272 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:52.763251    6996 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/21af0b9f-44b8-43fc-bcda-751e196384ff-openshift-service-ca\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:21:52.764795 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:52.764764    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21af0b9f-44b8-43fc-bcda-751e196384ff-maas-api-tls" pod "21af0b9f-44b8-43fc-bcda-751e196384ff" (UID: "21af0b9f-44b8-43fc-bcda-751e196384ff"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:21:52.765080 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:52.765062    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21af0b9f-44b8-43fc-bcda-751e196384ff-metrics-certs" pod "21af0b9f-44b8-43fc-bcda-751e196384ff" (UID: "21af0b9f-44b8-43fc-bcda-751e196384ff"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:21:52.765286 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:52.765257    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21af0b9f-44b8-43fc-bcda-751e196384ff-kube-api-access-6lqf7" pod "21af0b9f-44b8-43fc-bcda-751e196384ff" (UID: "21af0b9f-44b8-43fc-bcda-751e196384ff"). InnerVolumeSpecName "kube-api-access-6lqf7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:21:52.864545 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:52.864516    6996 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/21af0b9f-44b8-43fc-bcda-751e196384ff-metrics-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:21:52.864545 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:52.864542    6996 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/21af0b9f-44b8-43fc-bcda-751e196384ff-maas-api-tls\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:21:52.864737 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:52.864553    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6lqf7\" (UniqueName: \"kubernetes.io/projected/21af0b9f-44b8-43fc-bcda-751e196384ff-kube-api-access-6lqf7\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:21:53.037384 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:53.037291    6996 generic.go:356] "Generic (PLEG): container finished" podID="21af0b9f-44b8-43fc-bcda-751e196384ff" containerID="905d3b472d44ebce176ef0717cba463c5eb8db1718696af65c82fe7b75d32f83" exitCode=0
Oct 30 20:21:53.037384 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:53.037342    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bdf9e6-6b949fc948-zgdg6"
Oct 30 20:21:53.037717 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:53.037379    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bdf9e6-6b949fc948-zgdg6" event={"ID":"21af0b9f-44b8-43fc-bcda-751e196384ff","Type":"ContainerDied","Data":"905d3b472d44ebce176ef0717cba463c5eb8db1718696af65c82fe7b75d32f83"}
Oct 30 20:21:53.037717 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:53.037433    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bdf9e6-6b949fc948-zgdg6" event={"ID":"21af0b9f-44b8-43fc-bcda-751e196384ff","Type":"ContainerDied","Data":"c306bbd1a3170d9da9e586d81fd7fdc4ce8d51ddd7dab636e142d2466a97895f"}
Oct 30 20:21:53.037717 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:53.037463    6996 scope.go:122] "RemoveContainer" containerID="905d3b472d44ebce176ef0717cba463c5eb8db1718696af65c82fe7b75d32f83"
Oct 30 20:21:53.047590 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:53.047566    6996 scope.go:122] "RemoveContainer" containerID="905d3b472d44ebce176ef0717cba463c5eb8db1718696af65c82fe7b75d32f83"
Oct 30 20:21:53.047861 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:21:53.047841    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"905d3b472d44ebce176ef0717cba463c5eb8db1718696af65c82fe7b75d32f83\": container with ID starting with 905d3b472d44ebce176ef0717cba463c5eb8db1718696af65c82fe7b75d32f83 not found: ID does not exist" containerID="905d3b472d44ebce176ef0717cba463c5eb8db1718696af65c82fe7b75d32f83"
Oct 30 20:21:53.047919 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:53.047872    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"905d3b472d44ebce176ef0717cba463c5eb8db1718696af65c82fe7b75d32f83"} err="failed to get container status \"905d3b472d44ebce176ef0717cba463c5eb8db1718696af65c82fe7b75d32f83\": rpc error: code = NotFound desc = could not find container \"905d3b472d44ebce176ef0717cba463c5eb8db1718696af65c82fe7b75d32f83\": container with ID starting with 905d3b472d44ebce176ef0717cba463c5eb8db1718696af65c82fe7b75d32f83 not found: ID does not exist"
Oct 30 20:21:53.056000 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:53.055975    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-bdf9e6-6b949fc948-zgdg6"]
Oct 30 20:21:53.060877 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:53.060853    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-bdf9e6-6b949fc948-zgdg6"]
Oct 30 20:21:54.791096 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:21:54.791064    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="21af0b9f-44b8-43fc-bcda-751e196384ff" path="/var/lib/kubelet/pods/21af0b9f-44b8-43fc-bcda-751e196384ff/volumes"
Oct 30 20:22:01.191148 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:22:01.191108    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 20:22:04.325705 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:04.325670    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-5f9b3c-6c5f497c6d-sfvdl"]
Oct 30 20:22:04.326205 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:04.325901    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-5f9b3c-6c5f497c6d-sfvdl" podUID="91b87b65-f883-485d-abc4-69f270f6b3e9" containerName="maas-api" containerID="cri-o://bb01fb7c2693485285e671c3d9f424617798bc739a2107922641fd9b74bca4ca" gracePeriod=30
Oct 30 20:22:04.633469 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:04.633443    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-5f9b3c-6c5f497c6d-sfvdl"
Oct 30 20:22:04.726625 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:04.726590    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/91b87b65-f883-485d-abc4-69f270f6b3e9-kube-api-access-4cvn4\" (UniqueName: \"kubernetes.io/projected/91b87b65-f883-485d-abc4-69f270f6b3e9-kube-api-access-4cvn4\") pod \"91b87b65-f883-485d-abc4-69f270f6b3e9\" (UID: \"91b87b65-f883-485d-abc4-69f270f6b3e9\") "
Oct 30 20:22:04.726802 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:04.726702    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/91b87b65-f883-485d-abc4-69f270f6b3e9-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91b87b65-f883-485d-abc4-69f270f6b3e9-openshift-service-ca\") pod \"91b87b65-f883-485d-abc4-69f270f6b3e9\" (UID: \"91b87b65-f883-485d-abc4-69f270f6b3e9\") "
Oct 30 20:22:04.726802 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:04.726730    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/91b87b65-f883-485d-abc4-69f270f6b3e9-metrics-certs\" (UniqueName: \"kubernetes.io/secret/91b87b65-f883-485d-abc4-69f270f6b3e9-metrics-certs\") pod \"91b87b65-f883-485d-abc4-69f270f6b3e9\" (UID: \"91b87b65-f883-485d-abc4-69f270f6b3e9\") "
Oct 30 20:22:04.726890 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:04.726818    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/91b87b65-f883-485d-abc4-69f270f6b3e9-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91b87b65-f883-485d-abc4-69f270f6b3e9-maas-api-tls\") pod \"91b87b65-f883-485d-abc4-69f270f6b3e9\" (UID: \"91b87b65-f883-485d-abc4-69f270f6b3e9\") "
Oct 30 20:22:04.727107 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:04.727078    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91b87b65-f883-485d-abc4-69f270f6b3e9-openshift-service-ca" pod "91b87b65-f883-485d-abc4-69f270f6b3e9" (UID: "91b87b65-f883-485d-abc4-69f270f6b3e9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:22:04.728933 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:04.728900    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91b87b65-f883-485d-abc4-69f270f6b3e9-maas-api-tls" pod "91b87b65-f883-485d-abc4-69f270f6b3e9" (UID: "91b87b65-f883-485d-abc4-69f270f6b3e9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:22:04.728933 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:04.728919    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91b87b65-f883-485d-abc4-69f270f6b3e9-kube-api-access-4cvn4" pod "91b87b65-f883-485d-abc4-69f270f6b3e9" (UID: "91b87b65-f883-485d-abc4-69f270f6b3e9"). InnerVolumeSpecName "kube-api-access-4cvn4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:22:04.729101 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:04.729008    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91b87b65-f883-485d-abc4-69f270f6b3e9-metrics-certs" pod "91b87b65-f883-485d-abc4-69f270f6b3e9" (UID: "91b87b65-f883-485d-abc4-69f270f6b3e9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:22:04.828843 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:04.828804    6996 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91b87b65-f883-485d-abc4-69f270f6b3e9-openshift-service-ca\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:04.829002 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:04.828848    6996 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91b87b65-f883-485d-abc4-69f270f6b3e9-metrics-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:04.829002 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:04.828866    6996 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91b87b65-f883-485d-abc4-69f270f6b3e9-maas-api-tls\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:04.829002 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:04.828881    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4cvn4\" (UniqueName: \"kubernetes.io/projected/91b87b65-f883-485d-abc4-69f270f6b3e9-kube-api-access-4cvn4\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:05.110721 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:05.110689    6996 generic.go:356] "Generic (PLEG): container finished" podID="91b87b65-f883-485d-abc4-69f270f6b3e9" containerID="bb01fb7c2693485285e671c3d9f424617798bc739a2107922641fd9b74bca4ca" exitCode=0
Oct 30 20:22:05.110945 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:05.110740    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-5f9b3c-6c5f497c6d-sfvdl"
Oct 30 20:22:05.110945 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:05.110771    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-5f9b3c-6c5f497c6d-sfvdl" event={"ID":"91b87b65-f883-485d-abc4-69f270f6b3e9","Type":"ContainerDied","Data":"bb01fb7c2693485285e671c3d9f424617798bc739a2107922641fd9b74bca4ca"}
Oct 30 20:22:05.110945 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:05.110830    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-5f9b3c-6c5f497c6d-sfvdl" event={"ID":"91b87b65-f883-485d-abc4-69f270f6b3e9","Type":"ContainerDied","Data":"717cdc4269002bae7271a3dc6adc7920acae7d3f170162cfb37388c1f89933bc"}
Oct 30 20:22:05.110945 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:05.110854    6996 scope.go:122] "RemoveContainer" containerID="bb01fb7c2693485285e671c3d9f424617798bc739a2107922641fd9b74bca4ca"
Oct 30 20:22:05.123305 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:05.123283    6996 scope.go:122] "RemoveContainer" containerID="bb01fb7c2693485285e671c3d9f424617798bc739a2107922641fd9b74bca4ca"
Oct 30 20:22:05.123631 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:22:05.123611    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb01fb7c2693485285e671c3d9f424617798bc739a2107922641fd9b74bca4ca\": container with ID starting with bb01fb7c2693485285e671c3d9f424617798bc739a2107922641fd9b74bca4ca not found: ID does not exist" containerID="bb01fb7c2693485285e671c3d9f424617798bc739a2107922641fd9b74bca4ca"
Oct 30 20:22:05.123753 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:05.123641    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb01fb7c2693485285e671c3d9f424617798bc739a2107922641fd9b74bca4ca"} err="failed to get container status \"bb01fb7c2693485285e671c3d9f424617798bc739a2107922641fd9b74bca4ca\": rpc error: code = NotFound desc = could not find container \"bb01fb7c2693485285e671c3d9f424617798bc739a2107922641fd9b74bca4ca\": container with ID starting with bb01fb7c2693485285e671c3d9f424617798bc739a2107922641fd9b74bca4ca not found: ID does not exist"
Oct 30 20:22:05.130302 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:05.130276    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-5f9b3c-6c5f497c6d-sfvdl"]
Oct 30 20:22:05.133162 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:05.133140    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-5f9b3c-6c5f497c6d-sfvdl"]
Oct 30 20:22:06.145881 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:06.145848    6996 scope.go:122] "RemoveContainer" containerID="40041b3b73f16d71ce6ee630d3555932a30bdd2c62fc5b8dfac6c75fae446034"
Oct 30 20:22:06.792790 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:06.792759    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="91b87b65-f883-485d-abc4-69f270f6b3e9" path="/var/lib/kubelet/pods/91b87b65-f883-485d-abc4-69f270f6b3e9/volumes"
Oct 30 20:22:07.490726 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:07.490691    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-rbwvb"]
Oct 30 20:22:07.491229 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:07.490895    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-rbwvb" podUID="9a76e670-c0a4-49ee-9b5f-7fabddaef0e9" containerName="istio-proxy" containerID="cri-o://995b0f40241869bb396be27ca6069f8a7f394bd692c1436edb5d7c8dbb9f342b" gracePeriod=30
Oct 30 20:22:08.392116 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:08.392085    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 20:22:12.758058 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:12.758030    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-rbwvb"
Oct 30 20:22:12.821633 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:12.821603    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9a76e670-c0a4-49ee-9b5f-7fabddaef0e9-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a76e670-c0a4-49ee-9b5f-7fabddaef0e9-istio-podinfo\") pod \"9a76e670-c0a4-49ee-9b5f-7fabddaef0e9\" (UID: \"9a76e670-c0a4-49ee-9b5f-7fabddaef0e9\") "
Oct 30 20:22:12.821817 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:12.821665    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9a76e670-c0a4-49ee-9b5f-7fabddaef0e9-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a76e670-c0a4-49ee-9b5f-7fabddaef0e9-istio-envoy\") pod \"9a76e670-c0a4-49ee-9b5f-7fabddaef0e9\" (UID: \"9a76e670-c0a4-49ee-9b5f-7fabddaef0e9\") "
Oct 30 20:22:12.821817 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:12.821709    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9a76e670-c0a4-49ee-9b5f-7fabddaef0e9-kube-api-access-kg5q2\" (UniqueName: \"kubernetes.io/projected/9a76e670-c0a4-49ee-9b5f-7fabddaef0e9-kube-api-access-kg5q2\") pod \"9a76e670-c0a4-49ee-9b5f-7fabddaef0e9\" (UID: \"9a76e670-c0a4-49ee-9b5f-7fabddaef0e9\") "
Oct 30 20:22:12.821817 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:12.821746    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9a76e670-c0a4-49ee-9b5f-7fabddaef0e9-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a76e670-c0a4-49ee-9b5f-7fabddaef0e9-istiod-ca-cert\") pod \"9a76e670-c0a4-49ee-9b5f-7fabddaef0e9\" (UID: \"9a76e670-c0a4-49ee-9b5f-7fabddaef0e9\") "
Oct 30 20:22:12.821817 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:12.821783    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9a76e670-c0a4-49ee-9b5f-7fabddaef0e9-istio-token\" (UniqueName: \"kubernetes.io/projected/9a76e670-c0a4-49ee-9b5f-7fabddaef0e9-istio-token\") pod \"9a76e670-c0a4-49ee-9b5f-7fabddaef0e9\" (UID: \"9a76e670-c0a4-49ee-9b5f-7fabddaef0e9\") "
Oct 30 20:22:12.822072 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:12.821845    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9a76e670-c0a4-49ee-9b5f-7fabddaef0e9-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a76e670-c0a4-49ee-9b5f-7fabddaef0e9-workload-certs\") pod \"9a76e670-c0a4-49ee-9b5f-7fabddaef0e9\" (UID: \"9a76e670-c0a4-49ee-9b5f-7fabddaef0e9\") "
Oct 30 20:22:12.822072 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:12.821880    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9a76e670-c0a4-49ee-9b5f-7fabddaef0e9-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a76e670-c0a4-49ee-9b5f-7fabddaef0e9-istio-data\") pod \"9a76e670-c0a4-49ee-9b5f-7fabddaef0e9\" (UID: \"9a76e670-c0a4-49ee-9b5f-7fabddaef0e9\") "
Oct 30 20:22:12.822072 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:12.821965    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9a76e670-c0a4-49ee-9b5f-7fabddaef0e9-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a76e670-c0a4-49ee-9b5f-7fabddaef0e9-workload-socket\") pod \"9a76e670-c0a4-49ee-9b5f-7fabddaef0e9\" (UID: \"9a76e670-c0a4-49ee-9b5f-7fabddaef0e9\") "
Oct 30 20:22:12.822072 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:12.821998    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9a76e670-c0a4-49ee-9b5f-7fabddaef0e9-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a76e670-c0a4-49ee-9b5f-7fabddaef0e9-credential-socket\") pod \"9a76e670-c0a4-49ee-9b5f-7fabddaef0e9\" (UID: \"9a76e670-c0a4-49ee-9b5f-7fabddaef0e9\") "
Oct 30 20:22:12.822394 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:12.822359    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a76e670-c0a4-49ee-9b5f-7fabddaef0e9-istio-data" pod "9a76e670-c0a4-49ee-9b5f-7fabddaef0e9" (UID: "9a76e670-c0a4-49ee-9b5f-7fabddaef0e9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:12.823304 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:12.822665    6996 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a76e670-c0a4-49ee-9b5f-7fabddaef0e9-istio-data\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:12.823304 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:12.822878    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a76e670-c0a4-49ee-9b5f-7fabddaef0e9-workload-certs" pod "9a76e670-c0a4-49ee-9b5f-7fabddaef0e9" (UID: "9a76e670-c0a4-49ee-9b5f-7fabddaef0e9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:12.823304 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:12.823051    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a76e670-c0a4-49ee-9b5f-7fabddaef0e9-workload-socket" pod "9a76e670-c0a4-49ee-9b5f-7fabddaef0e9" (UID: "9a76e670-c0a4-49ee-9b5f-7fabddaef0e9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:12.823304 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:12.823053    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a76e670-c0a4-49ee-9b5f-7fabddaef0e9-istiod-ca-cert" pod "9a76e670-c0a4-49ee-9b5f-7fabddaef0e9" (UID: "9a76e670-c0a4-49ee-9b5f-7fabddaef0e9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:22:12.823304 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:12.823210    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a76e670-c0a4-49ee-9b5f-7fabddaef0e9-credential-socket" pod "9a76e670-c0a4-49ee-9b5f-7fabddaef0e9" (UID: "9a76e670-c0a4-49ee-9b5f-7fabddaef0e9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:12.824463 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:12.824435    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a76e670-c0a4-49ee-9b5f-7fabddaef0e9-istio-token" pod "9a76e670-c0a4-49ee-9b5f-7fabddaef0e9" (UID: "9a76e670-c0a4-49ee-9b5f-7fabddaef0e9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:22:12.824765 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:12.824735    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9a76e670-c0a4-49ee-9b5f-7fabddaef0e9-istio-podinfo" pod "9a76e670-c0a4-49ee-9b5f-7fabddaef0e9" (UID: "9a76e670-c0a4-49ee-9b5f-7fabddaef0e9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 20:22:12.824859 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:12.824820    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a76e670-c0a4-49ee-9b5f-7fabddaef0e9-istio-envoy" pod "9a76e670-c0a4-49ee-9b5f-7fabddaef0e9" (UID: "9a76e670-c0a4-49ee-9b5f-7fabddaef0e9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:12.825763 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:12.825739    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a76e670-c0a4-49ee-9b5f-7fabddaef0e9-kube-api-access-kg5q2" pod "9a76e670-c0a4-49ee-9b5f-7fabddaef0e9" (UID: "9a76e670-c0a4-49ee-9b5f-7fabddaef0e9"). InnerVolumeSpecName "kube-api-access-kg5q2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:22:12.923816 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:12.923779    6996 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a76e670-c0a4-49ee-9b5f-7fabddaef0e9-workload-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:12.923816 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:12.923811    6996 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a76e670-c0a4-49ee-9b5f-7fabddaef0e9-credential-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:12.924030 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:12.923823    6996 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a76e670-c0a4-49ee-9b5f-7fabddaef0e9-istio-podinfo\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:12.924030 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:12.923836    6996 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a76e670-c0a4-49ee-9b5f-7fabddaef0e9-istio-envoy\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:12.924030 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:12.923846    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kg5q2\" (UniqueName: \"kubernetes.io/projected/9a76e670-c0a4-49ee-9b5f-7fabddaef0e9-kube-api-access-kg5q2\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:12.924030 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:12.923856    6996 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a76e670-c0a4-49ee-9b5f-7fabddaef0e9-istiod-ca-cert\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:12.924030 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:12.923865    6996 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9a76e670-c0a4-49ee-9b5f-7fabddaef0e9-istio-token\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:12.924030 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:12.923873    6996 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a76e670-c0a4-49ee-9b5f-7fabddaef0e9-workload-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:13.187907 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:13.187817    6996 generic.go:356] "Generic (PLEG): container finished" podID="9a76e670-c0a4-49ee-9b5f-7fabddaef0e9" containerID="995b0f40241869bb396be27ca6069f8a7f394bd692c1436edb5d7c8dbb9f342b" exitCode=0
Oct 30 20:22:13.187907 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:13.187863    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-rbwvb"
Oct 30 20:22:13.187907 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:13.187897    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-rbwvb" event={"ID":"9a76e670-c0a4-49ee-9b5f-7fabddaef0e9","Type":"ContainerDied","Data":"995b0f40241869bb396be27ca6069f8a7f394bd692c1436edb5d7c8dbb9f342b"}
Oct 30 20:22:13.188168 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:13.187945    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-rbwvb" event={"ID":"9a76e670-c0a4-49ee-9b5f-7fabddaef0e9","Type":"ContainerDied","Data":"a9b2d1bdfc00960c1277ac3675fdbc8ea83aba580fd61d49df5a38405ce8264c"}
Oct 30 20:22:13.188168 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:13.187967    6996 scope.go:122] "RemoveContainer" containerID="995b0f40241869bb396be27ca6069f8a7f394bd692c1436edb5d7c8dbb9f342b"
Oct 30 20:22:13.199017 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:13.199000    6996 scope.go:122] "RemoveContainer" containerID="995b0f40241869bb396be27ca6069f8a7f394bd692c1436edb5d7c8dbb9f342b"
Oct 30 20:22:13.199285 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:22:13.199268    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"995b0f40241869bb396be27ca6069f8a7f394bd692c1436edb5d7c8dbb9f342b\": container with ID starting with 995b0f40241869bb396be27ca6069f8a7f394bd692c1436edb5d7c8dbb9f342b not found: ID does not exist" containerID="995b0f40241869bb396be27ca6069f8a7f394bd692c1436edb5d7c8dbb9f342b"
Oct 30 20:22:13.199323 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:13.199295    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"995b0f40241869bb396be27ca6069f8a7f394bd692c1436edb5d7c8dbb9f342b"} err="failed to get container status \"995b0f40241869bb396be27ca6069f8a7f394bd692c1436edb5d7c8dbb9f342b\": rpc error: code = NotFound desc = could not find container \"995b0f40241869bb396be27ca6069f8a7f394bd692c1436edb5d7c8dbb9f342b\": container with ID starting with 995b0f40241869bb396be27ca6069f8a7f394bd692c1436edb5d7c8dbb9f342b not found: ID does not exist"
Oct 30 20:22:13.215991 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:13.215962    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-rbwvb"]
Oct 30 20:22:13.223014 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:13.222978    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-rbwvb"]
Oct 30 20:22:14.791261 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:14.791218    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9a76e670-c0a4-49ee-9b5f-7fabddaef0e9" path="/var/lib/kubelet/pods/9a76e670-c0a4-49ee-9b5f-7fabddaef0e9/volumes"
Oct 30 20:22:23.534214 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:23.534177    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 20:22:31.196051 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:22:31.196009    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 20:22:34.560240 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:34.560207    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 20:22:35.175290 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:35.175180    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88"]
Oct 30 20:22:35.175513 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:35.175453    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88" podUID="a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d" containerName="istio-proxy" containerID="cri-o://069a1a3cdad8370d2251b30bc1ee006721315f513f0639354695499a67eb5f71" gracePeriod=30
Oct 30 20:22:37.835937 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:37.835897    6996 patch_prober.go:29] interesting pod/e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 20:22:37.836338 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:37.835983    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88" podUID="a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 20:22:38.013508 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:38.013454    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-4d5758-6985bf95bc-twp4p"]
Oct 30 20:22:40.380201 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:40.380165    6996 generic.go:356] "Generic (PLEG): container finished" podID="a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d" containerID="069a1a3cdad8370d2251b30bc1ee006721315f513f0639354695499a67eb5f71" exitCode=0
Oct 30 20:22:40.380712 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:40.380241    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88" event={"ID":"a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d","Type":"ContainerDied","Data":"069a1a3cdad8370d2251b30bc1ee006721315f513f0639354695499a67eb5f71"}
Oct 30 20:22:40.390099 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:40.390067    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-4d5758-766c8d5467-z6cst"]
Oct 30 20:22:40.390375 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:40.390343    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-4d5758-766c8d5467-z6cst" podUID="24beb51d-3611-47ca-8061-b17dbfc16c80" containerName="maas-api" containerID="cri-o://be44a5bc7e3d52e65d49e029306791833f63654d70982db4f6eb0624c7f163a2" gracePeriod=30
Oct 30 20:22:40.520223 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:40.520197    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88"
Oct 30 20:22:40.550348 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:40.550303    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d-kube-api-access-plk5w\" (UniqueName: \"kubernetes.io/projected/a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d-kube-api-access-plk5w\") pod \"a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d\" (UID: \"a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d\") "
Oct 30 20:22:40.550532 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:40.550368    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d-workload-certs\") pod \"a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d\" (UID: \"a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d\") "
Oct 30 20:22:40.550532 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:40.550402    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d-istio-data\") pod \"a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d\" (UID: \"a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d\") "
Oct 30 20:22:40.550532 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:40.550460    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d-workload-socket\") pod \"a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d\" (UID: \"a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d\") "
Oct 30 20:22:40.550532 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:40.550529    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d-istio-token\" (UniqueName: \"kubernetes.io/projected/a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d-istio-token\") pod \"a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d\" (UID: \"a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d\") "
Oct 30 20:22:40.550705 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:40.550647    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d-istiod-ca-cert\") pod \"a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d\" (UID: \"a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d\") "
Oct 30 20:22:40.550705 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:40.550689    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d-istio-podinfo\") pod \"a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d\" (UID: \"a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d\") "
Oct 30 20:22:40.550774 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:40.550731    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d-credential-socket\") pod \"a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d\" (UID: \"a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d\") "
Oct 30 20:22:40.550817 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:40.550780    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d-istio-envoy\") pod \"a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d\" (UID: \"a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d\") "
Oct 30 20:22:40.552137 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:40.552105    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d-workload-socket" pod "a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d" (UID: "a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:40.552850 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:40.552811    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d-workload-certs" pod "a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d" (UID: "a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:40.553322 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:40.553301    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d-istio-data" pod "a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d" (UID: "a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:40.553950 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:40.553569    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d-istio-envoy" pod "a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d" (UID: "a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:40.553950 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:40.553695    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d-credential-socket" pod "a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d" (UID: "a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:22:40.553950 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:40.553769    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d-istiod-ca-cert" pod "a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d" (UID: "a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:22:40.555145 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:40.555117    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d-kube-api-access-plk5w" pod "a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d" (UID: "a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d"). InnerVolumeSpecName "kube-api-access-plk5w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:22:40.556404 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:40.556384    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d-istio-token" pod "a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d" (UID: "a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:22:40.556725 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:40.556693    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d-istio-podinfo" pod "a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d" (UID: "a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 20:22:40.652289 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:40.652249    6996 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d-workload-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:40.652289 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:40.652289    6996 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d-istio-token\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:40.652620 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:40.652303    6996 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d-istiod-ca-cert\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:40.652620 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:40.652318    6996 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d-istio-podinfo\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:40.652620 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:40.652331    6996 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d-credential-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:40.652620 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:40.652346    6996 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d-istio-envoy\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:40.652620 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:40.652387    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-plk5w\" (UniqueName: \"kubernetes.io/projected/a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d-kube-api-access-plk5w\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:40.652620 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:40.652403    6996 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d-workload-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:40.652620 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:40.652416    6996 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d-istio-data\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:40.684534 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:40.684507    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-4d5758-766c8d5467-z6cst"
Oct 30 20:22:40.753878 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:40.753802    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/24beb51d-3611-47ca-8061-b17dbfc16c80-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/24beb51d-3611-47ca-8061-b17dbfc16c80-openshift-service-ca\") pod \"24beb51d-3611-47ca-8061-b17dbfc16c80\" (UID: \"24beb51d-3611-47ca-8061-b17dbfc16c80\") "
Oct 30 20:22:40.754027 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:40.753921    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/24beb51d-3611-47ca-8061-b17dbfc16c80-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/24beb51d-3611-47ca-8061-b17dbfc16c80-maas-api-tls\") pod \"24beb51d-3611-47ca-8061-b17dbfc16c80\" (UID: \"24beb51d-3611-47ca-8061-b17dbfc16c80\") "
Oct 30 20:22:40.754027 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:40.753976    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/24beb51d-3611-47ca-8061-b17dbfc16c80-metrics-certs\" (UniqueName: \"kubernetes.io/secret/24beb51d-3611-47ca-8061-b17dbfc16c80-metrics-certs\") pod \"24beb51d-3611-47ca-8061-b17dbfc16c80\" (UID: \"24beb51d-3611-47ca-8061-b17dbfc16c80\") "
Oct 30 20:22:40.754102 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:40.754029    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/24beb51d-3611-47ca-8061-b17dbfc16c80-kube-api-access-k7xpb\" (UniqueName: \"kubernetes.io/projected/24beb51d-3611-47ca-8061-b17dbfc16c80-kube-api-access-k7xpb\") pod \"24beb51d-3611-47ca-8061-b17dbfc16c80\" (UID: \"24beb51d-3611-47ca-8061-b17dbfc16c80\") "
Oct 30 20:22:40.754238 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:40.754216    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/24beb51d-3611-47ca-8061-b17dbfc16c80-openshift-service-ca" pod "24beb51d-3611-47ca-8061-b17dbfc16c80" (UID: "24beb51d-3611-47ca-8061-b17dbfc16c80"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:22:40.754420 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:40.754404    6996 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/24beb51d-3611-47ca-8061-b17dbfc16c80-openshift-service-ca\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:40.756134 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:40.756107    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/24beb51d-3611-47ca-8061-b17dbfc16c80-metrics-certs" pod "24beb51d-3611-47ca-8061-b17dbfc16c80" (UID: "24beb51d-3611-47ca-8061-b17dbfc16c80"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:22:40.756226 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:40.756135    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/24beb51d-3611-47ca-8061-b17dbfc16c80-maas-api-tls" pod "24beb51d-3611-47ca-8061-b17dbfc16c80" (UID: "24beb51d-3611-47ca-8061-b17dbfc16c80"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:22:40.756226 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:40.756174    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24beb51d-3611-47ca-8061-b17dbfc16c80-kube-api-access-k7xpb" pod "24beb51d-3611-47ca-8061-b17dbfc16c80" (UID: "24beb51d-3611-47ca-8061-b17dbfc16c80"). InnerVolumeSpecName "kube-api-access-k7xpb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:22:40.855390 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:40.855351    6996 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/24beb51d-3611-47ca-8061-b17dbfc16c80-maas-api-tls\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:40.855390 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:40.855392    6996 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24beb51d-3611-47ca-8061-b17dbfc16c80-metrics-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:40.855598 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:40.855407    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k7xpb\" (UniqueName: \"kubernetes.io/projected/24beb51d-3611-47ca-8061-b17dbfc16c80-kube-api-access-k7xpb\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:41.387371 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:41.387343    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88"
Oct 30 20:22:41.387371 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:41.387365    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88" event={"ID":"a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d","Type":"ContainerDied","Data":"c1b4c1fee9cedcddeeeb95ffc147cffc1b11f2f49221a406864b950a5becaab4"}
Oct 30 20:22:41.387925 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:41.387436    6996 scope.go:122] "RemoveContainer" containerID="069a1a3cdad8370d2251b30bc1ee006721315f513f0639354695499a67eb5f71"
Oct 30 20:22:41.388786 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:41.388763    6996 generic.go:356] "Generic (PLEG): container finished" podID="24beb51d-3611-47ca-8061-b17dbfc16c80" containerID="be44a5bc7e3d52e65d49e029306791833f63654d70982db4f6eb0624c7f163a2" exitCode=0
Oct 30 20:22:41.388901 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:41.388795    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-4d5758-766c8d5467-z6cst" event={"ID":"24beb51d-3611-47ca-8061-b17dbfc16c80","Type":"ContainerDied","Data":"be44a5bc7e3d52e65d49e029306791833f63654d70982db4f6eb0624c7f163a2"}
Oct 30 20:22:41.388901 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:41.388846    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-4d5758-766c8d5467-z6cst" event={"ID":"24beb51d-3611-47ca-8061-b17dbfc16c80","Type":"ContainerDied","Data":"ad2a67fef2d7ebc7eef1e98cf9302f9481073ba6973dc2c5c70ac34a8df1d6b6"}
Oct 30 20:22:41.388901 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:41.388804    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-4d5758-766c8d5467-z6cst"
Oct 30 20:22:41.398204 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:41.398187    6996 scope.go:122] "RemoveContainer" containerID="be44a5bc7e3d52e65d49e029306791833f63654d70982db4f6eb0624c7f163a2"
Oct 30 20:22:41.409817 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:41.409799    6996 scope.go:122] "RemoveContainer" containerID="be44a5bc7e3d52e65d49e029306791833f63654d70982db4f6eb0624c7f163a2"
Oct 30 20:22:41.410166 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:22:41.410143    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be44a5bc7e3d52e65d49e029306791833f63654d70982db4f6eb0624c7f163a2\": container with ID starting with be44a5bc7e3d52e65d49e029306791833f63654d70982db4f6eb0624c7f163a2 not found: ID does not exist" containerID="be44a5bc7e3d52e65d49e029306791833f63654d70982db4f6eb0624c7f163a2"
Oct 30 20:22:41.410224 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:41.410175    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be44a5bc7e3d52e65d49e029306791833f63654d70982db4f6eb0624c7f163a2"} err="failed to get container status \"be44a5bc7e3d52e65d49e029306791833f63654d70982db4f6eb0624c7f163a2\": rpc error: code = NotFound desc = could not find container \"be44a5bc7e3d52e65d49e029306791833f63654d70982db4f6eb0624c7f163a2\": container with ID starting with be44a5bc7e3d52e65d49e029306791833f63654d70982db4f6eb0624c7f163a2 not found: ID does not exist"
Oct 30 20:22:41.412220 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:41.412195    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88"]
Oct 30 20:22:41.414578 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:41.414557    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-7eaeed-openshift-default-694c55cd79-b4r88"]
Oct 30 20:22:41.425645 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:41.425623    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-4d5758-766c8d5467-z6cst"]
Oct 30 20:22:41.433436 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:41.433413    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-4d5758-766c8d5467-z6cst"]
Oct 30 20:22:42.790981 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:42.790946    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="24beb51d-3611-47ca-8061-b17dbfc16c80" path="/var/lib/kubelet/pods/24beb51d-3611-47ca-8061-b17dbfc16c80/volumes"
Oct 30 20:22:42.791600 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:42.791581    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d" path="/var/lib/kubelet/pods/a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d/volumes"
Oct 30 20:22:43.013863 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:43.013818    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-4d5758-6985bf95bc-twp4p" podUID="316882f2-58f3-404a-a515-310efc6f2462" containerName="payload-pre-processing" containerID="cri-o://5e03fe9f19e192b5b4a3d1cec291e20f83467107868ac65fc820e1ffd10c34c6" gracePeriod=25
Oct 30 20:22:43.725902 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:43.725864    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-4d5758-6985bf95bc-twp4p" podUID="316882f2-58f3-404a-a515-310efc6f2462" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 20:22:45.023718 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:45.023680    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 20:22:47.910956 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:47.910916    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-981ee5-dbb54d47d-sr5mp"]
Oct 30 20:22:47.911505 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:47.911173    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-981ee5-dbb54d47d-sr5mp" podUID="035ed3f1-789b-4a62-80e6-3f8841905629" containerName="maas-api" containerID="cri-o://5873c72003f514dbb67590ea158944ad24705be25e27c36d09dc0ba40b75d41c" gracePeriod=30
Oct 30 20:22:48.211260 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:48.211235    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-981ee5-dbb54d47d-sr5mp"
Oct 30 20:22:48.237182 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:48.237145    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/035ed3f1-789b-4a62-80e6-3f8841905629-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/035ed3f1-789b-4a62-80e6-3f8841905629-openshift-service-ca\") pod \"035ed3f1-789b-4a62-80e6-3f8841905629\" (UID: \"035ed3f1-789b-4a62-80e6-3f8841905629\") "
Oct 30 20:22:48.237360 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:48.237212    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/035ed3f1-789b-4a62-80e6-3f8841905629-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/035ed3f1-789b-4a62-80e6-3f8841905629-maas-api-tls\") pod \"035ed3f1-789b-4a62-80e6-3f8841905629\" (UID: \"035ed3f1-789b-4a62-80e6-3f8841905629\") "
Oct 30 20:22:48.237360 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:48.237338    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/035ed3f1-789b-4a62-80e6-3f8841905629-kube-api-access-f95fg\" (UniqueName: \"kubernetes.io/projected/035ed3f1-789b-4a62-80e6-3f8841905629-kube-api-access-f95fg\") pod \"035ed3f1-789b-4a62-80e6-3f8841905629\" (UID: \"035ed3f1-789b-4a62-80e6-3f8841905629\") "
Oct 30 20:22:48.237496 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:48.237414    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/035ed3f1-789b-4a62-80e6-3f8841905629-metrics-certs\" (UniqueName: \"kubernetes.io/secret/035ed3f1-789b-4a62-80e6-3f8841905629-metrics-certs\") pod \"035ed3f1-789b-4a62-80e6-3f8841905629\" (UID: \"035ed3f1-789b-4a62-80e6-3f8841905629\") "
Oct 30 20:22:48.237589 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:48.237560    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/035ed3f1-789b-4a62-80e6-3f8841905629-openshift-service-ca" pod "035ed3f1-789b-4a62-80e6-3f8841905629" (UID: "035ed3f1-789b-4a62-80e6-3f8841905629"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:22:48.237919 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:48.237894    6996 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/035ed3f1-789b-4a62-80e6-3f8841905629-openshift-service-ca\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:48.239671 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:48.239643    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/035ed3f1-789b-4a62-80e6-3f8841905629-metrics-certs" pod "035ed3f1-789b-4a62-80e6-3f8841905629" (UID: "035ed3f1-789b-4a62-80e6-3f8841905629"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:22:48.240207 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:48.240090    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/035ed3f1-789b-4a62-80e6-3f8841905629-maas-api-tls" pod "035ed3f1-789b-4a62-80e6-3f8841905629" (UID: "035ed3f1-789b-4a62-80e6-3f8841905629"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:22:48.240207 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:48.240111    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/035ed3f1-789b-4a62-80e6-3f8841905629-kube-api-access-f95fg" pod "035ed3f1-789b-4a62-80e6-3f8841905629" (UID: "035ed3f1-789b-4a62-80e6-3f8841905629"). InnerVolumeSpecName "kube-api-access-f95fg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:22:48.339407 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:48.339372    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f95fg\" (UniqueName: \"kubernetes.io/projected/035ed3f1-789b-4a62-80e6-3f8841905629-kube-api-access-f95fg\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:48.339407 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:48.339402    6996 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/035ed3f1-789b-4a62-80e6-3f8841905629-metrics-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:48.339407 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:48.339413    6996 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/035ed3f1-789b-4a62-80e6-3f8841905629-maas-api-tls\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:22:48.431889 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:48.431799    6996 generic.go:356] "Generic (PLEG): container finished" podID="035ed3f1-789b-4a62-80e6-3f8841905629" containerID="5873c72003f514dbb67590ea158944ad24705be25e27c36d09dc0ba40b75d41c" exitCode=0
Oct 30 20:22:48.431889 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:48.431845    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-981ee5-dbb54d47d-sr5mp"
Oct 30 20:22:48.432143 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:48.431884    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-981ee5-dbb54d47d-sr5mp" event={"ID":"035ed3f1-789b-4a62-80e6-3f8841905629","Type":"ContainerDied","Data":"5873c72003f514dbb67590ea158944ad24705be25e27c36d09dc0ba40b75d41c"}
Oct 30 20:22:48.432143 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:48.431947    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-981ee5-dbb54d47d-sr5mp" event={"ID":"035ed3f1-789b-4a62-80e6-3f8841905629","Type":"ContainerDied","Data":"8434220309ba767e180029f6c766c3dbc927f810184a6f53c22a481572e6df25"}
Oct 30 20:22:48.432143 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:48.431976    6996 scope.go:122] "RemoveContainer" containerID="5873c72003f514dbb67590ea158944ad24705be25e27c36d09dc0ba40b75d41c"
Oct 30 20:22:48.441717 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:48.441702    6996 scope.go:122] "RemoveContainer" containerID="5873c72003f514dbb67590ea158944ad24705be25e27c36d09dc0ba40b75d41c"
Oct 30 20:22:48.441978 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:22:48.441957    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5873c72003f514dbb67590ea158944ad24705be25e27c36d09dc0ba40b75d41c\": container with ID starting with 5873c72003f514dbb67590ea158944ad24705be25e27c36d09dc0ba40b75d41c not found: ID does not exist" containerID="5873c72003f514dbb67590ea158944ad24705be25e27c36d09dc0ba40b75d41c"
Oct 30 20:22:48.442045 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:48.441989    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5873c72003f514dbb67590ea158944ad24705be25e27c36d09dc0ba40b75d41c"} err="failed to get container status \"5873c72003f514dbb67590ea158944ad24705be25e27c36d09dc0ba40b75d41c\": rpc error: code = NotFound desc = could not find container \"5873c72003f514dbb67590ea158944ad24705be25e27c36d09dc0ba40b75d41c\": container with ID starting with 5873c72003f514dbb67590ea158944ad24705be25e27c36d09dc0ba40b75d41c not found: ID does not exist"
Oct 30 20:22:48.457454 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:48.457432    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-981ee5-dbb54d47d-sr5mp"]
Oct 30 20:22:48.461119 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:48.461100    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-981ee5-dbb54d47d-sr5mp"]
Oct 30 20:22:48.792084 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:48.792004    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="035ed3f1-789b-4a62-80e6-3f8841905629" path="/var/lib/kubelet/pods/035ed3f1-789b-4a62-80e6-3f8841905629/volumes"
Oct 30 20:22:53.725592 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:53.725552    6996 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-4d5758-6985bf95bc-twp4p" podUID="316882f2-58f3-404a-a515-310efc6f2462" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 20:22:59.726068 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:22:59.726024    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 20:23:00.539576 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:00.539538    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-4d2317-75c7f66df4-4784l"]
Oct 30 20:23:01.206066 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:23:01.206015    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 20:23:01.934977 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:01.934924    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-4d2317-7d6cc8fd8d-ltsxv"]
Oct 30 20:23:01.935294 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:01.935185    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-4d2317-7d6cc8fd8d-ltsxv" podUID="babec534-131b-46e9-9706-cdf7fb2f60c8" containerName="maas-api" containerID="cri-o://5733f8a8774c51cd8af27b5e721ebe34b9ba4d2e870c38698b54fe584c8397d1" gracePeriod=30
Oct 30 20:23:02.154708 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:02.154675    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s"]
Oct 30 20:23:02.252243 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:02.252216    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-4d2317-7d6cc8fd8d-ltsxv"
Oct 30 20:23:02.417838 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:02.417796    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/babec534-131b-46e9-9706-cdf7fb2f60c8-metrics-certs\" (UniqueName: \"kubernetes.io/secret/babec534-131b-46e9-9706-cdf7fb2f60c8-metrics-certs\") pod \"babec534-131b-46e9-9706-cdf7fb2f60c8\" (UID: \"babec534-131b-46e9-9706-cdf7fb2f60c8\") "
Oct 30 20:23:02.418065 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:02.417879    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/babec534-131b-46e9-9706-cdf7fb2f60c8-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/babec534-131b-46e9-9706-cdf7fb2f60c8-openshift-service-ca\") pod \"babec534-131b-46e9-9706-cdf7fb2f60c8\" (UID: \"babec534-131b-46e9-9706-cdf7fb2f60c8\") "
Oct 30 20:23:02.418065 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:02.417973    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/babec534-131b-46e9-9706-cdf7fb2f60c8-kube-api-access-tnsk5\" (UniqueName: \"kubernetes.io/projected/babec534-131b-46e9-9706-cdf7fb2f60c8-kube-api-access-tnsk5\") pod \"babec534-131b-46e9-9706-cdf7fb2f60c8\" (UID: \"babec534-131b-46e9-9706-cdf7fb2f60c8\") "
Oct 30 20:23:02.418065 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:02.418007    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/babec534-131b-46e9-9706-cdf7fb2f60c8-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/babec534-131b-46e9-9706-cdf7fb2f60c8-maas-api-tls\") pod \"babec534-131b-46e9-9706-cdf7fb2f60c8\" (UID: \"babec534-131b-46e9-9706-cdf7fb2f60c8\") "
Oct 30 20:23:02.419157 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:02.419124    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/babec534-131b-46e9-9706-cdf7fb2f60c8-openshift-service-ca" pod "babec534-131b-46e9-9706-cdf7fb2f60c8" (UID: "babec534-131b-46e9-9706-cdf7fb2f60c8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:23:02.421895 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:02.421865    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/babec534-131b-46e9-9706-cdf7fb2f60c8-metrics-certs" pod "babec534-131b-46e9-9706-cdf7fb2f60c8" (UID: "babec534-131b-46e9-9706-cdf7fb2f60c8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:23:02.422030 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:02.421969    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/babec534-131b-46e9-9706-cdf7fb2f60c8-maas-api-tls" pod "babec534-131b-46e9-9706-cdf7fb2f60c8" (UID: "babec534-131b-46e9-9706-cdf7fb2f60c8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:23:02.422428 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:02.422398    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/babec534-131b-46e9-9706-cdf7fb2f60c8-kube-api-access-tnsk5" pod "babec534-131b-46e9-9706-cdf7fb2f60c8" (UID: "babec534-131b-46e9-9706-cdf7fb2f60c8"). InnerVolumeSpecName "kube-api-access-tnsk5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:23:02.514587 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:02.514508    6996 generic.go:356] "Generic (PLEG): container finished" podID="babec534-131b-46e9-9706-cdf7fb2f60c8" containerID="5733f8a8774c51cd8af27b5e721ebe34b9ba4d2e870c38698b54fe584c8397d1" exitCode=0
Oct 30 20:23:02.514587 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:02.514552    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-4d2317-7d6cc8fd8d-ltsxv"
Oct 30 20:23:02.514763 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:02.514596    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-4d2317-7d6cc8fd8d-ltsxv" event={"ID":"babec534-131b-46e9-9706-cdf7fb2f60c8","Type":"ContainerDied","Data":"5733f8a8774c51cd8af27b5e721ebe34b9ba4d2e870c38698b54fe584c8397d1"}
Oct 30 20:23:02.514763 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:02.514650    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-4d2317-7d6cc8fd8d-ltsxv" event={"ID":"babec534-131b-46e9-9706-cdf7fb2f60c8","Type":"ContainerDied","Data":"5cdf80125b39b08b9012b24d1d929c7978e495eb3e9d61168ac9cc6d80954ffa"}
Oct 30 20:23:02.514763 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:02.514674    6996 scope.go:122] "RemoveContainer" containerID="5733f8a8774c51cd8af27b5e721ebe34b9ba4d2e870c38698b54fe584c8397d1"
Oct 30 20:23:02.519256 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:02.519230    6996 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/babec534-131b-46e9-9706-cdf7fb2f60c8-metrics-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:02.519256 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:02.519257    6996 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/babec534-131b-46e9-9706-cdf7fb2f60c8-openshift-service-ca\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:02.519405 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:02.519272    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tnsk5\" (UniqueName: \"kubernetes.io/projected/babec534-131b-46e9-9706-cdf7fb2f60c8-kube-api-access-tnsk5\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:02.519405 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:02.519288    6996 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/babec534-131b-46e9-9706-cdf7fb2f60c8-maas-api-tls\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:02.525429 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:02.525411    6996 scope.go:122] "RemoveContainer" containerID="5733f8a8774c51cd8af27b5e721ebe34b9ba4d2e870c38698b54fe584c8397d1"
Oct 30 20:23:02.525719 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:23:02.525698    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5733f8a8774c51cd8af27b5e721ebe34b9ba4d2e870c38698b54fe584c8397d1\": container with ID starting with 5733f8a8774c51cd8af27b5e721ebe34b9ba4d2e870c38698b54fe584c8397d1 not found: ID does not exist" containerID="5733f8a8774c51cd8af27b5e721ebe34b9ba4d2e870c38698b54fe584c8397d1"
Oct 30 20:23:02.525774 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:02.525725    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5733f8a8774c51cd8af27b5e721ebe34b9ba4d2e870c38698b54fe584c8397d1"} err="failed to get container status \"5733f8a8774c51cd8af27b5e721ebe34b9ba4d2e870c38698b54fe584c8397d1\": rpc error: code = NotFound desc = could not find container \"5733f8a8774c51cd8af27b5e721ebe34b9ba4d2e870c38698b54fe584c8397d1\": container with ID starting with 5733f8a8774c51cd8af27b5e721ebe34b9ba4d2e870c38698b54fe584c8397d1 not found: ID does not exist"
Oct 30 20:23:02.540369 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:02.540347    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-4d2317-7d6cc8fd8d-ltsxv"]
Oct 30 20:23:02.547354 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:02.547325    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-4d2317-7d6cc8fd8d-ltsxv"]
Oct 30 20:23:02.792102 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:02.792025    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="babec534-131b-46e9-9706-cdf7fb2f60c8" path="/var/lib/kubelet/pods/babec534-131b-46e9-9706-cdf7fb2f60c8/volumes"
Oct 30 20:23:03.274298 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:03.274270    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-4d5758-6985bf95bc-twp4p"
Oct 30 20:23:03.429637 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:03.429542    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/316882f2-58f3-404a-a515-310efc6f2462-kube-api-access-nbmnf\" (UniqueName: \"kubernetes.io/projected/316882f2-58f3-404a-a515-310efc6f2462-kube-api-access-nbmnf\") pod \"316882f2-58f3-404a-a515-310efc6f2462\" (UID: \"316882f2-58f3-404a-a515-310efc6f2462\") "
Oct 30 20:23:03.429824 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:03.429685    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/316882f2-58f3-404a-a515-310efc6f2462-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/316882f2-58f3-404a-a515-310efc6f2462-plugins-config-volume\") pod \"316882f2-58f3-404a-a515-310efc6f2462\" (UID: \"316882f2-58f3-404a-a515-310efc6f2462\") "
Oct 30 20:23:03.430046 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:03.430023    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/316882f2-58f3-404a-a515-310efc6f2462-plugins-config-volume" pod "316882f2-58f3-404a-a515-310efc6f2462" (UID: "316882f2-58f3-404a-a515-310efc6f2462"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:23:03.431724 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:03.431698    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/316882f2-58f3-404a-a515-310efc6f2462-kube-api-access-nbmnf" pod "316882f2-58f3-404a-a515-310efc6f2462" (UID: "316882f2-58f3-404a-a515-310efc6f2462"). InnerVolumeSpecName "kube-api-access-nbmnf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:23:03.523378 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:03.523341    6996 generic.go:356] "Generic (PLEG): container finished" podID="316882f2-58f3-404a-a515-310efc6f2462" containerID="5e03fe9f19e192b5b4a3d1cec291e20f83467107868ac65fc820e1ffd10c34c6" exitCode=0
Oct 30 20:23:03.523575 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:03.523398    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-4d5758-6985bf95bc-twp4p"
Oct 30 20:23:03.523575 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:03.523429    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-4d5758-6985bf95bc-twp4p" event={"ID":"316882f2-58f3-404a-a515-310efc6f2462","Type":"ContainerDied","Data":"5e03fe9f19e192b5b4a3d1cec291e20f83467107868ac65fc820e1ffd10c34c6"}
Oct 30 20:23:03.523575 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:03.523526    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-4d5758-6985bf95bc-twp4p" event={"ID":"316882f2-58f3-404a-a515-310efc6f2462","Type":"ContainerDied","Data":"cb1acc42cd62b2cd63a762ce76ca5cff98716c3db71d24ebe540434547e821b4"}
Oct 30 20:23:03.523575 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:03.523549    6996 scope.go:122] "RemoveContainer" containerID="5e03fe9f19e192b5b4a3d1cec291e20f83467107868ac65fc820e1ffd10c34c6"
Oct 30 20:23:03.530619 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:03.530596    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nbmnf\" (UniqueName: \"kubernetes.io/projected/316882f2-58f3-404a-a515-310efc6f2462-kube-api-access-nbmnf\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:03.530727 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:03.530619    6996 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/316882f2-58f3-404a-a515-310efc6f2462-plugins-config-volume\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:03.533656 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:03.533637    6996 scope.go:122] "RemoveContainer" containerID="5e03fe9f19e192b5b4a3d1cec291e20f83467107868ac65fc820e1ffd10c34c6"
Oct 30 20:23:03.533918 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:23:03.533896    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e03fe9f19e192b5b4a3d1cec291e20f83467107868ac65fc820e1ffd10c34c6\": container with ID starting with 5e03fe9f19e192b5b4a3d1cec291e20f83467107868ac65fc820e1ffd10c34c6 not found: ID does not exist" containerID="5e03fe9f19e192b5b4a3d1cec291e20f83467107868ac65fc820e1ffd10c34c6"
Oct 30 20:23:03.534013 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:03.533927    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e03fe9f19e192b5b4a3d1cec291e20f83467107868ac65fc820e1ffd10c34c6"} err="failed to get container status \"5e03fe9f19e192b5b4a3d1cec291e20f83467107868ac65fc820e1ffd10c34c6\": rpc error: code = NotFound desc = could not find container \"5e03fe9f19e192b5b4a3d1cec291e20f83467107868ac65fc820e1ffd10c34c6\": container with ID starting with 5e03fe9f19e192b5b4a3d1cec291e20f83467107868ac65fc820e1ffd10c34c6 not found: ID does not exist"
Oct 30 20:23:03.547168 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:03.547144    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-4d5758-6985bf95bc-twp4p"]
Oct 30 20:23:03.552926 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:03.552905    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-4d5758-6985bf95bc-twp4p"]
Oct 30 20:23:04.791778 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:04.791746    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="316882f2-58f3-404a-a515-310efc6f2462" path="/var/lib/kubelet/pods/316882f2-58f3-404a-a515-310efc6f2462/volumes"
Oct 30 20:23:05.540802 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:05.540766    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-4d2317-75c7f66df4-4784l" podUID="f6e147bc-ed53-4e12-b8f3-23a9e0ab12e2" containerName="payload-processing" containerID="cri-o://72d820e41dcd9bed1cd811b9cc99af216e115f32384a4a5db944bedf663acc08" gracePeriod=25
Oct 30 20:23:05.800780 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:05.800683    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-4d2317-75c7f66df4-4784l"
Oct 30 20:23:05.853830 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:05.853797    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f6e147bc-ed53-4e12-b8f3-23a9e0ab12e2-kube-api-access-rd6j7\" (UniqueName: \"kubernetes.io/projected/f6e147bc-ed53-4e12-b8f3-23a9e0ab12e2-kube-api-access-rd6j7\") pod \"f6e147bc-ed53-4e12-b8f3-23a9e0ab12e2\" (UID: \"f6e147bc-ed53-4e12-b8f3-23a9e0ab12e2\") "
Oct 30 20:23:05.853995 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:05.853938    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f6e147bc-ed53-4e12-b8f3-23a9e0ab12e2-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6e147bc-ed53-4e12-b8f3-23a9e0ab12e2-plugins-config-volume\") pod \"f6e147bc-ed53-4e12-b8f3-23a9e0ab12e2\" (UID: \"f6e147bc-ed53-4e12-b8f3-23a9e0ab12e2\") "
Oct 30 20:23:05.854421 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:05.854381    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6e147bc-ed53-4e12-b8f3-23a9e0ab12e2-plugins-config-volume" pod "f6e147bc-ed53-4e12-b8f3-23a9e0ab12e2" (UID: "f6e147bc-ed53-4e12-b8f3-23a9e0ab12e2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:23:05.856028 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:05.856000    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6e147bc-ed53-4e12-b8f3-23a9e0ab12e2-kube-api-access-rd6j7" pod "f6e147bc-ed53-4e12-b8f3-23a9e0ab12e2" (UID: "f6e147bc-ed53-4e12-b8f3-23a9e0ab12e2"). InnerVolumeSpecName "kube-api-access-rd6j7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:23:05.955389 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:05.955352    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rd6j7\" (UniqueName: \"kubernetes.io/projected/f6e147bc-ed53-4e12-b8f3-23a9e0ab12e2-kube-api-access-rd6j7\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:05.955389 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:05.955385    6996 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6e147bc-ed53-4e12-b8f3-23a9e0ab12e2-plugins-config-volume\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:06.196520 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:06.192888    6996 scope.go:122] "RemoveContainer" containerID="72d820e41dcd9bed1cd811b9cc99af216e115f32384a4a5db944bedf663acc08"
Oct 30 20:23:06.541867 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:06.541777    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-4d2317-75c7f66df4-4784l" event={"ID":"f6e147bc-ed53-4e12-b8f3-23a9e0ab12e2","Type":"ContainerDied","Data":"72d820e41dcd9bed1cd811b9cc99af216e115f32384a4a5db944bedf663acc08"}
Oct 30 20:23:06.541867 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:06.541835    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-4d2317-75c7f66df4-4784l" event={"ID":"f6e147bc-ed53-4e12-b8f3-23a9e0ab12e2","Type":"ContainerDied","Data":"5351e9a7cfa46714bc2c0e309be7eb6577e5f0b5cec4c048cbe7e9c2b83ed640"}
Oct 30 20:23:06.541867 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:06.541836    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-4d2317-75c7f66df4-4784l"
Oct 30 20:23:06.585207 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:06.585166    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-4d2317-75c7f66df4-4784l"]
Oct 30 20:23:06.588426 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:06.588407    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-4d2317-75c7f66df4-4784l"]
Oct 30 20:23:06.790729 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:06.790693    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f6e147bc-ed53-4e12-b8f3-23a9e0ab12e2" path="/var/lib/kubelet/pods/f6e147bc-ed53-4e12-b8f3-23a9e0ab12e2/volumes"
Oct 30 20:23:10.265696 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:10.265655    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 20:23:17.203200 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:17.203099    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s" podUID="ae905dfb-f30d-480a-899b-ea676abc1d8d" containerName="main" containerID="cri-o://ff871009009b34989b690b556cdb584fb6ca5e78b6839c1f8d5b9bd8dda8455b" gracePeriod=45
Oct 30 20:23:17.478704 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:17.478671    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s"
Oct 30 20:23:17.597072 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:17.597032    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ae905dfb-f30d-480a-899b-ea676abc1d8d-kube-api-access-c2mgk\" (UniqueName: \"kubernetes.io/projected/ae905dfb-f30d-480a-899b-ea676abc1d8d-kube-api-access-c2mgk\") pod \"ae905dfb-f30d-480a-899b-ea676abc1d8d\" (UID: \"ae905dfb-f30d-480a-899b-ea676abc1d8d\") "
Oct 30 20:23:17.597277 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:17.597105    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ae905dfb-f30d-480a-899b-ea676abc1d8d-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae905dfb-f30d-480a-899b-ea676abc1d8d-tmp-dir\") pod \"ae905dfb-f30d-480a-899b-ea676abc1d8d\" (UID: \"ae905dfb-f30d-480a-899b-ea676abc1d8d\") "
Oct 30 20:23:17.597277 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:17.597136    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ae905dfb-f30d-480a-899b-ea676abc1d8d-dshm\" (UniqueName: \"kubernetes.io/empty-dir/ae905dfb-f30d-480a-899b-ea676abc1d8d-dshm\") pod \"ae905dfb-f30d-480a-899b-ea676abc1d8d\" (UID: \"ae905dfb-f30d-480a-899b-ea676abc1d8d\") "
Oct 30 20:23:17.597277 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:17.597177    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ae905dfb-f30d-480a-899b-ea676abc1d8d-home\" (UniqueName: \"kubernetes.io/empty-dir/ae905dfb-f30d-480a-899b-ea676abc1d8d-home\") pod \"ae905dfb-f30d-480a-899b-ea676abc1d8d\" (UID: \"ae905dfb-f30d-480a-899b-ea676abc1d8d\") "
Oct 30 20:23:17.597460 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:17.597275    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ae905dfb-f30d-480a-899b-ea676abc1d8d-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ae905dfb-f30d-480a-899b-ea676abc1d8d-model-cache\") pod \"ae905dfb-f30d-480a-899b-ea676abc1d8d\" (UID: \"ae905dfb-f30d-480a-899b-ea676abc1d8d\") "
Oct 30 20:23:17.597460 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:17.597353    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ae905dfb-f30d-480a-899b-ea676abc1d8d-tls-certs\" (UniqueName: \"kubernetes.io/secret/ae905dfb-f30d-480a-899b-ea676abc1d8d-tls-certs\") pod \"ae905dfb-f30d-480a-899b-ea676abc1d8d\" (UID: \"ae905dfb-f30d-480a-899b-ea676abc1d8d\") "
Oct 30 20:23:17.597991 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:17.597830    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae905dfb-f30d-480a-899b-ea676abc1d8d-tmp-dir" pod "ae905dfb-f30d-480a-899b-ea676abc1d8d" (UID: "ae905dfb-f30d-480a-899b-ea676abc1d8d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:23:17.597991 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:17.597904    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae905dfb-f30d-480a-899b-ea676abc1d8d-model-cache" pod "ae905dfb-f30d-480a-899b-ea676abc1d8d" (UID: "ae905dfb-f30d-480a-899b-ea676abc1d8d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:23:17.597991 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:17.597966    6996 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae905dfb-f30d-480a-899b-ea676abc1d8d-tmp-dir\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:17.598297 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:17.598101    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae905dfb-f30d-480a-899b-ea676abc1d8d-home" pod "ae905dfb-f30d-480a-899b-ea676abc1d8d" (UID: "ae905dfb-f30d-480a-899b-ea676abc1d8d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:23:17.599929 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:17.599890    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae905dfb-f30d-480a-899b-ea676abc1d8d-dshm" pod "ae905dfb-f30d-480a-899b-ea676abc1d8d" (UID: "ae905dfb-f30d-480a-899b-ea676abc1d8d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:23:17.600649 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:17.600616    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae905dfb-f30d-480a-899b-ea676abc1d8d-kube-api-access-c2mgk" pod "ae905dfb-f30d-480a-899b-ea676abc1d8d" (UID: "ae905dfb-f30d-480a-899b-ea676abc1d8d"). InnerVolumeSpecName "kube-api-access-c2mgk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:23:17.600773 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:17.600676    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae905dfb-f30d-480a-899b-ea676abc1d8d-tls-certs" pod "ae905dfb-f30d-480a-899b-ea676abc1d8d" (UID: "ae905dfb-f30d-480a-899b-ea676abc1d8d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:23:17.609522 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:17.609496    6996 generic.go:356] "Generic (PLEG): container finished" podID="ae905dfb-f30d-480a-899b-ea676abc1d8d" containerID="ff871009009b34989b690b556cdb584fb6ca5e78b6839c1f8d5b9bd8dda8455b" exitCode=0
Oct 30 20:23:17.609672 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:17.609654    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s" event={"ID":"ae905dfb-f30d-480a-899b-ea676abc1d8d","Type":"ContainerDied","Data":"ff871009009b34989b690b556cdb584fb6ca5e78b6839c1f8d5b9bd8dda8455b"}
Oct 30 20:23:17.609744 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:17.609696    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s" event={"ID":"ae905dfb-f30d-480a-899b-ea676abc1d8d","Type":"ContainerDied","Data":"de12172389f6110cc3f1f4ea9c1dba85538360bcfed66d08dd6103098ffbfb41"}
Oct 30 20:23:17.609744 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:17.609722    6996 scope.go:122] "RemoveContainer" containerID="ff871009009b34989b690b556cdb584fb6ca5e78b6839c1f8d5b9bd8dda8455b"
Oct 30 20:23:17.609921 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:17.609882    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s"
Oct 30 20:23:17.626615 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:17.626590    6996 scope.go:122] "RemoveContainer" containerID="ff871009009b34989b690b556cdb584fb6ca5e78b6839c1f8d5b9bd8dda8455b"
Oct 30 20:23:17.626983 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:23:17.626959    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff871009009b34989b690b556cdb584fb6ca5e78b6839c1f8d5b9bd8dda8455b\": container with ID starting with ff871009009b34989b690b556cdb584fb6ca5e78b6839c1f8d5b9bd8dda8455b not found: ID does not exist" containerID="ff871009009b34989b690b556cdb584fb6ca5e78b6839c1f8d5b9bd8dda8455b"
Oct 30 20:23:17.627071 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:17.626992    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff871009009b34989b690b556cdb584fb6ca5e78b6839c1f8d5b9bd8dda8455b"} err="failed to get container status \"ff871009009b34989b690b556cdb584fb6ca5e78b6839c1f8d5b9bd8dda8455b\": rpc error: code = NotFound desc = could not find container \"ff871009009b34989b690b556cdb584fb6ca5e78b6839c1f8d5b9bd8dda8455b\": container with ID starting with ff871009009b34989b690b556cdb584fb6ca5e78b6839c1f8d5b9bd8dda8455b not found: ID does not exist"
Oct 30 20:23:17.645321 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:17.645289    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s"]
Oct 30 20:23:17.649163 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:17.649138    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-7a745c/premium-w1-7a745c-kserve-c584476d-qvm9s"]
Oct 30 20:23:17.698655 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:17.698620    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c2mgk\" (UniqueName: \"kubernetes.io/projected/ae905dfb-f30d-480a-899b-ea676abc1d8d-kube-api-access-c2mgk\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:17.698655 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:17.698653    6996 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ae905dfb-f30d-480a-899b-ea676abc1d8d-dshm\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:17.698655 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:17.698663    6996 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ae905dfb-f30d-480a-899b-ea676abc1d8d-home\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:17.698909 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:17.698672    6996 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ae905dfb-f30d-480a-899b-ea676abc1d8d-model-cache\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:17.698909 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:17.698682    6996 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae905dfb-f30d-480a-899b-ea676abc1d8d-tls-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:18.790723 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:18.790687    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ae905dfb-f30d-480a-899b-ea676abc1d8d" path="/var/lib/kubelet/pods/ae905dfb-f30d-480a-899b-ea676abc1d8d/volumes"
Oct 30 20:23:21.555830 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:21.555794    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-7a745c-74784c5fb8-mgg8p"]
Oct 30 20:23:21.556370 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:21.556024    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-7a745c-74784c5fb8-mgg8p" podUID="bc6b5019-f6ed-44c3-8390-8525a8e75910" containerName="maas-api" containerID="cri-o://a049874ece5fdc772f649a9e9eed6f0c90269bec1920b395e22928b1d42eb155" gracePeriod=30
Oct 30 20:23:21.842857 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:21.842826    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-7a745c-74784c5fb8-mgg8p"
Oct 30 20:23:21.944453 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:21.944416    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bc6b5019-f6ed-44c3-8390-8525a8e75910-metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc6b5019-f6ed-44c3-8390-8525a8e75910-metrics-certs\") pod \"bc6b5019-f6ed-44c3-8390-8525a8e75910\" (UID: \"bc6b5019-f6ed-44c3-8390-8525a8e75910\") "
Oct 30 20:23:21.944639 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:21.944596    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bc6b5019-f6ed-44c3-8390-8525a8e75910-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc6b5019-f6ed-44c3-8390-8525a8e75910-maas-api-tls\") pod \"bc6b5019-f6ed-44c3-8390-8525a8e75910\" (UID: \"bc6b5019-f6ed-44c3-8390-8525a8e75910\") "
Oct 30 20:23:21.944707 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:21.944662    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bc6b5019-f6ed-44c3-8390-8525a8e75910-kube-api-access-tg6ws\" (UniqueName: \"kubernetes.io/projected/bc6b5019-f6ed-44c3-8390-8525a8e75910-kube-api-access-tg6ws\") pod \"bc6b5019-f6ed-44c3-8390-8525a8e75910\" (UID: \"bc6b5019-f6ed-44c3-8390-8525a8e75910\") "
Oct 30 20:23:21.944755 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:21.944710    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bc6b5019-f6ed-44c3-8390-8525a8e75910-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc6b5019-f6ed-44c3-8390-8525a8e75910-openshift-service-ca\") pod \"bc6b5019-f6ed-44c3-8390-8525a8e75910\" (UID: \"bc6b5019-f6ed-44c3-8390-8525a8e75910\") "
Oct 30 20:23:21.945222 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:21.945190    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc6b5019-f6ed-44c3-8390-8525a8e75910-openshift-service-ca" pod "bc6b5019-f6ed-44c3-8390-8525a8e75910" (UID: "bc6b5019-f6ed-44c3-8390-8525a8e75910"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:23:21.946741 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:21.946709    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc6b5019-f6ed-44c3-8390-8525a8e75910-kube-api-access-tg6ws" pod "bc6b5019-f6ed-44c3-8390-8525a8e75910" (UID: "bc6b5019-f6ed-44c3-8390-8525a8e75910"). InnerVolumeSpecName "kube-api-access-tg6ws". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:23:21.946741 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:21.946718    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc6b5019-f6ed-44c3-8390-8525a8e75910-maas-api-tls" pod "bc6b5019-f6ed-44c3-8390-8525a8e75910" (UID: "bc6b5019-f6ed-44c3-8390-8525a8e75910"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:23:21.946922 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:21.946802    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc6b5019-f6ed-44c3-8390-8525a8e75910-metrics-certs" pod "bc6b5019-f6ed-44c3-8390-8525a8e75910" (UID: "bc6b5019-f6ed-44c3-8390-8525a8e75910"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:23:22.046039 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:22.045998    6996 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc6b5019-f6ed-44c3-8390-8525a8e75910-maas-api-tls\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:22.046039 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:22.046040    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tg6ws\" (UniqueName: \"kubernetes.io/projected/bc6b5019-f6ed-44c3-8390-8525a8e75910-kube-api-access-tg6ws\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:22.046246 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:22.046058    6996 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc6b5019-f6ed-44c3-8390-8525a8e75910-openshift-service-ca\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:22.046246 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:22.046073    6996 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc6b5019-f6ed-44c3-8390-8525a8e75910-metrics-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:22.638296 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:22.638259    6996 generic.go:356] "Generic (PLEG): container finished" podID="bc6b5019-f6ed-44c3-8390-8525a8e75910" containerID="a049874ece5fdc772f649a9e9eed6f0c90269bec1920b395e22928b1d42eb155" exitCode=0
Oct 30 20:23:22.638296 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:22.638306    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-7a745c-74784c5fb8-mgg8p"
Oct 30 20:23:22.638917 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:22.638344    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-7a745c-74784c5fb8-mgg8p" event={"ID":"bc6b5019-f6ed-44c3-8390-8525a8e75910","Type":"ContainerDied","Data":"a049874ece5fdc772f649a9e9eed6f0c90269bec1920b395e22928b1d42eb155"}
Oct 30 20:23:22.638917 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:22.638415    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-7a745c-74784c5fb8-mgg8p" event={"ID":"bc6b5019-f6ed-44c3-8390-8525a8e75910","Type":"ContainerDied","Data":"316bcd70db29ebea51b8a2a2a89ff9fe0358a654047561abe33aa56d7eed45db"}
Oct 30 20:23:22.638917 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:22.638448    6996 scope.go:122] "RemoveContainer" containerID="a049874ece5fdc772f649a9e9eed6f0c90269bec1920b395e22928b1d42eb155"
Oct 30 20:23:22.648807 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:22.648788    6996 scope.go:122] "RemoveContainer" containerID="a049874ece5fdc772f649a9e9eed6f0c90269bec1920b395e22928b1d42eb155"
Oct 30 20:23:22.649142 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:23:22.649114    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a049874ece5fdc772f649a9e9eed6f0c90269bec1920b395e22928b1d42eb155\": container with ID starting with a049874ece5fdc772f649a9e9eed6f0c90269bec1920b395e22928b1d42eb155 not found: ID does not exist" containerID="a049874ece5fdc772f649a9e9eed6f0c90269bec1920b395e22928b1d42eb155"
Oct 30 20:23:22.649275 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:22.649147    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a049874ece5fdc772f649a9e9eed6f0c90269bec1920b395e22928b1d42eb155"} err="failed to get container status \"a049874ece5fdc772f649a9e9eed6f0c90269bec1920b395e22928b1d42eb155\": rpc error: code = NotFound desc = could not find container \"a049874ece5fdc772f649a9e9eed6f0c90269bec1920b395e22928b1d42eb155\": container with ID starting with a049874ece5fdc772f649a9e9eed6f0c90269bec1920b395e22928b1d42eb155 not found: ID does not exist"
Oct 30 20:23:22.666406 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:22.666374    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-7a745c-74784c5fb8-mgg8p"]
Oct 30 20:23:22.684216 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:22.684184    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-7a745c-74784c5fb8-mgg8p"]
Oct 30 20:23:22.791066 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:22.791028    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bc6b5019-f6ed-44c3-8390-8525a8e75910" path="/var/lib/kubelet/pods/bc6b5019-f6ed-44c3-8390-8525a8e75910/volumes"
Oct 30 20:23:26.544425 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:26.544338    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 20:23:29.935735 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:29.935693    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv"]
Oct 30 20:23:29.936149 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:29.935917    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv" podUID="b55d9c75-1365-4bac-852e-326f4380cca7" containerName="istio-proxy" containerID="cri-o://2c1eeb7a523b74593fc04116358c713dacee6477b1fe5c487c1ac55cf8907c6c" gracePeriod=30
Oct 30 20:23:31.191090 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:23:31.191046    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 20:23:33.269983 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:33.267709    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8"]
Oct 30 20:23:33.269983 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:33.268165    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8" podUID="a98a3fe9-bb1b-4a7f-af39-8486b0053f39" containerName="istio-proxy" containerID="cri-o://0a3b23b37f92712ac1f5d1951e138d3eaeee7b2af6d12e8be9f96126b1e7b811" gracePeriod=30
Oct 30 20:23:34.164700 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:34.164660    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 20:23:35.201665 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:35.201632    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv"
Oct 30 20:23:35.302973 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:35.302884    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b55d9c75-1365-4bac-852e-326f4380cca7-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b55d9c75-1365-4bac-852e-326f4380cca7-workload-certs\") pod \"b55d9c75-1365-4bac-852e-326f4380cca7\" (UID: \"b55d9c75-1365-4bac-852e-326f4380cca7\") "
Oct 30 20:23:35.302973 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:35.302948    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b55d9c75-1365-4bac-852e-326f4380cca7-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b55d9c75-1365-4bac-852e-326f4380cca7-istio-podinfo\") pod \"b55d9c75-1365-4bac-852e-326f4380cca7\" (UID: \"b55d9c75-1365-4bac-852e-326f4380cca7\") "
Oct 30 20:23:35.303210 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:35.302994    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b55d9c75-1365-4bac-852e-326f4380cca7-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b55d9c75-1365-4bac-852e-326f4380cca7-istiod-ca-cert\") pod \"b55d9c75-1365-4bac-852e-326f4380cca7\" (UID: \"b55d9c75-1365-4bac-852e-326f4380cca7\") "
Oct 30 20:23:35.303210 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:35.303040    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b55d9c75-1365-4bac-852e-326f4380cca7-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b55d9c75-1365-4bac-852e-326f4380cca7-istio-envoy\") pod \"b55d9c75-1365-4bac-852e-326f4380cca7\" (UID: \"b55d9c75-1365-4bac-852e-326f4380cca7\") "
Oct 30 20:23:35.303210 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:35.303064    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b55d9c75-1365-4bac-852e-326f4380cca7-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b55d9c75-1365-4bac-852e-326f4380cca7-istio-data\") pod \"b55d9c75-1365-4bac-852e-326f4380cca7\" (UID: \"b55d9c75-1365-4bac-852e-326f4380cca7\") "
Oct 30 20:23:35.303210 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:35.303090    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b55d9c75-1365-4bac-852e-326f4380cca7-istio-token\" (UniqueName: \"kubernetes.io/projected/b55d9c75-1365-4bac-852e-326f4380cca7-istio-token\") pod \"b55d9c75-1365-4bac-852e-326f4380cca7\" (UID: \"b55d9c75-1365-4bac-852e-326f4380cca7\") "
Oct 30 20:23:35.303210 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:35.303127    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b55d9c75-1365-4bac-852e-326f4380cca7-kube-api-access-kpfh7\" (UniqueName: \"kubernetes.io/projected/b55d9c75-1365-4bac-852e-326f4380cca7-kube-api-access-kpfh7\") pod \"b55d9c75-1365-4bac-852e-326f4380cca7\" (UID: \"b55d9c75-1365-4bac-852e-326f4380cca7\") "
Oct 30 20:23:35.303210 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:35.303171    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b55d9c75-1365-4bac-852e-326f4380cca7-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b55d9c75-1365-4bac-852e-326f4380cca7-workload-socket\") pod \"b55d9c75-1365-4bac-852e-326f4380cca7\" (UID: \"b55d9c75-1365-4bac-852e-326f4380cca7\") "
Oct 30 20:23:35.303581 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:35.303218    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b55d9c75-1365-4bac-852e-326f4380cca7-workload-certs" pod "b55d9c75-1365-4bac-852e-326f4380cca7" (UID: "b55d9c75-1365-4bac-852e-326f4380cca7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:23:35.303581 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:35.303253    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b55d9c75-1365-4bac-852e-326f4380cca7-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b55d9c75-1365-4bac-852e-326f4380cca7-credential-socket\") pod \"b55d9c75-1365-4bac-852e-326f4380cca7\" (UID: \"b55d9c75-1365-4bac-852e-326f4380cca7\") "
Oct 30 20:23:35.303581 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:35.303531    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b55d9c75-1365-4bac-852e-326f4380cca7-workload-socket" pod "b55d9c75-1365-4bac-852e-326f4380cca7" (UID: "b55d9c75-1365-4bac-852e-326f4380cca7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:23:35.303758 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:35.303624    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b55d9c75-1365-4bac-852e-326f4380cca7-istio-data" pod "b55d9c75-1365-4bac-852e-326f4380cca7" (UID: "b55d9c75-1365-4bac-852e-326f4380cca7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:23:35.303758 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:35.303735    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b55d9c75-1365-4bac-852e-326f4380cca7-credential-socket" pod "b55d9c75-1365-4bac-852e-326f4380cca7" (UID: "b55d9c75-1365-4bac-852e-326f4380cca7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:23:35.303876 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:35.303841    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b55d9c75-1365-4bac-852e-326f4380cca7-istiod-ca-cert" pod "b55d9c75-1365-4bac-852e-326f4380cca7" (UID: "b55d9c75-1365-4bac-852e-326f4380cca7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:23:35.303876 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:35.303859    6996 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b55d9c75-1365-4bac-852e-326f4380cca7-workload-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:35.303986 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:35.303877    6996 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b55d9c75-1365-4bac-852e-326f4380cca7-istio-data\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:35.303986 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:35.303891    6996 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b55d9c75-1365-4bac-852e-326f4380cca7-workload-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:35.303986 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:35.303905    6996 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b55d9c75-1365-4bac-852e-326f4380cca7-credential-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:35.305364 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:35.305329    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b55d9c75-1365-4bac-852e-326f4380cca7-istio-podinfo" pod "b55d9c75-1365-4bac-852e-326f4380cca7" (UID: "b55d9c75-1365-4bac-852e-326f4380cca7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 20:23:35.305577 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:35.305446    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b55d9c75-1365-4bac-852e-326f4380cca7-kube-api-access-kpfh7" pod "b55d9c75-1365-4bac-852e-326f4380cca7" (UID: "b55d9c75-1365-4bac-852e-326f4380cca7"). InnerVolumeSpecName "kube-api-access-kpfh7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:23:35.305759 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:35.305743    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b55d9c75-1365-4bac-852e-326f4380cca7-istio-token" pod "b55d9c75-1365-4bac-852e-326f4380cca7" (UID: "b55d9c75-1365-4bac-852e-326f4380cca7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:23:35.305759 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:35.305748    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b55d9c75-1365-4bac-852e-326f4380cca7-istio-envoy" pod "b55d9c75-1365-4bac-852e-326f4380cca7" (UID: "b55d9c75-1365-4bac-852e-326f4380cca7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:23:35.405161 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:35.405102    6996 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b55d9c75-1365-4bac-852e-326f4380cca7-istio-podinfo\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:35.405161 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:35.405154    6996 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b55d9c75-1365-4bac-852e-326f4380cca7-istiod-ca-cert\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:35.405161 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:35.405164    6996 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b55d9c75-1365-4bac-852e-326f4380cca7-istio-envoy\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:35.405161 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:35.405172    6996 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b55d9c75-1365-4bac-852e-326f4380cca7-istio-token\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:35.405161 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:35.405182    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kpfh7\" (UniqueName: \"kubernetes.io/projected/b55d9c75-1365-4bac-852e-326f4380cca7-kube-api-access-kpfh7\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:35.715677 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:35.715641    6996 generic.go:356] "Generic (PLEG): container finished" podID="b55d9c75-1365-4bac-852e-326f4380cca7" containerID="2c1eeb7a523b74593fc04116358c713dacee6477b1fe5c487c1ac55cf8907c6c" exitCode=0
Oct 30 20:23:35.715886 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:35.715696    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv" event={"ID":"b55d9c75-1365-4bac-852e-326f4380cca7","Type":"ContainerDied","Data":"2c1eeb7a523b74593fc04116358c713dacee6477b1fe5c487c1ac55cf8907c6c"}
Oct 30 20:23:35.715886 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:35.715734    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv" event={"ID":"b55d9c75-1365-4bac-852e-326f4380cca7","Type":"ContainerDied","Data":"ff0cb40db78fdab142becb97060df1921d6522cbaa02771830c8fb557a96d833"}
Oct 30 20:23:35.715886 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:35.715755    6996 scope.go:122] "RemoveContainer" containerID="2c1eeb7a523b74593fc04116358c713dacee6477b1fe5c487c1ac55cf8907c6c"
Oct 30 20:23:35.715886 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:35.715695    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv"
Oct 30 20:23:35.726154 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:35.726137    6996 scope.go:122] "RemoveContainer" containerID="2c1eeb7a523b74593fc04116358c713dacee6477b1fe5c487c1ac55cf8907c6c"
Oct 30 20:23:35.726448 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:23:35.726429    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c1eeb7a523b74593fc04116358c713dacee6477b1fe5c487c1ac55cf8907c6c\": container with ID starting with 2c1eeb7a523b74593fc04116358c713dacee6477b1fe5c487c1ac55cf8907c6c not found: ID does not exist" containerID="2c1eeb7a523b74593fc04116358c713dacee6477b1fe5c487c1ac55cf8907c6c"
Oct 30 20:23:35.726520 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:35.726458    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c1eeb7a523b74593fc04116358c713dacee6477b1fe5c487c1ac55cf8907c6c"} err="failed to get container status \"2c1eeb7a523b74593fc04116358c713dacee6477b1fe5c487c1ac55cf8907c6c\": rpc error: code = NotFound desc = could not find container \"2c1eeb7a523b74593fc04116358c713dacee6477b1fe5c487c1ac55cf8907c6c\": container with ID starting with 2c1eeb7a523b74593fc04116358c713dacee6477b1fe5c487c1ac55cf8907c6c not found: ID does not exist"
Oct 30 20:23:35.751912 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:35.751879    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv"]
Oct 30 20:23:35.759860 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:35.759826    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-7a745c-openshift-default-57b8b749-6c4cv"]
Oct 30 20:23:36.791218 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:36.791187    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b55d9c75-1365-4bac-852e-326f4380cca7" path="/var/lib/kubelet/pods/b55d9c75-1365-4bac-852e-326f4380cca7/volumes"
Oct 30 20:23:38.531029 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:38.531000    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8"
Oct 30 20:23:38.640837 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:38.640802    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a98a3fe9-bb1b-4a7f-af39-8486b0053f39-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a98a3fe9-bb1b-4a7f-af39-8486b0053f39-workload-socket\") pod \"a98a3fe9-bb1b-4a7f-af39-8486b0053f39\" (UID: \"a98a3fe9-bb1b-4a7f-af39-8486b0053f39\") "
Oct 30 20:23:38.640837 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:38.640844    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a98a3fe9-bb1b-4a7f-af39-8486b0053f39-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a98a3fe9-bb1b-4a7f-af39-8486b0053f39-istiod-ca-cert\") pod \"a98a3fe9-bb1b-4a7f-af39-8486b0053f39\" (UID: \"a98a3fe9-bb1b-4a7f-af39-8486b0053f39\") "
Oct 30 20:23:38.641096 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:38.640865    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a98a3fe9-bb1b-4a7f-af39-8486b0053f39-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a98a3fe9-bb1b-4a7f-af39-8486b0053f39-istio-envoy\") pod \"a98a3fe9-bb1b-4a7f-af39-8486b0053f39\" (UID: \"a98a3fe9-bb1b-4a7f-af39-8486b0053f39\") "
Oct 30 20:23:38.641096 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:38.640885    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a98a3fe9-bb1b-4a7f-af39-8486b0053f39-kube-api-access-bn84m\" (UniqueName: \"kubernetes.io/projected/a98a3fe9-bb1b-4a7f-af39-8486b0053f39-kube-api-access-bn84m\") pod \"a98a3fe9-bb1b-4a7f-af39-8486b0053f39\" (UID: \"a98a3fe9-bb1b-4a7f-af39-8486b0053f39\") "
Oct 30 20:23:38.641096 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:38.640914    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a98a3fe9-bb1b-4a7f-af39-8486b0053f39-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a98a3fe9-bb1b-4a7f-af39-8486b0053f39-credential-socket\") pod \"a98a3fe9-bb1b-4a7f-af39-8486b0053f39\" (UID: \"a98a3fe9-bb1b-4a7f-af39-8486b0053f39\") "
Oct 30 20:23:38.641096 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:38.640942    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a98a3fe9-bb1b-4a7f-af39-8486b0053f39-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a98a3fe9-bb1b-4a7f-af39-8486b0053f39-workload-certs\") pod \"a98a3fe9-bb1b-4a7f-af39-8486b0053f39\" (UID: \"a98a3fe9-bb1b-4a7f-af39-8486b0053f39\") "
Oct 30 20:23:38.641096 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:38.641025    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a98a3fe9-bb1b-4a7f-af39-8486b0053f39-istio-token\" (UniqueName: \"kubernetes.io/projected/a98a3fe9-bb1b-4a7f-af39-8486b0053f39-istio-token\") pod \"a98a3fe9-bb1b-4a7f-af39-8486b0053f39\" (UID: \"a98a3fe9-bb1b-4a7f-af39-8486b0053f39\") "
Oct 30 20:23:38.641096 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:38.641055    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a98a3fe9-bb1b-4a7f-af39-8486b0053f39-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a98a3fe9-bb1b-4a7f-af39-8486b0053f39-istio-podinfo\") pod \"a98a3fe9-bb1b-4a7f-af39-8486b0053f39\" (UID: \"a98a3fe9-bb1b-4a7f-af39-8486b0053f39\") "
Oct 30 20:23:38.641096 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:38.641089    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a98a3fe9-bb1b-4a7f-af39-8486b0053f39-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a98a3fe9-bb1b-4a7f-af39-8486b0053f39-istio-data\") pod \"a98a3fe9-bb1b-4a7f-af39-8486b0053f39\" (UID: \"a98a3fe9-bb1b-4a7f-af39-8486b0053f39\") "
Oct 30 20:23:38.641503 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:38.641185    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a98a3fe9-bb1b-4a7f-af39-8486b0053f39-workload-socket" pod "a98a3fe9-bb1b-4a7f-af39-8486b0053f39" (UID: "a98a3fe9-bb1b-4a7f-af39-8486b0053f39"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:23:38.641503 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:38.641353    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a98a3fe9-bb1b-4a7f-af39-8486b0053f39-istiod-ca-cert" pod "a98a3fe9-bb1b-4a7f-af39-8486b0053f39" (UID: "a98a3fe9-bb1b-4a7f-af39-8486b0053f39"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:23:38.641627 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:38.641514    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a98a3fe9-bb1b-4a7f-af39-8486b0053f39-istio-data" pod "a98a3fe9-bb1b-4a7f-af39-8486b0053f39" (UID: "a98a3fe9-bb1b-4a7f-af39-8486b0053f39"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:23:38.641627 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:38.641608    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a98a3fe9-bb1b-4a7f-af39-8486b0053f39-credential-socket" pod "a98a3fe9-bb1b-4a7f-af39-8486b0053f39" (UID: "a98a3fe9-bb1b-4a7f-af39-8486b0053f39"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:23:38.641742 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:38.641681    6996 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a98a3fe9-bb1b-4a7f-af39-8486b0053f39-workload-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:38.641742 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:38.641703    6996 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a98a3fe9-bb1b-4a7f-af39-8486b0053f39-istiod-ca-cert\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:38.641742 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:38.641718    6996 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a98a3fe9-bb1b-4a7f-af39-8486b0053f39-istio-data\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:38.641875 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:38.641740    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a98a3fe9-bb1b-4a7f-af39-8486b0053f39-workload-certs" pod "a98a3fe9-bb1b-4a7f-af39-8486b0053f39" (UID: "a98a3fe9-bb1b-4a7f-af39-8486b0053f39"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:23:38.643364 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:38.643327    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a98a3fe9-bb1b-4a7f-af39-8486b0053f39-istio-envoy" pod "a98a3fe9-bb1b-4a7f-af39-8486b0053f39" (UID: "a98a3fe9-bb1b-4a7f-af39-8486b0053f39"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:23:38.643526 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:38.643375    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a98a3fe9-bb1b-4a7f-af39-8486b0053f39-kube-api-access-bn84m" pod "a98a3fe9-bb1b-4a7f-af39-8486b0053f39" (UID: "a98a3fe9-bb1b-4a7f-af39-8486b0053f39"). InnerVolumeSpecName "kube-api-access-bn84m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:23:38.643645 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:38.643627    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a98a3fe9-bb1b-4a7f-af39-8486b0053f39-istio-token" pod "a98a3fe9-bb1b-4a7f-af39-8486b0053f39" (UID: "a98a3fe9-bb1b-4a7f-af39-8486b0053f39"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:23:38.643717 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:38.643681    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a98a3fe9-bb1b-4a7f-af39-8486b0053f39-istio-podinfo" pod "a98a3fe9-bb1b-4a7f-af39-8486b0053f39" (UID: "a98a3fe9-bb1b-4a7f-af39-8486b0053f39"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 20:23:38.734421 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:38.734386    6996 generic.go:356] "Generic (PLEG): container finished" podID="a98a3fe9-bb1b-4a7f-af39-8486b0053f39" containerID="0a3b23b37f92712ac1f5d1951e138d3eaeee7b2af6d12e8be9f96126b1e7b811" exitCode=0
Oct 30 20:23:38.734638 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:38.734431    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8"
Oct 30 20:23:38.734638 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:38.734440    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8" event={"ID":"a98a3fe9-bb1b-4a7f-af39-8486b0053f39","Type":"ContainerDied","Data":"0a3b23b37f92712ac1f5d1951e138d3eaeee7b2af6d12e8be9f96126b1e7b811"}
Oct 30 20:23:38.734638 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:38.734507    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8" event={"ID":"a98a3fe9-bb1b-4a7f-af39-8486b0053f39","Type":"ContainerDied","Data":"ecd6d3b32e73e84fd34ee2649dbd186f14006292532554ee3e56ebe9bbee1084"}
Oct 30 20:23:38.734638 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:38.734537    6996 scope.go:122] "RemoveContainer" containerID="0a3b23b37f92712ac1f5d1951e138d3eaeee7b2af6d12e8be9f96126b1e7b811"
Oct 30 20:23:38.742515 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:38.742352    6996 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a98a3fe9-bb1b-4a7f-af39-8486b0053f39-istio-envoy\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:38.742515 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:38.742374    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bn84m\" (UniqueName: \"kubernetes.io/projected/a98a3fe9-bb1b-4a7f-af39-8486b0053f39-kube-api-access-bn84m\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:38.742515 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:38.742386    6996 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a98a3fe9-bb1b-4a7f-af39-8486b0053f39-credential-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:38.742515 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:38.742396    6996 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a98a3fe9-bb1b-4a7f-af39-8486b0053f39-workload-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:38.742515 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:38.742405    6996 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a98a3fe9-bb1b-4a7f-af39-8486b0053f39-istio-token\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:38.742515 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:38.742413    6996 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a98a3fe9-bb1b-4a7f-af39-8486b0053f39-istio-podinfo\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:23:38.745281 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:38.745261    6996 scope.go:122] "RemoveContainer" containerID="0a3b23b37f92712ac1f5d1951e138d3eaeee7b2af6d12e8be9f96126b1e7b811"
Oct 30 20:23:38.745602 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:23:38.745578    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a3b23b37f92712ac1f5d1951e138d3eaeee7b2af6d12e8be9f96126b1e7b811\": container with ID starting with 0a3b23b37f92712ac1f5d1951e138d3eaeee7b2af6d12e8be9f96126b1e7b811 not found: ID does not exist" containerID="0a3b23b37f92712ac1f5d1951e138d3eaeee7b2af6d12e8be9f96126b1e7b811"
Oct 30 20:23:38.745668 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:38.745610    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a3b23b37f92712ac1f5d1951e138d3eaeee7b2af6d12e8be9f96126b1e7b811"} err="failed to get container status \"0a3b23b37f92712ac1f5d1951e138d3eaeee7b2af6d12e8be9f96126b1e7b811\": rpc error: code = NotFound desc = could not find container \"0a3b23b37f92712ac1f5d1951e138d3eaeee7b2af6d12e8be9f96126b1e7b811\": container with ID starting with 0a3b23b37f92712ac1f5d1951e138d3eaeee7b2af6d12e8be9f96126b1e7b811 not found: ID does not exist"
Oct 30 20:23:38.759013 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:38.758978    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8"]
Oct 30 20:23:38.766017 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:38.765987    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-75b361-openshift-default-d779c697c-5s7j8"]
Oct 30 20:23:38.791248 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:38.791219    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a98a3fe9-bb1b-4a7f-af39-8486b0053f39" path="/var/lib/kubelet/pods/a98a3fe9-bb1b-4a7f-af39-8486b0053f39/volumes"
Oct 30 20:23:57.255858 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:57.255812    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 20:23:59.943398 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:59.943352    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-8836a8-7fbfb4b97d-x9fs9"]
Oct 30 20:23:59.943943 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:23:59.943681    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8836a8-7fbfb4b97d-x9fs9" podUID="e538f71a-8196-4fad-a86b-4750900feb8c" containerName="maas-api" containerID="cri-o://11bc7eb4258b08d007e332da83f9dc24ef86d3a87ae1810e3c7009e862f11217" gracePeriod=30
Oct 30 20:24:00.218573 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:00.218544    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8836a8-7fbfb4b97d-x9fs9"
Oct 30 20:24:00.371917 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:00.371873    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e538f71a-8196-4fad-a86b-4750900feb8c-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e538f71a-8196-4fad-a86b-4750900feb8c-openshift-service-ca\") pod \"e538f71a-8196-4fad-a86b-4750900feb8c\" (UID: \"e538f71a-8196-4fad-a86b-4750900feb8c\") "
Oct 30 20:24:00.372124 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:00.371931    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e538f71a-8196-4fad-a86b-4750900feb8c-kube-api-access-xjfcg\" (UniqueName: \"kubernetes.io/projected/e538f71a-8196-4fad-a86b-4750900feb8c-kube-api-access-xjfcg\") pod \"e538f71a-8196-4fad-a86b-4750900feb8c\" (UID: \"e538f71a-8196-4fad-a86b-4750900feb8c\") "
Oct 30 20:24:00.372124 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:00.371954    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e538f71a-8196-4fad-a86b-4750900feb8c-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e538f71a-8196-4fad-a86b-4750900feb8c-maas-api-tls\") pod \"e538f71a-8196-4fad-a86b-4750900feb8c\" (UID: \"e538f71a-8196-4fad-a86b-4750900feb8c\") "
Oct 30 20:24:00.372124 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:00.371984    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e538f71a-8196-4fad-a86b-4750900feb8c-metrics-certs\" (UniqueName: \"kubernetes.io/secret/e538f71a-8196-4fad-a86b-4750900feb8c-metrics-certs\") pod \"e538f71a-8196-4fad-a86b-4750900feb8c\" (UID: \"e538f71a-8196-4fad-a86b-4750900feb8c\") "
Oct 30 20:24:00.372869 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:00.372794    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e538f71a-8196-4fad-a86b-4750900feb8c-openshift-service-ca" pod "e538f71a-8196-4fad-a86b-4750900feb8c" (UID: "e538f71a-8196-4fad-a86b-4750900feb8c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:24:00.374832 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:00.374788    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e538f71a-8196-4fad-a86b-4750900feb8c-maas-api-tls" pod "e538f71a-8196-4fad-a86b-4750900feb8c" (UID: "e538f71a-8196-4fad-a86b-4750900feb8c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:24:00.375011 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:00.374989    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e538f71a-8196-4fad-a86b-4750900feb8c-kube-api-access-xjfcg" pod "e538f71a-8196-4fad-a86b-4750900feb8c" (UID: "e538f71a-8196-4fad-a86b-4750900feb8c"). InnerVolumeSpecName "kube-api-access-xjfcg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:24:00.375182 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:00.375157    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e538f71a-8196-4fad-a86b-4750900feb8c-metrics-certs" pod "e538f71a-8196-4fad-a86b-4750900feb8c" (UID: "e538f71a-8196-4fad-a86b-4750900feb8c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:24:00.473199 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:00.473094    6996 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e538f71a-8196-4fad-a86b-4750900feb8c-openshift-service-ca\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:24:00.473199 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:00.473149    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xjfcg\" (UniqueName: \"kubernetes.io/projected/e538f71a-8196-4fad-a86b-4750900feb8c-kube-api-access-xjfcg\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:24:00.473199 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:00.473167    6996 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e538f71a-8196-4fad-a86b-4750900feb8c-maas-api-tls\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:24:00.473199 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:00.473181    6996 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e538f71a-8196-4fad-a86b-4750900feb8c-metrics-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:24:00.860392 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:00.860359    6996 generic.go:356] "Generic (PLEG): container finished" podID="e538f71a-8196-4fad-a86b-4750900feb8c" containerID="11bc7eb4258b08d007e332da83f9dc24ef86d3a87ae1810e3c7009e862f11217" exitCode=0
Oct 30 20:24:00.860620 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:00.860420    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8836a8-7fbfb4b97d-x9fs9"
Oct 30 20:24:00.860620 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:00.860451    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8836a8-7fbfb4b97d-x9fs9" event={"ID":"e538f71a-8196-4fad-a86b-4750900feb8c","Type":"ContainerDied","Data":"11bc7eb4258b08d007e332da83f9dc24ef86d3a87ae1810e3c7009e862f11217"}
Oct 30 20:24:00.860620 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:00.860518    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8836a8-7fbfb4b97d-x9fs9" event={"ID":"e538f71a-8196-4fad-a86b-4750900feb8c","Type":"ContainerDied","Data":"3c213ce7c652e176dccb88562256154852d32b769bb45c52ef5daea655d20c69"}
Oct 30 20:24:00.860620 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:00.860544    6996 scope.go:122] "RemoveContainer" containerID="11bc7eb4258b08d007e332da83f9dc24ef86d3a87ae1810e3c7009e862f11217"
Oct 30 20:24:00.870397 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:00.870368    6996 scope.go:122] "RemoveContainer" containerID="11bc7eb4258b08d007e332da83f9dc24ef86d3a87ae1810e3c7009e862f11217"
Oct 30 20:24:00.870674 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:24:00.870655    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11bc7eb4258b08d007e332da83f9dc24ef86d3a87ae1810e3c7009e862f11217\": container with ID starting with 11bc7eb4258b08d007e332da83f9dc24ef86d3a87ae1810e3c7009e862f11217 not found: ID does not exist" containerID="11bc7eb4258b08d007e332da83f9dc24ef86d3a87ae1810e3c7009e862f11217"
Oct 30 20:24:00.870735 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:00.870682    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11bc7eb4258b08d007e332da83f9dc24ef86d3a87ae1810e3c7009e862f11217"} err="failed to get container status \"11bc7eb4258b08d007e332da83f9dc24ef86d3a87ae1810e3c7009e862f11217\": rpc error: code = NotFound desc = could not find container \"11bc7eb4258b08d007e332da83f9dc24ef86d3a87ae1810e3c7009e862f11217\": container with ID starting with 11bc7eb4258b08d007e332da83f9dc24ef86d3a87ae1810e3c7009e862f11217 not found: ID does not exist"
Oct 30 20:24:00.878220 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:00.878190    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-8836a8-7fbfb4b97d-x9fs9"]
Oct 30 20:24:00.880985 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:00.880964    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-8836a8-7fbfb4b97d-x9fs9"]
Oct 30 20:24:01.197024 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:24:01.196981    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 20:24:02.790211 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:02.790178    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e538f71a-8196-4fad-a86b-4750900feb8c" path="/var/lib/kubelet/pods/e538f71a-8196-4fad-a86b-4750900feb8c/volumes"
Oct 30 20:24:03.078800 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:03.078771    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-2kq47"]
Oct 30 20:24:07.758359 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:07.758318    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp"]
Oct 30 20:24:07.759322 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:07.758583    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp" podUID="cf35d600-7eac-42ff-b1f9-79ad561fea3c" containerName="istio-proxy" containerID="cri-o://61e17e881b70d1f754c6a22b9c7fd84ea43951fd13fae2deb934ddf05866a1c0" gracePeriod=30
Oct 30 20:24:08.881369 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:08.881331    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5677757b6d-m57tz_a8eded1a-5c8c-4740-8f7a-dfc5c05c0bcd/authorino/0.log"
Oct 30 20:24:09.486639 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:09.486609    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-gmgtn_bede9e81-8025-4db4-b21d-91f8c4b233e0/kuadrant-console-plugin/0.log"
Oct 30 20:24:10.335522 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:10.335468    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-2kq47_f1319006-f8c2-4f14-9d3d-e7d44737c884/limitador/0.log"
Oct 30 20:24:11.315567 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:11.315531    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-5c494664d6-7lvd2_50e907ce-5794-4533-9bda-d0e3abaf4e21/manager/0.log"
Oct 30 20:24:12.620033 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:12.620003    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-wwgvc_77b41da1-d9be-4723-886a-fd568313384b/server/0.log"
Oct 30 20:24:12.804070 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:12.804045    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-blshw_4e58facf-d3be-4f40-94b2-13cc119c8651/manager/0.log"
Oct 30 20:24:12.933964 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:12.933937    6996 generic.go:356] "Generic (PLEG): container finished" podID="cf35d600-7eac-42ff-b1f9-79ad561fea3c" containerID="61e17e881b70d1f754c6a22b9c7fd84ea43951fd13fae2deb934ddf05866a1c0" exitCode=0
Oct 30 20:24:12.934100 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:12.934020    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp" event={"ID":"cf35d600-7eac-42ff-b1f9-79ad561fea3c","Type":"ContainerDied","Data":"61e17e881b70d1f754c6a22b9c7fd84ea43951fd13fae2deb934ddf05866a1c0"}
Oct 30 20:24:13.029260 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:13.029234    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp"
Oct 30 20:24:13.095755 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:13.095726    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf35d600-7eac-42ff-b1f9-79ad561fea3c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf35d600-7eac-42ff-b1f9-79ad561fea3c-workload-socket\") pod \"cf35d600-7eac-42ff-b1f9-79ad561fea3c\" (UID: \"cf35d600-7eac-42ff-b1f9-79ad561fea3c\") "
Oct 30 20:24:13.095945 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:13.095765    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf35d600-7eac-42ff-b1f9-79ad561fea3c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf35d600-7eac-42ff-b1f9-79ad561fea3c-istio-envoy\") pod \"cf35d600-7eac-42ff-b1f9-79ad561fea3c\" (UID: \"cf35d600-7eac-42ff-b1f9-79ad561fea3c\") "
Oct 30 20:24:13.095945 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:13.095799    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/cf35d600-7eac-42ff-b1f9-79ad561fea3c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf35d600-7eac-42ff-b1f9-79ad561fea3c-istio-podinfo\") pod \"cf35d600-7eac-42ff-b1f9-79ad561fea3c\" (UID: \"cf35d600-7eac-42ff-b1f9-79ad561fea3c\") "
Oct 30 20:24:13.095945 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:13.095850    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cf35d600-7eac-42ff-b1f9-79ad561fea3c-istio-token\" (UniqueName: \"kubernetes.io/projected/cf35d600-7eac-42ff-b1f9-79ad561fea3c-istio-token\") pod \"cf35d600-7eac-42ff-b1f9-79ad561fea3c\" (UID: \"cf35d600-7eac-42ff-b1f9-79ad561fea3c\") "
Oct 30 20:24:13.095945 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:13.095874    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cf35d600-7eac-42ff-b1f9-79ad561fea3c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf35d600-7eac-42ff-b1f9-79ad561fea3c-istiod-ca-cert\") pod \"cf35d600-7eac-42ff-b1f9-79ad561fea3c\" (UID: \"cf35d600-7eac-42ff-b1f9-79ad561fea3c\") "
Oct 30 20:24:13.095945 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:13.095920    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf35d600-7eac-42ff-b1f9-79ad561fea3c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf35d600-7eac-42ff-b1f9-79ad561fea3c-istio-data\") pod \"cf35d600-7eac-42ff-b1f9-79ad561fea3c\" (UID: \"cf35d600-7eac-42ff-b1f9-79ad561fea3c\") "
Oct 30 20:24:13.096296 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:13.096008    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf35d600-7eac-42ff-b1f9-79ad561fea3c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf35d600-7eac-42ff-b1f9-79ad561fea3c-workload-certs\") pod \"cf35d600-7eac-42ff-b1f9-79ad561fea3c\" (UID: \"cf35d600-7eac-42ff-b1f9-79ad561fea3c\") "
Oct 30 20:24:13.096296 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:13.096039    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf35d600-7eac-42ff-b1f9-79ad561fea3c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf35d600-7eac-42ff-b1f9-79ad561fea3c-credential-socket\") pod \"cf35d600-7eac-42ff-b1f9-79ad561fea3c\" (UID: \"cf35d600-7eac-42ff-b1f9-79ad561fea3c\") "
Oct 30 20:24:13.096296 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:13.096092    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cf35d600-7eac-42ff-b1f9-79ad561fea3c-kube-api-access-7vkgw\" (UniqueName: \"kubernetes.io/projected/cf35d600-7eac-42ff-b1f9-79ad561fea3c-kube-api-access-7vkgw\") pod \"cf35d600-7eac-42ff-b1f9-79ad561fea3c\" (UID: \"cf35d600-7eac-42ff-b1f9-79ad561fea3c\") "
Oct 30 20:24:13.096840 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:13.096470    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf35d600-7eac-42ff-b1f9-79ad561fea3c-istio-data" pod "cf35d600-7eac-42ff-b1f9-79ad561fea3c" (UID: "cf35d600-7eac-42ff-b1f9-79ad561fea3c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:24:13.096840 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:13.096571    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf35d600-7eac-42ff-b1f9-79ad561fea3c-workload-socket" pod "cf35d600-7eac-42ff-b1f9-79ad561fea3c" (UID: "cf35d600-7eac-42ff-b1f9-79ad561fea3c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:24:13.096840 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:13.096597    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf35d600-7eac-42ff-b1f9-79ad561fea3c-workload-certs" pod "cf35d600-7eac-42ff-b1f9-79ad561fea3c" (UID: "cf35d600-7eac-42ff-b1f9-79ad561fea3c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:24:13.096840 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:13.096801    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf35d600-7eac-42ff-b1f9-79ad561fea3c-credential-socket" pod "cf35d600-7eac-42ff-b1f9-79ad561fea3c" (UID: "cf35d600-7eac-42ff-b1f9-79ad561fea3c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:24:13.097033 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:13.096842    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf35d600-7eac-42ff-b1f9-79ad561fea3c-istiod-ca-cert" pod "cf35d600-7eac-42ff-b1f9-79ad561fea3c" (UID: "cf35d600-7eac-42ff-b1f9-79ad561fea3c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:24:13.098117 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:13.098089    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf35d600-7eac-42ff-b1f9-79ad561fea3c-istio-envoy" pod "cf35d600-7eac-42ff-b1f9-79ad561fea3c" (UID: "cf35d600-7eac-42ff-b1f9-79ad561fea3c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:24:13.098498 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:13.098450    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf35d600-7eac-42ff-b1f9-79ad561fea3c-istio-token" pod "cf35d600-7eac-42ff-b1f9-79ad561fea3c" (UID: "cf35d600-7eac-42ff-b1f9-79ad561fea3c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:24:13.098600 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:13.098552    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cf35d600-7eac-42ff-b1f9-79ad561fea3c-istio-podinfo" pod "cf35d600-7eac-42ff-b1f9-79ad561fea3c" (UID: "cf35d600-7eac-42ff-b1f9-79ad561fea3c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 20:24:13.098926 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:13.098898    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf35d600-7eac-42ff-b1f9-79ad561fea3c-kube-api-access-7vkgw" pod "cf35d600-7eac-42ff-b1f9-79ad561fea3c" (UID: "cf35d600-7eac-42ff-b1f9-79ad561fea3c"). InnerVolumeSpecName "kube-api-access-7vkgw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:24:13.197022 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:13.196954    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7vkgw\" (UniqueName: \"kubernetes.io/projected/cf35d600-7eac-42ff-b1f9-79ad561fea3c-kube-api-access-7vkgw\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:24:13.197022 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:13.196979    6996 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf35d600-7eac-42ff-b1f9-79ad561fea3c-workload-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:24:13.197022 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:13.196989    6996 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf35d600-7eac-42ff-b1f9-79ad561fea3c-istio-envoy\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:24:13.197022 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:13.196998    6996 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf35d600-7eac-42ff-b1f9-79ad561fea3c-istio-podinfo\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:24:13.197022 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:13.197006    6996 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf35d600-7eac-42ff-b1f9-79ad561fea3c-istio-token\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:24:13.197022 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:13.197015    6996 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf35d600-7eac-42ff-b1f9-79ad561fea3c-istiod-ca-cert\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:24:13.197022 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:13.197023    6996 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf35d600-7eac-42ff-b1f9-79ad561fea3c-istio-data\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:24:13.197349 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:13.197033    6996 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf35d600-7eac-42ff-b1f9-79ad561fea3c-workload-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:24:13.197349 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:13.197041    6996 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf35d600-7eac-42ff-b1f9-79ad561fea3c-credential-socket\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:24:13.944062 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:13.944026    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp"
Oct 30 20:24:13.944563 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:13.944030    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp" event={"ID":"cf35d600-7eac-42ff-b1f9-79ad561fea3c","Type":"ContainerDied","Data":"64fc659b35a5efe5a4097d5ad969dd34b8875b692baed9ab052ca851320943a2"}
Oct 30 20:24:13.944563 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:13.944186    6996 scope.go:122] "RemoveContainer" containerID="61e17e881b70d1f754c6a22b9c7fd84ea43951fd13fae2deb934ddf05866a1c0"
Oct 30 20:24:13.969640 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:13.969610    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp"]
Oct 30 20:24:13.973918 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:13.973879    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5677757b6d-m57tz_a8eded1a-5c8c-4740-8f7a-dfc5c05c0bcd/authorino/0.log"
Oct 30 20:24:13.974253 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:13.974232    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-8836a8-openshift-default-84f65665d5-mpltp"]
Oct 30 20:24:14.790491 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:14.790445    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cf35d600-7eac-42ff-b1f9-79ad561fea3c" path="/var/lib/kubelet/pods/cf35d600-7eac-42ff-b1f9-79ad561fea3c/volumes"
Oct 30 20:24:18.123987 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:18.123946    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lhxgc"]
Oct 30 20:24:18.124433 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:18.124134    6996 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lhxgc" podUID="95f15479-e511-4755-9380-a8f23af90ac6" containerName="discovery" containerID="cri-o://efefd1fc5b872df9ed6e084a2a092561fcf9347645ab4e37282a5a02ab43f92c" gracePeriod=30
Oct 30 20:24:18.382667 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:18.382608    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lhxgc"
Oct 30 20:24:18.454501 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:18.454451    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/95f15479-e511-4755-9380-a8f23af90ac6-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/95f15479-e511-4755-9380-a8f23af90ac6-local-certs\") pod \"95f15479-e511-4755-9380-a8f23af90ac6\" (UID: \"95f15479-e511-4755-9380-a8f23af90ac6\") "
Oct 30 20:24:18.454712 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:18.454522    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/95f15479-e511-4755-9380-a8f23af90ac6-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/95f15479-e511-4755-9380-a8f23af90ac6-istio-csr-dns-cert\") pod \"95f15479-e511-4755-9380-a8f23af90ac6\" (UID: \"95f15479-e511-4755-9380-a8f23af90ac6\") "
Oct 30 20:24:18.454712 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:18.454551    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/95f15479-e511-4755-9380-a8f23af90ac6-cacerts\" (UniqueName: \"kubernetes.io/secret/95f15479-e511-4755-9380-a8f23af90ac6-cacerts\") pod \"95f15479-e511-4755-9380-a8f23af90ac6\" (UID: \"95f15479-e511-4755-9380-a8f23af90ac6\") "
Oct 30 20:24:18.454712 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:18.454581    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/95f15479-e511-4755-9380-a8f23af90ac6-kube-api-access-4hnvc\" (UniqueName: \"kubernetes.io/projected/95f15479-e511-4755-9380-a8f23af90ac6-kube-api-access-4hnvc\") pod \"95f15479-e511-4755-9380-a8f23af90ac6\" (UID: \"95f15479-e511-4755-9380-a8f23af90ac6\") "
Oct 30 20:24:18.454712 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:18.454608    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/95f15479-e511-4755-9380-a8f23af90ac6-istio-token\" (UniqueName: \"kubernetes.io/projected/95f15479-e511-4755-9380-a8f23af90ac6-istio-token\") pod \"95f15479-e511-4755-9380-a8f23af90ac6\" (UID: \"95f15479-e511-4755-9380-a8f23af90ac6\") "
Oct 30 20:24:18.454712 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:18.454653    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/95f15479-e511-4755-9380-a8f23af90ac6-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/95f15479-e511-4755-9380-a8f23af90ac6-istio-csr-ca-configmap\") pod \"95f15479-e511-4755-9380-a8f23af90ac6\" (UID: \"95f15479-e511-4755-9380-a8f23af90ac6\") "
Oct 30 20:24:18.455017 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:18.454733    6996 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/95f15479-e511-4755-9380-a8f23af90ac6-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/95f15479-e511-4755-9380-a8f23af90ac6-istio-kubeconfig\") pod \"95f15479-e511-4755-9380-a8f23af90ac6\" (UID: \"95f15479-e511-4755-9380-a8f23af90ac6\") "
Oct 30 20:24:18.455109 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:18.455081    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95f15479-e511-4755-9380-a8f23af90ac6-istio-csr-ca-configmap" pod "95f15479-e511-4755-9380-a8f23af90ac6" (UID: "95f15479-e511-4755-9380-a8f23af90ac6"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:24:18.456939 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:18.456909    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95f15479-e511-4755-9380-a8f23af90ac6-istio-token" pod "95f15479-e511-4755-9380-a8f23af90ac6" (UID: "95f15479-e511-4755-9380-a8f23af90ac6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:24:18.457060 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:18.456953    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95f15479-e511-4755-9380-a8f23af90ac6-kube-api-access-4hnvc" pod "95f15479-e511-4755-9380-a8f23af90ac6" (UID: "95f15479-e511-4755-9380-a8f23af90ac6"). InnerVolumeSpecName "kube-api-access-4hnvc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:24:18.457283 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:18.457257    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95f15479-e511-4755-9380-a8f23af90ac6-istio-csr-dns-cert" pod "95f15479-e511-4755-9380-a8f23af90ac6" (UID: "95f15479-e511-4755-9380-a8f23af90ac6"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:24:18.457366 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:18.457300    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95f15479-e511-4755-9380-a8f23af90ac6-cacerts" pod "95f15479-e511-4755-9380-a8f23af90ac6" (UID: "95f15479-e511-4755-9380-a8f23af90ac6"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:24:18.457409 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:18.457362    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95f15479-e511-4755-9380-a8f23af90ac6-istio-kubeconfig" pod "95f15479-e511-4755-9380-a8f23af90ac6" (UID: "95f15479-e511-4755-9380-a8f23af90ac6"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:24:18.457409 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:18.457371    6996 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95f15479-e511-4755-9380-a8f23af90ac6-local-certs" pod "95f15479-e511-4755-9380-a8f23af90ac6" (UID: "95f15479-e511-4755-9380-a8f23af90ac6"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:24:18.556193 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:18.556154    6996 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/95f15479-e511-4755-9380-a8f23af90ac6-istio-kubeconfig\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:24:18.556193 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:18.556184    6996 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/95f15479-e511-4755-9380-a8f23af90ac6-local-certs\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:24:18.556193 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:18.556194    6996 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/95f15479-e511-4755-9380-a8f23af90ac6-istio-csr-dns-cert\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:24:18.556413 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:18.556203    6996 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/95f15479-e511-4755-9380-a8f23af90ac6-cacerts\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:24:18.556413 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:18.556213    6996 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4hnvc\" (UniqueName: \"kubernetes.io/projected/95f15479-e511-4755-9380-a8f23af90ac6-kube-api-access-4hnvc\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:24:18.556413 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:18.556221    6996 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95f15479-e511-4755-9380-a8f23af90ac6-istio-token\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:24:18.556413 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:18.556229    6996 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/95f15479-e511-4755-9380-a8f23af90ac6-istio-csr-ca-configmap\") on node \"ip-10-0-141-42.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:24:18.973649 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:18.973612    6996 generic.go:356] "Generic (PLEG): container finished" podID="95f15479-e511-4755-9380-a8f23af90ac6" containerID="efefd1fc5b872df9ed6e084a2a092561fcf9347645ab4e37282a5a02ab43f92c" exitCode=0
Oct 30 20:24:18.973845 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:18.973665    6996 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lhxgc"
Oct 30 20:24:18.973845 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:18.973695    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lhxgc" event={"ID":"95f15479-e511-4755-9380-a8f23af90ac6","Type":"ContainerDied","Data":"efefd1fc5b872df9ed6e084a2a092561fcf9347645ab4e37282a5a02ab43f92c"}
Oct 30 20:24:18.973845 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:18.973756    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lhxgc" event={"ID":"95f15479-e511-4755-9380-a8f23af90ac6","Type":"ContainerDied","Data":"92c67ce0553b18addf91a564687e1ba937f92145daae6dda61b01297f797539d"}
Oct 30 20:24:18.973845 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:18.973783    6996 scope.go:122] "RemoveContainer" containerID="efefd1fc5b872df9ed6e084a2a092561fcf9347645ab4e37282a5a02ab43f92c"
Oct 30 20:24:18.986426 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:18.986407    6996 scope.go:122] "RemoveContainer" containerID="efefd1fc5b872df9ed6e084a2a092561fcf9347645ab4e37282a5a02ab43f92c"
Oct 30 20:24:18.986706 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:24:18.986686    6996 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"efefd1fc5b872df9ed6e084a2a092561fcf9347645ab4e37282a5a02ab43f92c\": container with ID starting with efefd1fc5b872df9ed6e084a2a092561fcf9347645ab4e37282a5a02ab43f92c not found: ID does not exist" containerID="efefd1fc5b872df9ed6e084a2a092561fcf9347645ab4e37282a5a02ab43f92c"
Oct 30 20:24:18.986760 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:18.986716    6996 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"efefd1fc5b872df9ed6e084a2a092561fcf9347645ab4e37282a5a02ab43f92c"} err="failed to get container status \"efefd1fc5b872df9ed6e084a2a092561fcf9347645ab4e37282a5a02ab43f92c\": rpc error: code = NotFound desc = could not find container \"efefd1fc5b872df9ed6e084a2a092561fcf9347645ab4e37282a5a02ab43f92c\": container with ID starting with efefd1fc5b872df9ed6e084a2a092561fcf9347645ab4e37282a5a02ab43f92c not found: ID does not exist"
Oct 30 20:24:18.992661 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:18.992638    6996 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lhxgc"]
Oct 30 20:24:19.000763 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:19.000736    6996 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lhxgc"]
Oct 30 20:24:20.789961 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:20.789917    6996 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="95f15479-e511-4755-9380-a8f23af90ac6" path="/var/lib/kubelet/pods/95f15479-e511-4755-9380-a8f23af90ac6/volumes"
Oct 30 20:24:24.698421 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:24.698389    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-5c494664d6-7lvd2_50e907ce-5794-4533-9bda-d0e3abaf4e21/manager/0.log"
Oct 30 20:24:25.977409 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:25.977379    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-wwgvc_77b41da1-d9be-4723-886a-fd568313384b/server/0.log"
Oct 30 20:24:26.166396 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:26.166351    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-blshw_4e58facf-d3be-4f40-94b2-13cc119c8651/manager/0.log"
Oct 30 20:24:27.344543 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:27.344509    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-5cc5c6d754-r8xbw_9df8a8aa-54f9-43a0-b047-4b6e121259ed/maas-api/0.log"
Oct 30 20:24:30.529075 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:30.529040    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5677757b6d-m57tz_a8eded1a-5c8c-4740-8f7a-dfc5c05c0bcd/authorino/0.log"
Oct 30 20:24:31.139959 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:31.139934    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-gmgtn_bede9e81-8025-4db4-b21d-91f8c4b233e0/kuadrant-console-plugin/0.log"
Oct 30 20:24:31.189585 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:24:31.189550    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 20:24:31.961344 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:31.961315    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-2kq47_f1319006-f8c2-4f14-9d3d-e7d44737c884/limitador/0.log"
Oct 30 20:24:33.264574 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:33.264542    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b99845f6f-qb5pq_b2e83193-5313-47da-96aa-d3e85a5d4e29/kube-auth-proxy/0.log"
Oct 30 20:24:33.815567 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:33.815539    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5db77fbd9f-c2lhh_ac5ca902-49d8-4354-8c3d-57f0db27912b/istio-proxy/0.log"
Oct 30 20:24:34.196550 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:34.196442    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5db77fbd9f-sjbzh_b79c69dc-9d5b-4b3b-802e-ac414149df33/istio-proxy/0.log"
Oct 30 20:24:34.936404 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:34.936372    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-2j4r6_8879e7d6-4b91-4abf-9952-feb51aea83e6/payload-pre-processing/0.log"
Oct 30 20:24:35.115164 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:35.115137    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5f78687d5b-dkcrs_c72164f7-cc46-4f11-b1f0-f50adf9c4c0b/payload-processing/0.log"
Oct 30 20:24:36.067403 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:36.067373    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-cbd4f5586-k5q55_77c1aa9d-afba-4bb3-b583-d26d66592136/main/0.log"
Oct 30 20:24:36.424614 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:36.424535    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-67bf594cd6-6jh27_41ed96e2-0939-4c7b-9c32-ab8d4025c577/main/0.log"
Oct 30 20:24:36.600850 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:36.600821    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-69b588897d-jt9lh_b701a7d1-2f99-40d8-bc31-6f120f695f6b/main/0.log"
Oct 30 20:24:36.788220 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:36.788139    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zcm5m_bbb6c18e-0752-4ad7-a42a-ee5ffb73d58b/main/0.log"
Oct 30 20:24:51.830215 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:51.830135    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-526rb_141cf3eb-d99b-4876-b65a-a2847584ff5d/global-pull-secret-syncer/0.log"
Oct 30 20:24:52.257166 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:52.257084    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-bx5vk_17f17805-edc3-4dc5-8e75-819350666927/konnectivity-agent/0.log"
Oct 30 20:24:52.402288 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:24:52.402255    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-42.us-east-2.compute.internal_cb07204d0709daabd91966727e3665a5/haproxy/0.log"
Oct 30 20:25:00.256970 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:00.256932    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5677757b6d-m57tz_a8eded1a-5c8c-4740-8f7a-dfc5c05c0bcd/authorino/0.log"
Oct 30 20:25:00.408448 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:00.408410    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-gmgtn_bede9e81-8025-4db4-b21d-91f8c4b233e0/kuadrant-console-plugin/0.log"
Oct 30 20:25:00.990308 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:00.990280    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-2kq47_f1319006-f8c2-4f14-9d3d-e7d44737c884/limitador/0.log"
Oct 30 20:25:01.190444 ip-10-0-141-42 kubenswrapper[6996]: E1030 20:25:01.190396    6996 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qdkpz" containerName="extractor"
Oct 30 20:25:03.104800 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.104771    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5cb8677769-45w2n_136dc928-99f8-48d6-ae3b-7b7cf5d6bfeb/metrics-server/0.log"
Oct 30 20:25:03.166156 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.166128    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-77f4f76594-ph577_1cc9603f-1c61-43e4-b987-0e185cf26252/monitoring-plugin/0.log"
Oct 30 20:25:03.229464 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.229430    6996 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-4jlph/perf-node-gather-daemonset-p22hh"]
Oct 30 20:25:03.229859 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.229846    6996 cpu_manager.go:414] "containerMap: removing container" podUID="e538f71a-8196-4fad-a86b-4750900feb8c" containerName="maas-api"
Oct 30 20:25:03.229904 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.229861    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e538f71a-8196-4fad-a86b-4750900feb8c" containerName="maas-api"
Oct 30 20:25:03.229904 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.229871    6996 cpu_manager.go:414] "containerMap: removing container" podUID="d1d1de99-5578-4b8a-b94d-c8c4a095d760" containerName="maas-api"
Oct 30 20:25:03.229904 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.229877    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d1d1de99-5578-4b8a-b94d-c8c4a095d760" containerName="maas-api"
Oct 30 20:25:03.229904 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.229888    6996 cpu_manager.go:414] "containerMap: removing container" podUID="a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d" containerName="istio-proxy"
Oct 30 20:25:03.229904 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.229893    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d" containerName="istio-proxy"
Oct 30 20:25:03.229904 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.229902    6996 cpu_manager.go:414] "containerMap: removing container" podUID="91b87b65-f883-485d-abc4-69f270f6b3e9" containerName="maas-api"
Oct 30 20:25:03.229904 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.229907    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="91b87b65-f883-485d-abc4-69f270f6b3e9" containerName="maas-api"
Oct 30 20:25:03.230134 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.229915    6996 cpu_manager.go:414] "containerMap: removing container" podUID="a21bbe95-547d-41b0-b7c8-e1a296545947" containerName="istio-proxy"
Oct 30 20:25:03.230134 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.229920    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a21bbe95-547d-41b0-b7c8-e1a296545947" containerName="istio-proxy"
Oct 30 20:25:03.230134 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.229926    6996 cpu_manager.go:414] "containerMap: removing container" podUID="95f15479-e511-4755-9380-a8f23af90ac6" containerName="discovery"
Oct 30 20:25:03.230134 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.229931    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="95f15479-e511-4755-9380-a8f23af90ac6" containerName="discovery"
Oct 30 20:25:03.230134 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.229937    6996 cpu_manager.go:414] "containerMap: removing container" podUID="21af0b9f-44b8-43fc-bcda-751e196384ff" containerName="maas-api"
Oct 30 20:25:03.230134 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.229941    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="21af0b9f-44b8-43fc-bcda-751e196384ff" containerName="maas-api"
Oct 30 20:25:03.230134 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.229948    6996 cpu_manager.go:414] "containerMap: removing container" podUID="cf35d600-7eac-42ff-b1f9-79ad561fea3c" containerName="istio-proxy"
Oct 30 20:25:03.230134 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.229952    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cf35d600-7eac-42ff-b1f9-79ad561fea3c" containerName="istio-proxy"
Oct 30 20:25:03.230134 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.229961    6996 cpu_manager.go:414] "containerMap: removing container" podUID="babec534-131b-46e9-9706-cdf7fb2f60c8" containerName="maas-api"
Oct 30 20:25:03.230134 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.229965    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="babec534-131b-46e9-9706-cdf7fb2f60c8" containerName="maas-api"
Oct 30 20:25:03.230134 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.229972    6996 cpu_manager.go:414] "containerMap: removing container" podUID="9a76e670-c0a4-49ee-9b5f-7fabddaef0e9" containerName="istio-proxy"
Oct 30 20:25:03.230134 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.229976    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9a76e670-c0a4-49ee-9b5f-7fabddaef0e9" containerName="istio-proxy"
Oct 30 20:25:03.230134 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.229984    6996 cpu_manager.go:414] "containerMap: removing container" podUID="f6e147bc-ed53-4e12-b8f3-23a9e0ab12e2" containerName="payload-processing"
Oct 30 20:25:03.230134 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.229989    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f6e147bc-ed53-4e12-b8f3-23a9e0ab12e2" containerName="payload-processing"
Oct 30 20:25:03.230134 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.229997    6996 cpu_manager.go:414] "containerMap: removing container" podUID="a98a3fe9-bb1b-4a7f-af39-8486b0053f39" containerName="istio-proxy"
Oct 30 20:25:03.230134 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.230002    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a98a3fe9-bb1b-4a7f-af39-8486b0053f39" containerName="istio-proxy"
Oct 30 20:25:03.230134 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.230010    6996 cpu_manager.go:414] "containerMap: removing container" podUID="60867c87-cc28-4053-92d6-7f3f6024edc2" containerName="maas-api"
Oct 30 20:25:03.230134 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.230016    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="60867c87-cc28-4053-92d6-7f3f6024edc2" containerName="maas-api"
Oct 30 20:25:03.230134 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.230022    6996 cpu_manager.go:414] "containerMap: removing container" podUID="2eb68260-155c-4f10-84ec-0411e8fd0adb" containerName="main"
Oct 30 20:25:03.230134 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.230027    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2eb68260-155c-4f10-84ec-0411e8fd0adb" containerName="main"
Oct 30 20:25:03.230134 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.230034    6996 cpu_manager.go:414] "containerMap: removing container" podUID="bc6b5019-f6ed-44c3-8390-8525a8e75910" containerName="maas-api"
Oct 30 20:25:03.230134 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.230038    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bc6b5019-f6ed-44c3-8390-8525a8e75910" containerName="maas-api"
Oct 30 20:25:03.230134 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.230045    6996 cpu_manager.go:414] "containerMap: removing container" podUID="b55d9c75-1365-4bac-852e-326f4380cca7" containerName="istio-proxy"
Oct 30 20:25:03.230134 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.230049    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b55d9c75-1365-4bac-852e-326f4380cca7" containerName="istio-proxy"
Oct 30 20:25:03.230134 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.230055    6996 cpu_manager.go:414] "containerMap: removing container" podUID="24beb51d-3611-47ca-8061-b17dbfc16c80" containerName="maas-api"
Oct 30 20:25:03.230134 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.230060    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="24beb51d-3611-47ca-8061-b17dbfc16c80" containerName="maas-api"
Oct 30 20:25:03.230134 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.230069    6996 cpu_manager.go:414] "containerMap: removing container" podUID="ae905dfb-f30d-480a-899b-ea676abc1d8d" containerName="main"
Oct 30 20:25:03.230134 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.230074    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ae905dfb-f30d-480a-899b-ea676abc1d8d" containerName="main"
Oct 30 20:25:03.230134 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.230082    6996 cpu_manager.go:414] "containerMap: removing container" podUID="316882f2-58f3-404a-a515-310efc6f2462" containerName="payload-pre-processing"
Oct 30 20:25:03.230134 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.230087    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="316882f2-58f3-404a-a515-310efc6f2462" containerName="payload-pre-processing"
Oct 30 20:25:03.230134 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.230096    6996 cpu_manager.go:414] "containerMap: removing container" podUID="a21bbe95-547d-41b0-b7c8-e1a296545947" containerName="istio-proxy"
Oct 30 20:25:03.230134 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.230100    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a21bbe95-547d-41b0-b7c8-e1a296545947" containerName="istio-proxy"
Oct 30 20:25:03.230134 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.230110    6996 cpu_manager.go:414] "containerMap: removing container" podUID="035ed3f1-789b-4a62-80e6-3f8841905629" containerName="maas-api"
Oct 30 20:25:03.230134 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.230115    6996 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="035ed3f1-789b-4a62-80e6-3f8841905629" containerName="maas-api"
Oct 30 20:25:03.231287 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.230172    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="f6e147bc-ed53-4e12-b8f3-23a9e0ab12e2" containerName="payload-processing"
Oct 30 20:25:03.231287 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.230181    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="a98a3fe9-bb1b-4a7f-af39-8486b0053f39" containerName="istio-proxy"
Oct 30 20:25:03.231287 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.230187    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="60867c87-cc28-4053-92d6-7f3f6024edc2" containerName="maas-api"
Oct 30 20:25:03.231287 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.230193    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="2eb68260-155c-4f10-84ec-0411e8fd0adb" containerName="main"
Oct 30 20:25:03.231287 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.230198    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="bc6b5019-f6ed-44c3-8390-8525a8e75910" containerName="maas-api"
Oct 30 20:25:03.231287 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.230205    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="b55d9c75-1365-4bac-852e-326f4380cca7" containerName="istio-proxy"
Oct 30 20:25:03.231287 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.230210    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="24beb51d-3611-47ca-8061-b17dbfc16c80" containerName="maas-api"
Oct 30 20:25:03.231287 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.230216    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="ae905dfb-f30d-480a-899b-ea676abc1d8d" containerName="main"
Oct 30 20:25:03.231287 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.230221    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="316882f2-58f3-404a-a515-310efc6f2462" containerName="payload-pre-processing"
Oct 30 20:25:03.231287 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.230227    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="a21bbe95-547d-41b0-b7c8-e1a296545947" containerName="istio-proxy"
Oct 30 20:25:03.231287 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.230232    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="035ed3f1-789b-4a62-80e6-3f8841905629" containerName="maas-api"
Oct 30 20:25:03.231287 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.230239    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="e538f71a-8196-4fad-a86b-4750900feb8c" containerName="maas-api"
Oct 30 20:25:03.231287 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.230244    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="d1d1de99-5578-4b8a-b94d-c8c4a095d760" containerName="maas-api"
Oct 30 20:25:03.231287 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.230248    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="a1c3b8a0-7dde-4d9c-97e2-8ca9a326a31d" containerName="istio-proxy"
Oct 30 20:25:03.231287 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.230254    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="91b87b65-f883-485d-abc4-69f270f6b3e9" containerName="maas-api"
Oct 30 20:25:03.231287 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.230259    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="a21bbe95-547d-41b0-b7c8-e1a296545947" containerName="istio-proxy"
Oct 30 20:25:03.231287 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.230264    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="95f15479-e511-4755-9380-a8f23af90ac6" containerName="discovery"
Oct 30 20:25:03.231287 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.230270    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="21af0b9f-44b8-43fc-bcda-751e196384ff" containerName="maas-api"
Oct 30 20:25:03.231287 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.230276    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="cf35d600-7eac-42ff-b1f9-79ad561fea3c" containerName="istio-proxy"
Oct 30 20:25:03.231287 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.230282    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="babec534-131b-46e9-9706-cdf7fb2f60c8" containerName="maas-api"
Oct 30 20:25:03.231287 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.230286    6996 memory_manager.go:365] "RemoveStaleState removing state" podUID="9a76e670-c0a4-49ee-9b5f-7fabddaef0e9" containerName="istio-proxy"
Oct 30 20:25:03.233931 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.233912    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4jlph/perf-node-gather-daemonset-p22hh"
Oct 30 20:25:03.238103 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.238082    6996 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-4jlph\"/\"default-dockercfg-q7rn9\""
Oct 30 20:25:03.241665 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.241631    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4jlph\"/\"kube-root-ca.crt\""
Oct 30 20:25:03.241771 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.241722    6996 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4jlph\"/\"openshift-service-ca.crt\""
Oct 30 20:25:03.242923 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.242902    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4jlph/perf-node-gather-daemonset-p22hh"]
Oct 30 20:25:03.397511 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.397393    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/63425e3a-4a4b-4793-9cae-2745ae7c6e36-lib-modules\") pod \"perf-node-gather-daemonset-p22hh\" (UID: \"63425e3a-4a4b-4793-9cae-2745ae7c6e36\") " pod="openshift-must-gather-4jlph/perf-node-gather-daemonset-p22hh"
Oct 30 20:25:03.397511 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.397433    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/63425e3a-4a4b-4793-9cae-2745ae7c6e36-podres\") pod \"perf-node-gather-daemonset-p22hh\" (UID: \"63425e3a-4a4b-4793-9cae-2745ae7c6e36\") " pod="openshift-must-gather-4jlph/perf-node-gather-daemonset-p22hh"
Oct 30 20:25:03.397740 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.397548    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/63425e3a-4a4b-4793-9cae-2745ae7c6e36-proc\") pod \"perf-node-gather-daemonset-p22hh\" (UID: \"63425e3a-4a4b-4793-9cae-2745ae7c6e36\") " pod="openshift-must-gather-4jlph/perf-node-gather-daemonset-p22hh"
Oct 30 20:25:03.397740 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.397691    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znqzv\" (UniqueName: \"kubernetes.io/projected/63425e3a-4a4b-4793-9cae-2745ae7c6e36-kube-api-access-znqzv\") pod \"perf-node-gather-daemonset-p22hh\" (UID: \"63425e3a-4a4b-4793-9cae-2745ae7c6e36\") " pod="openshift-must-gather-4jlph/perf-node-gather-daemonset-p22hh"
Oct 30 20:25:03.397869 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.397739    6996 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/63425e3a-4a4b-4793-9cae-2745ae7c6e36-sys\") pod \"perf-node-gather-daemonset-p22hh\" (UID: \"63425e3a-4a4b-4793-9cae-2745ae7c6e36\") " pod="openshift-must-gather-4jlph/perf-node-gather-daemonset-p22hh"
Oct 30 20:25:03.463654 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.463628    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sx8t4_0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd/node-exporter/0.log"
Oct 30 20:25:03.498032 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.498003    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sx8t4_0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd/kube-rbac-proxy/0.log"
Oct 30 20:25:03.498267 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.498245    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/63425e3a-4a4b-4793-9cae-2745ae7c6e36-lib-modules\") pod \"perf-node-gather-daemonset-p22hh\" (UID: \"63425e3a-4a4b-4793-9cae-2745ae7c6e36\") " pod="openshift-must-gather-4jlph/perf-node-gather-daemonset-p22hh"
Oct 30 20:25:03.498324 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.498274    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/63425e3a-4a4b-4793-9cae-2745ae7c6e36-podres\") pod \"perf-node-gather-daemonset-p22hh\" (UID: \"63425e3a-4a4b-4793-9cae-2745ae7c6e36\") " pod="openshift-must-gather-4jlph/perf-node-gather-daemonset-p22hh"
Oct 30 20:25:03.498324 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.498299    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/63425e3a-4a4b-4793-9cae-2745ae7c6e36-proc\") pod \"perf-node-gather-daemonset-p22hh\" (UID: \"63425e3a-4a4b-4793-9cae-2745ae7c6e36\") " pod="openshift-must-gather-4jlph/perf-node-gather-daemonset-p22hh"
Oct 30 20:25:03.498397 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.498337    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znqzv\" (UniqueName: \"kubernetes.io/projected/63425e3a-4a4b-4793-9cae-2745ae7c6e36-kube-api-access-znqzv\") pod \"perf-node-gather-daemonset-p22hh\" (UID: \"63425e3a-4a4b-4793-9cae-2745ae7c6e36\") " pod="openshift-must-gather-4jlph/perf-node-gather-daemonset-p22hh"
Oct 30 20:25:03.498397 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.498361    6996 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/63425e3a-4a4b-4793-9cae-2745ae7c6e36-sys\") pod \"perf-node-gather-daemonset-p22hh\" (UID: \"63425e3a-4a4b-4793-9cae-2745ae7c6e36\") " pod="openshift-must-gather-4jlph/perf-node-gather-daemonset-p22hh"
Oct 30 20:25:03.498397 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.498377    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/63425e3a-4a4b-4793-9cae-2745ae7c6e36-podres\") pod \"perf-node-gather-daemonset-p22hh\" (UID: \"63425e3a-4a4b-4793-9cae-2745ae7c6e36\") " pod="openshift-must-gather-4jlph/perf-node-gather-daemonset-p22hh"
Oct 30 20:25:03.498517 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.498431    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/63425e3a-4a4b-4793-9cae-2745ae7c6e36-proc\") pod \"perf-node-gather-daemonset-p22hh\" (UID: \"63425e3a-4a4b-4793-9cae-2745ae7c6e36\") " pod="openshift-must-gather-4jlph/perf-node-gather-daemonset-p22hh"
Oct 30 20:25:03.498517 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.498435    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/63425e3a-4a4b-4793-9cae-2745ae7c6e36-sys\") pod \"perf-node-gather-daemonset-p22hh\" (UID: \"63425e3a-4a4b-4793-9cae-2745ae7c6e36\") " pod="openshift-must-gather-4jlph/perf-node-gather-daemonset-p22hh"
Oct 30 20:25:03.498517 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.498377    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/63425e3a-4a4b-4793-9cae-2745ae7c6e36-lib-modules\") pod \"perf-node-gather-daemonset-p22hh\" (UID: \"63425e3a-4a4b-4793-9cae-2745ae7c6e36\") " pod="openshift-must-gather-4jlph/perf-node-gather-daemonset-p22hh"
Oct 30 20:25:03.511095 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.511072    6996 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-znqzv\" (UniqueName: \"kubernetes.io/projected/63425e3a-4a4b-4793-9cae-2745ae7c6e36-kube-api-access-znqzv\") pod \"perf-node-gather-daemonset-p22hh\" (UID: \"63425e3a-4a4b-4793-9cae-2745ae7c6e36\") " pod="openshift-must-gather-4jlph/perf-node-gather-daemonset-p22hh"
Oct 30 20:25:03.532892 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.532872    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sx8t4_0b3b1fc9-d34f-46dd-a354-ef07ab2b37fd/init-textfile/0.log"
Oct 30 20:25:03.548409 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.548394    6996 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4jlph/perf-node-gather-daemonset-p22hh"
Oct 30 20:25:03.576536 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.576513    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-nq89t_8cde7f30-1d5b-4bea-ad21-2264b567ca1d/kube-rbac-proxy-main/0.log"
Oct 30 20:25:03.612215 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.612156    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-nq89t_8cde7f30-1d5b-4bea-ad21-2264b567ca1d/kube-rbac-proxy-self/0.log"
Oct 30 20:25:03.647162 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.647137    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-nq89t_8cde7f30-1d5b-4bea-ad21-2264b567ca1d/openshift-state-metrics/0.log"
Oct 30 20:25:03.702082 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.702021    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c1e7407b-e78a-4b5c-b34f-44f5a24bc56f/prometheus/0.log"
Oct 30 20:25:03.742810 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.742784    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c1e7407b-e78a-4b5c-b34f-44f5a24bc56f/config-reloader/0.log"
Oct 30 20:25:03.777950 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.777927    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c1e7407b-e78a-4b5c-b34f-44f5a24bc56f/thanos-sidecar/0.log"
Oct 30 20:25:03.815143 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.815123    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c1e7407b-e78a-4b5c-b34f-44f5a24bc56f/kube-rbac-proxy-web/0.log"
Oct 30 20:25:03.826205 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.826183    6996 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4jlph/perf-node-gather-daemonset-p22hh"]
Oct 30 20:25:03.829230 ip-10-0-141-42 kubenswrapper[6996]: W1030 20:25:03.829204    6996 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod63425e3a_4a4b_4793_9cae_2745ae7c6e36.slice/crio-608185f721898e5af86d188e897b08bcf3b52acc131fdd7549dce9a3b751b14e WatchSource:0}: Error finding container 608185f721898e5af86d188e897b08bcf3b52acc131fdd7549dce9a3b751b14e: Status 404 returned error can't find the container with id 608185f721898e5af86d188e897b08bcf3b52acc131fdd7549dce9a3b751b14e
Oct 30 20:25:03.859028 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.859008    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c1e7407b-e78a-4b5c-b34f-44f5a24bc56f/kube-rbac-proxy/0.log"
Oct 30 20:25:03.895743 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.895724    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c1e7407b-e78a-4b5c-b34f-44f5a24bc56f/kube-rbac-proxy-thanos/0.log"
Oct 30 20:25:03.930129 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:03.930110    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c1e7407b-e78a-4b5c-b34f-44f5a24bc56f/init-config-reloader/0.log"
Oct 30 20:25:04.080268 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:04.080243    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6d56c5d745-9xkj5_bfd79074-21c8-4caf-8c63-a2c5ff45e26d/telemeter-client/0.log"
Oct 30 20:25:04.115317 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:04.115296    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6d56c5d745-9xkj5_bfd79074-21c8-4caf-8c63-a2c5ff45e26d/reload/0.log"
Oct 30 20:25:04.150404 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:04.150383    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6d56c5d745-9xkj5_bfd79074-21c8-4caf-8c63-a2c5ff45e26d/kube-rbac-proxy/0.log"
Oct 30 20:25:04.195979 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:04.195955    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-996bc4dbd-wr65k_94ecba26-0809-414d-b084-8de0a693795e/thanos-query/0.log"
Oct 30 20:25:04.215491 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:04.215454    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4jlph/perf-node-gather-daemonset-p22hh" event={"ID":"63425e3a-4a4b-4793-9cae-2745ae7c6e36","Type":"ContainerStarted","Data":"1c5209b01f214eb025f161a80f9c98954b5cbe554c0627fda63f6bf3d3c733ce"}
Oct 30 20:25:04.215605 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:04.215515    6996 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4jlph/perf-node-gather-daemonset-p22hh" event={"ID":"63425e3a-4a4b-4793-9cae-2745ae7c6e36","Type":"ContainerStarted","Data":"608185f721898e5af86d188e897b08bcf3b52acc131fdd7549dce9a3b751b14e"}
Oct 30 20:25:04.215660 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:04.215616    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-4jlph/perf-node-gather-daemonset-p22hh"
Oct 30 20:25:04.229277 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:04.229259    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-996bc4dbd-wr65k_94ecba26-0809-414d-b084-8de0a693795e/kube-rbac-proxy-web/0.log"
Oct 30 20:25:04.234838 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:04.234798    6996 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-4jlph/perf-node-gather-daemonset-p22hh" podStartSLOduration=1.234787467 podStartE2EDuration="1.234787467s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 20:25:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:25:04.232525701 +0000 UTC m=+3179.560643856" watchObservedRunningTime="2026-10-30 20:25:04.234787467 +0000 UTC m=+3179.562905621"
Oct 30 20:25:04.262683 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:04.262664    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-996bc4dbd-wr65k_94ecba26-0809-414d-b084-8de0a693795e/kube-rbac-proxy/0.log"
Oct 30 20:25:04.315520 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:04.315491    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-996bc4dbd-wr65k_94ecba26-0809-414d-b084-8de0a693795e/prom-label-proxy/0.log"
Oct 30 20:25:04.355916 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:04.355859    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-996bc4dbd-wr65k_94ecba26-0809-414d-b084-8de0a693795e/kube-rbac-proxy-rules/0.log"
Oct 30 20:25:04.393495 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:04.393454    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-996bc4dbd-wr65k_94ecba26-0809-414d-b084-8de0a693795e/kube-rbac-proxy-metrics/0.log"
Oct 30 20:25:07.714977 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:07.714940    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-5c494664d6-7lvd2_50e907ce-5794-4533-9bda-d0e3abaf4e21/manager/0.log"
Oct 30 20:25:07.766355 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:07.766323    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5cf7f796c9-hsqwz_3a5d313c-b78d-4bba-aeae-218fa0cb2e85/console/0.log"
Oct 30 20:25:07.918847 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:07.918815    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-5c494664d6-7lvd2_50e907ce-5794-4533-9bda-d0e3abaf4e21/manager/0.log"
Oct 30 20:25:09.508534 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:09.508508    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-wwgvc_77b41da1-d9be-4723-886a-fd568313384b/server/0.log"
Oct 30 20:25:09.870400 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:09.870369    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-wwgvc_77b41da1-d9be-4723-886a-fd568313384b/server/0.log"
Oct 30 20:25:10.233630 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:10.233552    6996 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-4jlph/perf-node-gather-daemonset-p22hh"
Oct 30 20:25:10.696218 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:10.696184    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-blshw_4e58facf-d3be-4f40-94b2-13cc119c8651/manager/0.log"
Oct 30 20:25:10.974431 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:10.974347    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-blshw_4e58facf-d3be-4f40-94b2-13cc119c8651/manager/0.log"
Oct 30 20:25:11.028161 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:11.028134    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-8dn55_7bb33e32-f8ac-44a2-95a4-cb949b08710f/dns/0.log"
Oct 30 20:25:11.063291 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:11.063264    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-8dn55_7bb33e32-f8ac-44a2-95a4-cb949b08710f/kube-rbac-proxy/0.log"
Oct 30 20:25:11.253319 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:11.253233    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-859jf_1977f3d7-c7f6-44cc-8a59-9d4f35adc0f7/dns-node-resolver/0.log"
Oct 30 20:25:12.130375 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:12.130343    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-5c494664d6-7lvd2_50e907ce-5794-4533-9bda-d0e3abaf4e21/manager/0.log"
Oct 30 20:25:12.415251 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:12.415156    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-5c494664d6-7lvd2_50e907ce-5794-4533-9bda-d0e3abaf4e21/manager/0.log"
Oct 30 20:25:12.588213 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:12.588185    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-j4cdl_6eca1a1e-101a-43bb-9fde-505c3182c10a/node-ca/0.log"
Oct 30 20:25:14.035360 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:14.035324    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-wwgvc_77b41da1-d9be-4723-886a-fd568313384b/server/0.log"
Oct 30 20:25:14.354640 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:14.354613    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-wwgvc_77b41da1-d9be-4723-886a-fd568313384b/server/0.log"
Oct 30 20:25:15.219291 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:15.219247    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-blshw_4e58facf-d3be-4f40-94b2-13cc119c8651/manager/0.log"
Oct 30 20:25:15.512959 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:15.512869    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-blshw_4e58facf-d3be-4f40-94b2-13cc119c8651/manager/0.log"
Oct 30 20:25:16.809784 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:16.809748    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-5c494664d6-7lvd2_50e907ce-5794-4533-9bda-d0e3abaf4e21/manager/0.log"
Oct 30 20:25:16.959364 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:16.959311    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-5c494664d6-7lvd2_50e907ce-5794-4533-9bda-d0e3abaf4e21/manager/0.log"
Oct 30 20:25:18.062957 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:18.062920    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b99845f6f-qb5pq_b2e83193-5313-47da-96aa-d3e85a5d4e29/kube-auth-proxy/0.log"
Oct 30 20:25:18.228242 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:18.228202    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5db77fbd9f-c2lhh_ac5ca902-49d8-4354-8c3d-57f0db27912b/istio-proxy/0.log"
Oct 30 20:25:18.320359 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:18.320325    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5db77fbd9f-sjbzh_b79c69dc-9d5b-4b3b-802e-ac414149df33/istio-proxy/0.log"
Oct 30 20:25:18.482687 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:18.482652    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-2j4r6_8879e7d6-4b91-4abf-9952-feb51aea83e6/payload-pre-processing/0.log"
Oct 30 20:25:18.523356 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:18.523249    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5f78687d5b-dkcrs_c72164f7-cc46-4f11-b1f0-f50adf9c4c0b/payload-processing/0.log"
Oct 30 20:25:18.587682 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:18.587589    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-wwgvc_77b41da1-d9be-4723-886a-fd568313384b/server/0.log"
Oct 30 20:25:18.930127 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:18.930014    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-wwgvc_77b41da1-d9be-4723-886a-fd568313384b/server/0.log"
Oct 30 20:25:19.798287 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:19.798245    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-blshw_4e58facf-d3be-4f40-94b2-13cc119c8651/manager/0.log"
Oct 30 20:25:19.994167 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:19.994118    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b99845f6f-qb5pq_b2e83193-5313-47da-96aa-d3e85a5d4e29/kube-auth-proxy/0.log"
Oct 30 20:25:20.077113 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:20.077081    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-blshw_4e58facf-d3be-4f40-94b2-13cc119c8651/manager/0.log"
Oct 30 20:25:20.084319 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:20.084291    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5db77fbd9f-c2lhh_ac5ca902-49d8-4354-8c3d-57f0db27912b/istio-proxy/0.log"
Oct 30 20:25:20.132125 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:20.132086    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5db77fbd9f-sjbzh_b79c69dc-9d5b-4b3b-802e-ac414149df33/istio-proxy/0.log"
Oct 30 20:25:20.239841 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:20.239807    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-2j4r6_8879e7d6-4b91-4abf-9952-feb51aea83e6/payload-pre-processing/0.log"
Oct 30 20:25:20.264916 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:20.264885    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5f78687d5b-dkcrs_c72164f7-cc46-4f11-b1f0-f50adf9c4c0b/payload-processing/0.log"
Oct 30 20:25:21.731476 ip-10-0-141-42 kubenswrapper[6996]: I1030 20:25:21.731442    6996 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-8hjtk_517d6b00-0305-40cc-95c3-bba4adb2f849/serve-healthcheck-canary/0.log"